Hi all, I was calculating some quantities for a halo list with 32 processors, and it broke when it tried to calculate derived quantities for small halos with less than 32 grids within its sphere. I fixed it for enzo datasets and posted the diff here. http://paste.enzotools.org/show/1511/ First it broke in the preload function when the grid list is null, and it tried to get a field list from the first grid. That was easily fixed by returning data = None. Then I had to modify the mpi_catarray function to consider a null dataset (len(data) == 0). I don't like how I modified the preload function because it only applies for the enzo frontend. Is there another routine that can be modified that affects all data formats, so this fix can be generalized? Thanks, John
Hi John, On Sun, Feb 20, 2011 at 5:22 PM, John Wise <jwise@astro.princeton.edu> wrote:
Hi all,
I was calculating some quantities for a halo list with 32 processors, and it broke when it tried to calculate derived quantities for small halos with less than 32 grids within its sphere. I fixed it for enzo datasets and posted the diff here.
http://paste.enzotools.org/show/1511/
First it broke in the preload function when the grid list is null, and it tried to get a field list from the first grid. That was easily fixed by returning data = None. Then I had to modify the mpi_catarray function to consider a null dataset (len(data) == 0).
The mpi_catarray function has caused some problems lately. We had to change a few things around to get it to work with inline in a particular way, but the specifics escape me now. I think that it was this change that broke the ngrids<ncpu issue that you bring up; I'm sorry it caused you an issue! Your fix looks fine to me.
I don't like how I modified the preload function because it only applies for the enzo frontend. Is there another routine that can be modified that affects all data formats, so this fix can be generalized?
Nope! As of right now, Enzo's the only code that implements preload. I suspect that any fix would have to be applied specific to each code, but my gut feeling is that the only reason it came up in Enzo is because of carelessness on my part; for some reason the IO Handler instance doesn't know about its parent parameter file or hierarchy. I think that a longer-term fix would be to avoid this ignorance, and simply give unto the IOHandler a reference to the parameter file. I think that if we removed the grids[0] call, otherwise preload() should succeed. Your fix is great, thought. I'd say push these changes, they look good. Thanks for the catch on these bugs! -Matt
Thanks, John _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
The mpi_catarray function has caused some problems lately. We had to change a few things around to get it to work with inline in a particular way, but the specifics escape me now. I think that it was this change that broke the ngrids<ncpu issue that you bring up; I'm sorry it caused you an issue! Your fix looks fine to me.
Thanks for letting me know how this bug possibly arose. I'll go ahead and push the change.
Nope! As of right now, Enzo's the only code that implements preload. I suspect that any fix would have to be applied specific to each code, but my gut feeling is that the only reason it came up in Enzo is because of carelessness on my part; for some reason the IO Handler instance doesn't know about its parent parameter file or hierarchy. I think that a longer-term fix would be to avoid this ignorance, and simply give unto the IOHandler a reference to the parameter file. I think that if we removed the grids[0] call, otherwise preload() should succeed. Your fix is great, thought.
Good to know. I just didn't want to push a change that could've been extended to other formats :) Thanks again, John
The changes to the mpi_catarray were made to fix issues that came up from doing slices in parallel where some processors contained no grids of interest. As long as that still works, there shouldn't be a problem. Britton On Sun, Feb 20, 2011 at 6:09 PM, John Wise <jwise@astro.princeton.edu>wrote:
The mpi_catarray function has caused some problems lately. We had to
change a few things around to get it to work with inline in a particular way, but the specifics escape me now. I think that it was this change that broke the ngrids<ncpu issue that you bring up; I'm sorry it caused you an issue! Your fix looks fine to me.
Thanks for letting me know how this bug possibly arose. I'll go ahead and push the change.
Nope! As of right now, Enzo's the only code that implements preload.
I suspect that any fix would have to be applied specific to each code, but my gut feeling is that the only reason it came up in Enzo is because of carelessness on my part; for some reason the IO Handler instance doesn't know about its parent parameter file or hierarchy. I think that a longer-term fix would be to avoid this ignorance, and simply give unto the IOHandler a reference to the parameter file. I think that if we removed the grids[0] call, otherwise preload() should succeed. Your fix is great, thought.
Good to know. I just didn't want to push a change that could've been extended to other formats :)
Thanks again,
John _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Britton Smith
-
John Wise
-
Matthew Turk