Hi all, I've been trying to make a series of plots from the data dumps of an Enzo simulation, but am encountering problems when yt attempts to make slices. At first I thought it was due to some error in the recursion, but after making a much more direct script, it appears to come up as soon as slices are to be created. For what it's worth, projections are made without any problem, it's the slices that are giving me trouble. The output file is attached. Thanks, Brian
Hi Brian, You probably don't need to slice in parallel -- slices are very memory efficient, compared to things like projections. Parallel slices are typically only needed for in-situ analysis. But, that being said, they should still work. I actually made a change this morning that backported a bug with the data files to yt-1.7 Could you try "yt instinfo -u" and run a parallel slice (no halo finding necessary) script again? If that does not work, please let me know, and include the script if possible? Thanks! -Matt On Wed, Jun 30, 2010 at 10:36 AM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Brian, In this particular example, you need to set up the PlotCollection -- pc = PlotCollection(pf, center=[0.5,0.5,0.5]) for instance. If you give that a shot and run it again, can you send me the output/script that gets you to the error you had previously? -Matt On Wed, Jun 30, 2010 at 1:11 PM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Brian, Apologies for the delay (the Enzo Workshop has been occupying a bit of my time :) but I think I understand the problem that is going on, and I have a few ideas for solutions. The problem is that there isn't enough work for all the processors; the simulation is not big enough to warrant the number of processors you are using to slice through it. However, it should not fail in this manner if that's the case, so we should still fix it! As it stands, this file: /share/home/01450/bcrosby/yt-x86_64/src/yt-1.7-svn/yt/lagos/BaseDataTypes.py on line 592 should read: if len(data) == 0: data = None What's happening is that this line is executed when the processor has no grids assigned to it, and thus no data to share with the other processors. This then gets passed into the joining routine, which makes the assumption that it's receiving an array. I believe if you change this to read: if len(data) == 0: data = na.array([]) then I believe it will work. (I could be wrong.) I have been unable to duplicate this problem locally, so let me know if this works and I'll commit the fix! Thanks, Matt On Thu, Jul 1, 2010 at 1:38 PM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Matt, Thanks for getting back to me. I tried putting in that changes and running them, but no luck. I tried not running in parallel as well, and it still didn't work, so there's a decent chance that the problem is in my script rather than YT. What's very curious that I ran the same script on a different data set to test it (in /scratch/01450/bcrosby/little_cluster) and that ran successfully when not run in parallel, but crashed when I tried to run it in parallel. The output file and script are attached. Does it look like that changed got us any closer to a solution? Thanks, Brian On Jul 4, 2010, at 3:28 AM, Matthew Turk wrote:
Hi Brian, (We had some off-list discussion.) I was eventually able to reproduce your error with the data you sent me, but the fix I provided actually worked for me. I've committed it in revision 1794 to trunk and 1795 to 1.7, so if you run "yt instinfo -u" and it is able to auto-update your installation, you should get that fix -- there might have just been something odd and subtle about the way I asked you to edit the file, so give that a go and let me know if it works. -Matt On Mon, Jul 5, 2010 at 8:21 AM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Brian, You probably don't need to slice in parallel -- slices are very memory efficient, compared to things like projections. Parallel slices are typically only needed for in-situ analysis. But, that being said, they should still work. I actually made a change this morning that backported a bug with the data files to yt-1.7 Could you try "yt instinfo -u" and run a parallel slice (no halo finding necessary) script again? If that does not work, please let me know, and include the script if possible? Thanks! -Matt On Wed, Jun 30, 2010 at 10:36 AM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Brian, In this particular example, you need to set up the PlotCollection -- pc = PlotCollection(pf, center=[0.5,0.5,0.5]) for instance. If you give that a shot and run it again, can you send me the output/script that gets you to the error you had previously? -Matt On Wed, Jun 30, 2010 at 1:11 PM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Brian, Apologies for the delay (the Enzo Workshop has been occupying a bit of my time :) but I think I understand the problem that is going on, and I have a few ideas for solutions. The problem is that there isn't enough work for all the processors; the simulation is not big enough to warrant the number of processors you are using to slice through it. However, it should not fail in this manner if that's the case, so we should still fix it! As it stands, this file: /share/home/01450/bcrosby/yt-x86_64/src/yt-1.7-svn/yt/lagos/BaseDataTypes.py on line 592 should read: if len(data) == 0: data = None What's happening is that this line is executed when the processor has no grids assigned to it, and thus no data to share with the other processors. This then gets passed into the joining routine, which makes the assumption that it's receiving an array. I believe if you change this to read: if len(data) == 0: data = na.array([]) then I believe it will work. (I could be wrong.) I have been unable to duplicate this problem locally, so let me know if this works and I'll commit the fix! Thanks, Matt On Thu, Jul 1, 2010 at 1:38 PM, Brian Crosby <crosby.bd@gmail.com> wrote:
Hi Matt, Thanks for getting back to me. I tried putting in that changes and running them, but no luck. I tried not running in parallel as well, and it still didn't work, so there's a decent chance that the problem is in my script rather than YT. What's very curious that I ran the same script on a different data set to test it (in /scratch/01450/bcrosby/little_cluster) and that ran successfully when not run in parallel, but crashed when I tried to run it in parallel. The output file and script are attached. Does it look like that changed got us any closer to a solution? Thanks, Brian On Jul 4, 2010, at 3:28 AM, Matthew Turk wrote:
Hi Brian, (We had some off-list discussion.) I was eventually able to reproduce your error with the data you sent me, but the fix I provided actually worked for me. I've committed it in revision 1794 to trunk and 1795 to 1.7, so if you run "yt instinfo -u" and it is able to auto-update your installation, you should get that fix -- there might have just been something odd and subtle about the way I asked you to edit the file, so give that a go and let me know if it works. -Matt On Mon, Jul 5, 2010 at 8:21 AM, Brian Crosby <crosby.bd@gmail.com> wrote:
participants (2)
-
Brian Crosby -
Matthew Turk