Hi Stephen, The parallel proxy class works so that it does all the barriers you need. In the second script, what is happening is that proc 0 is inside the function parallel_simple_proxy: http://yt.enzotools.org/browser/trunk/yt/lagos/ParallelTools.py#L114 because its metaclass is ParallelDummy: http://yt.enzotools.org/browser/trunk/yt/lagos/ParallelTools.py#L130 These functions are not meant to be manually parallelized; this is a flaw concealed inside a bonus. If you want to try it again, you might consider manually setting (and then unsetting) the _processing attribute of your HopGroup to True; this should set it to turn off the locking/barrierizing. Keep in mind, though, that this means that all of your lists of halo attributes will be processor specific and not shared amongst the processors. Locking of HDF5 files should also be considered, so you'll need to handle that somehow as well, to ensure you either have only one processor writing to your HDF5 files at a time, or to use whatever (if any) parallel IO support PyTables has. -Matt On Fri, Feb 13, 2009 at 9:43 AM, Stephen Skory <stephenskory@yahoo.com> wrote:
Matt,
I have a script that runs parallel hop, and then writes out the results to an HDF5 file. I'd like to more efficiently parallelize a certain part of the process, where I retrieve and store the particle particulars, but it doesn't seem to work the way I'd like. This first example works, where I get the various values on all processors, and then store it only on the one that owns the halo:
http://paste.enzotools.org/show/43/
But this one doesn't work, where I only retrieve the values if the proc owns the halo. And by not working, I mean it hangs on halo 0 and doesn't finish after 10 minutes, while the script above takes less than a minute:
http://paste.enzotools.org/show/44/
Can you give me any ideas? In particular, the part of the script where it retrieves the particle_indexes is going serially in the functioning script (I can watch it in the output log) and it's taking far longer than it should. Thanks!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org