Matt, is the logic correct on lines 138-139 in your diff? The way you wrote it is giving me a syntax error. Can you take another look? http://paste.enzotools.org/show/46/ _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Matt, That doesn't work either: http://paste.enzotools.org/show/47/ do you want "if attrname not in extra or attrname.startswith("_") \"? I tried that and got a different, but more encouraging error: http://paste.enzotools.org/show/48/ _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Matt,

When I made reference to removing the getter methods, I did not mean getter methods that are not for particle fields. get_size does not refer to, for instance, a "particle_size" value. So get_size needs to stay.

I haven't removed any of the get_ methods from SS_HopOutput.py. get_size is still there. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

I've done it in r1168. On Fri, Feb 13, 2009 at 1:52 PM, Matthew Turk wrote:

Alright, I'll take it from here and get back to you.

On Fri, Feb 13, 2009 at 1:52 PM, Stephen Skory wrote:

...

Matt,

...

When I made reference to removing the getter methods, I did not mean getter methods that are not for particle fields. get_size does not refer to, for instance, a "particle_size" value. So get_size needs to stay.

I haven't removed any of the get_ methods from SS_HopOutput.py. get_size is still there. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

This was not meant to be relevant to setting _processing. _processing exists so that sub-calls withing a given calculation of halo properties don't block. The original intent of the proxy class was to avoid any per-processor changes in operation; that's why the _processing flag still has to be set manually. I mean, if you put the particle dumping routines *as methods of the class itself* they should work perfectly fine, because they'll be inside the proxy handling. This is the path I would recommend. If PyTables had parallel HDF5 IO, we could look into how to write multiple at a time. It doesn't, so we can simply write a routine that accepts a filename, opens it for append, writes, returns. The proxy class was not designed to handled differently based on which processor each group is on. The idea here is to avoid that, and to make the API independent of processor location; this amounts to a contract between the user and the code: "If you promise to do your best to make it so I don't have to mess with the internals, I promise not to touch them." If you start ignoring the tacit agreement that in the process of executing a script, you're going to have to recognize that the system might break down, and you might have to mess with internals. Things aren't perfect, but they do sort of work. -Matt On Fri, Feb 13, 2009 at 2:54 PM, Stephen Skory wrote:

Matt,

...

I've done it in r1168.

It works now without giving an error, but I still need to set _processing = True for it to work. Was that your intention? Without fippping _processing, it just hangs, as it did at the beginning of this thread.

Thanks!

_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Matt, I think I don't quite understand something fundamental about how parallel python works. In particular, how do I make the loop at line 29 run on all processors at the same time? As an experiment I'm trying to write separate HDF5 files per processor to speed things up, but from the timestamp (line 73) I can see that it's going serially in order of halo.id. For example, the first halo on proc 0 is the sixth, but it isn't written until the first five are written by proc 1, so it's going in strict order. http://paste.enzotools.org/show/49/ wrote halo 0 from proc 1 at 2009-02-14 10:53:05.206974 wrote halo 1 from proc 1 at 2009-02-14 10:53:05.296788 wrote halo 2 from proc 1 at 2009-02-14 10:53:05.406035 wrote halo 3 from proc 1 at 2009-02-14 10:53:05.506907 wrote halo 4 from proc 1 at 2009-02-14 10:53:05.626588 wrote halo 5 from proc 0 at 2009-02-14 10:53:06.474605 wrote halo 6 from proc 0 at 2009-02-14 10:53:06.553487 wrote halo 7 from proc 1 at 2009-02-14 10:53:06.688611 wrote halo 8 from proc 1 at 2009-02-14 10:53:06.790755 wrote halo 9 from proc 0 at 2009-02-14 10:53:06.879232 wrote halo 10 from proc 1 at 2009-02-14 10:53:06.978932 Thanks for answering my naive questions... _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Matt,

what's causing the problem is get_size.

That makes sense, and indeed I just confirmed it by commenting out get_size. Thank you very much. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________