Parallelism and iteration
Hi all, As some of you may know, I'm working on the geometry subsystem. Right now I'm in the process of constructing incremental reads ("chunking") off disk, with intermediate reduction operations. In the past, this was usually handled by individual tasks, such as profiles, in a pretty gross and manual way. In fixing this up, I noticed we have three different parallel iteration methods. This question is probably best directed at Sam or Britton, but others might also have an opinion. Is there any reason why, from within a subclass of ParallelAnalysisInterface, I wouldn't just use parallel_objects instead of the GridObjectIterator or GridIterator? As a related question, can I get rid of the warning for when running parallel_objects on a single processor? (The geometry handling is coming along nicely, I think. Right now reading from objects works well (and is faster) and the abstraction of grids as opposed to octs or particles has worked well. It's a ways from being feature complete, but if you want to play with it it's all in bb://MatthewTurk/yt-refactor .) -Matt
It looks to me like parallel_objects is more flexible and handles comms better than the ParallelObjectIterator. I think switching to them would make it simpler to customize the parallel strategy of any given operation. I'm definitely in favor of removing the serial warning for parallel_objects. I have put it into the HaloProfiler as a means of dividing up halos for processor groups and often run the HaloProfiler in serial. It works just fine in this mode, so I don't see the need to have the warning. On Thu, Feb 23, 2012 at 9:34 AM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi all,
As some of you may know, I'm working on the geometry subsystem. Right now I'm in the process of constructing incremental reads ("chunking") off disk, with intermediate reduction operations. In the past, this was usually handled by individual tasks, such as profiles, in a pretty gross and manual way. In fixing this up, I noticed we have three different parallel iteration methods.
This question is probably best directed at Sam or Britton, but others might also have an opinion. Is there any reason why, from within a subclass of ParallelAnalysisInterface, I wouldn't just use parallel_objects instead of the GridObjectIterator or GridIterator?
As a related question, can I get rid of the warning for when running parallel_objects on a single processor?
(The geometry handling is coming along nicely, I think. Right now reading from objects works well (and is faster) and the abstraction of grids as opposed to octs or particles has worked well. It's a ways from being feature complete, but if you want to play with it it's all in bb://MatthewTurk/yt-refactor .)
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Britton Smith
-
Matthew Turk