Re: [yt-users] Running yt in parallel on a subset of available processes

Hi Cameron, Yeah, no problem. I'm not exactly sure how to do that. Do I have to fork yt first, or can I use my existing installation since it's already using Mercurial? I'm familiar with subversion, since that's what we use in our research group, but I'm a bit hazy on how the whole thing is set up with yt. When I installed yt using the installation script, it created a yt-hg directory. Is that a checked-out version of the code that I can commit? Thanks! Dan 2014-12-03 20:59 GMT-05:00 Cameron Hummels :

Hey Daniel,

This is great that you got this working, and it sounds like something from which others could benefit! Would you mind submitting this change to the main codebase in a pull request?

Cameron

On Wed, Dec 3, 2014 at 6:46 PM, Daniel Fenn wrote:

...

Hi Matt,

I was able to get my MPI problem fixed by using subcommunicators. I modified enable_parallelism() to take an optional argument that specifies the MPI communicator yt should use. If none is specified, it defaults to COMM_WORLD. Seems to be working pretty well so far.

Thanks for the advice,

Dan

2014-12-03 19:16 GMT-05:00 Matthew Turk :

Hi Dan,

...

I've done this before by modifying how the global communicator is created. The way they work in yt is through pushing new communicators onto a stack, which can vary based on which processors you're on. I don't think I ever wrote it up in a PR, but I think it could be done without too much work.

...

Is there a way to restrict yt to only use a subset of the available processes when it's running in parallel? I have a set of ipython engines running, and I'm trying to launch a yt script on a subset of them, but it just hangs. It works fine if I launch the same script on all the

On Wed, Dec 3, 2014 at 11:47 AM, Daniel Fenn wrote: processes.

...

Printing out MPI.COMM_WORLD.Get_size() from the script gives the total number of engines, not the size of the subset I'm trying to use.

I haven't done a lot of parallel programming, but I would guess that

the

...

running processes are waiting on one that's not running a yt script, since they can see the total number of processes available, but not the size of the subset of processes that are actually running a yt script.

Does anyone have any thoughts on how to do this?

Thanks,

Dan

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org