Hi all, Does anyone, perhaps Kacper, know if we can do any answer testing in parallel? I would like to add a test for the rockstar halo finder, but it has to be run with a minimum of 3 MPI tasks. This would not require 3 actual CPUs to run on since each the tasks sort of take turns. Is there a way to do this? Thanks! Britton
On 07/28/2014 04:05 PM, Britton Smith wrote:
Hi all,
Does anyone, perhaps Kacper, know if we can do any answer testing in parallel? I would like to add a test for the rockstar halo finder, but it has to be run with a minimum of 3 MPI tasks. This would not require 3 actual CPUs to run on since each the tasks sort of take turns. Is there a way to do this?
Hi Britton, bear in mind that I don't have any experience with mixing mpi and nose, but I'd go for mpi_spawn. Roughly, it would look like that: shakuras ~/codes/britton $ cat test_foo.py #!/usr/bin/env python from mpi4py import MPI import sys def test_blah(): comm = MPI.COMM_SELF.Spawn("blah.py", maxprocs=3) status = MPI.Status() comm.Disconnect() assert True shakuras ~/codes/britton $ cat blah.py #!/usr/bin/env python from mpi4py import MPI import sys comm = MPI.Comm.Get_parent() size = comm.Get_size() rank = comm.Get_rank() sys.stderr.write("size = %i, rank = %i\n" % (size, rank)) comm.Disconnect() blah.py would be an actual test in your case. I think it'd be safer to evaluate if test passed/failed after comm.Disconnect() so that mpi processes are not left hanging, but it needs to be verified empirically ;) Cheers, Kacper P.s. There's a project that does something similar: https://github.com/dagss/mpinoseutils but I don't know its state.
Hi Kacper, Thank you, this is great! I will look into this once we get this release out. Britton On Mon, Jul 28, 2014 at 4:19 PM, Kacper Kowalik <xarthisius.kk@gmail.com> wrote:
On 07/28/2014 04:05 PM, Britton Smith wrote:
Hi all,
Does anyone, perhaps Kacper, know if we can do any answer testing in parallel? I would like to add a test for the rockstar halo finder, but it has to be run with a minimum of 3 MPI tasks. This would not require 3 actual CPUs to run on since each the tasks sort of take turns. Is there a way to do this?
Hi Britton, bear in mind that I don't have any experience with mixing mpi and nose, but I'd go for mpi_spawn. Roughly, it would look like that:
shakuras ~/codes/britton $ cat test_foo.py #!/usr/bin/env python
from mpi4py import MPI import sys def test_blah(): comm = MPI.COMM_SELF.Spawn("blah.py", maxprocs=3) status = MPI.Status() comm.Disconnect() assert True
shakuras ~/codes/britton $ cat blah.py #!/usr/bin/env python
from mpi4py import MPI import sys
comm = MPI.Comm.Get_parent() size = comm.Get_size() rank = comm.Get_rank()
sys.stderr.write("size = %i, rank = %i\n" % (size, rank)) comm.Disconnect()
blah.py would be an actual test in your case. I think it'd be safer to evaluate if test passed/failed after comm.Disconnect() so that mpi processes are not left hanging, but it needs to be verified empirically ;)
Cheers, Kacper
P.s. There's a project that does something similar: https://github.com/dagss/mpinoseutils but I don't know its state.
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Britton Smith -
Kacper Kowalik