embarrassingly parallel analysis
Hi all, I was trying out pobj_demo.py from the workshop to see if I can parallelize my halo analysis script, but I ran into the problem where I see:
mpirun -n 2 python pobj_demo.py --parallel
gives a warning at the end yt : [WARNING ] 2012-03-16 16:25:05,445 parallel_objects() is being used when parallel_capable is false. The loop is not being run in parallel. This may not be what was expected. I've done a yt instinfo -u and got the latest tip: 052fac826701 (yt) tip but the problem persists. I've tried printing inside the parallel loops for sto, sp in parallel_objects(spheres, num_procs, storage = my_storage): print ytcfg.getint("yt", "__global_parallel_rank") sto.result = sp.quantities['TotalQuantity']('CellMass') sto.result_id = '%4e %4e %4e' % (sp.center[0], sp.center[1],sp.center[2]) and I always get "0", but 20 of them, so I'm guessing that confirms the loop isn't running in parallel, and just runs the 10 spheres serially on 2 processors, so I get double the results. Am I doing something wrong or missed something? From G.S.
Hi Geoffrey, You may need to build an mpi-capable python interpreter by hand. See: http://mpi4py.scipy.org/docs/usrman/appendix.html#mpi-enabled-python-interpr... Nathan Goldbaum Graduate Student Astronomy & Astrophysics, UCSC goldbaum@ucolick.org http://www.ucolick.org/~goldbaum On Mar 16, 2012, at 4:37 PM, Geoffrey So wrote:
Hi all, I was trying out pobj_demo.py from the workshop to see if I can parallelize my halo analysis script, but I ran into the problem where I see:
mpirun -n 2 python pobj_demo.py --parallel
gives a warning at the end
yt : [WARNING ] 2012-03-16 16:25:05,445 parallel_objects() is being used when parallel_capable is false. The loop is not being run in parallel. This may not be what was expected.
I've done a yt instinfo -u and got the latest tip:
052fac826701 (yt) tip
but the problem persists. I've tried printing inside the parallel loops
for sto, sp in parallel_objects(spheres, num_procs, storage = my_storage): print ytcfg.getint("yt", "__global_parallel_rank") sto.result = sp.quantities['TotalQuantity']('CellMass') sto.result_id = '%4e %4e %4e' % (sp.center[0], sp.center[1],sp.center[2])
and I always get "0", but 20 of them, so I'm guessing that confirms the loop isn't running in parallel, and just runs the 10 spheres serially on 2 processors, so I get double the results.
Am I doing something wrong or missed something?
From G.S.
!DSPAM:10175,4f63cec725372860820662! _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
!DSPAM:10175,4f63cec725372860820662!
I can run parallelHOP in parallel, doesn't that mean I already have an mpi-capable interpreter? From G.S. On Fri, Mar 16, 2012 at 4:43 PM, Nathan Goldbaum <goldbaum@ucolick.org>wrote:
Hi Geoffrey,
You may need to build an mpi-capable python interpreter by hand. See: http://mpi4py.scipy.org/docs/usrman/appendix.html#mpi-enabled-python-interpr...
Nathan Goldbaum Graduate Student Astronomy & Astrophysics, UCSC goldbaum@ucolick.org http://www.ucolick.org/~goldbaum
On Mar 16, 2012, at 4:37 PM, Geoffrey So wrote:
Hi all, I was trying out pobj_demo.py from the workshop to see if I can parallelize my halo analysis script, but I ran into the problem where I see:
mpirun -n 2 python pobj_demo.py --parallel
gives a warning at the end
yt : [WARNING ] 2012-03-16 16:25:05,445 parallel_objects() is being used when parallel_capable is false. The loop is not being run in parallel. This may not be what was expected.
I've done a yt instinfo -u and got the latest tip:
052fac826701 (yt) tip
but the problem persists. I've tried printing inside the parallel loops
for sto, sp in parallel_objects(spheres, num_procs, storage = my_storage): print ytcfg.getint("yt", "__global_parallel_rank") sto.result = sp.quantities['TotalQuantity']('CellMass') sto.result_id = '%4e %4e %4e' % (sp.center[0], sp.center[1],sp.center[2])
and I always get "0", but 20 of them, so I'm guessing that confirms the loop isn't running in parallel, and just runs the 10 spheres serially on 2 processors, so I get double the results.
Am I doing something wrong or missed something?
From G.S.
!DSPAM:10175,4f63cec725372860820662! _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
!DSPAM:10175,4f63cec725372860820662!
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Nevermind, it was something stupid, I didn't install mpi4py for this version of yt I was using, now it gives 1 and 0s for print ytcfg.getint("yt", "__global_parallel_rank") after installing mpi4py 1.3. From G.S. On Fri, Mar 16, 2012 at 4:59 PM, Geoffrey So <gsiisg@gmail.com> wrote:
I can run parallelHOP in parallel, doesn't that mean I already have an mpi-capable interpreter?
From G.S.
On Fri, Mar 16, 2012 at 4:43 PM, Nathan Goldbaum <goldbaum@ucolick.org>wrote:
Hi Geoffrey,
You may need to build an mpi-capable python interpreter by hand. See: http://mpi4py.scipy.org/docs/usrman/appendix.html#mpi-enabled-python-interpr...
Nathan Goldbaum Graduate Student Astronomy & Astrophysics, UCSC goldbaum@ucolick.org http://www.ucolick.org/~goldbaum
On Mar 16, 2012, at 4:37 PM, Geoffrey So wrote:
Hi all, I was trying out pobj_demo.py from the workshop to see if I can parallelize my halo analysis script, but I ran into the problem where I see:
mpirun -n 2 python pobj_demo.py --parallel
gives a warning at the end
yt : [WARNING ] 2012-03-16 16:25:05,445 parallel_objects() is being used when parallel_capable is false. The loop is not being run in parallel. This may not be what was expected.
I've done a yt instinfo -u and got the latest tip:
052fac826701 (yt) tip
but the problem persists. I've tried printing inside the parallel loops
for sto, sp in parallel_objects(spheres, num_procs, storage = my_storage): print ytcfg.getint("yt", "__global_parallel_rank") sto.result = sp.quantities['TotalQuantity']('CellMass') sto.result_id = '%4e %4e %4e' % (sp.center[0], sp.center[1],sp.center[2])
and I always get "0", but 20 of them, so I'm guessing that confirms the loop isn't running in parallel, and just runs the 10 spheres serially on 2 processors, so I get double the results.
Am I doing something wrong or missed something?
From G.S.
!DSPAM:10175,4f63cec725372860820662! _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
!DSPAM:10175,4f63cec725372860820662!
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Geoffrey So -
Nathan Goldbaum