Hello,
I’m pretty new to yt… I’m trying to speed things up significantly since I have rather large RAMSES datasets (running 3Mpc sims with 2^9^3 particles). I’m using STAMPEDE at TACC, https://vis.tacc.utexas.edu/# , to startup a notebook on their visualization nodes (each has 16 cores). I’m trying to use
import yt yt.enable_parallelism() ds = yt.load("./output_00011/info_00011.txt”)
and then
# Looking through the "z" axis... Plot temp weighted by density
p = yt.ProjectionPlot(ds, "z", "temperature", weight_field="density") p.annotate_title(z) p.show()
but I only see one python process via top … and NO speedup. What do I need to do to take advantage of the 16 cores?
Cheers!
Rick Sarmento SESE Astronomy/Astrophysics Grad Student rsarment@asu.edu
Hi Rick,
You need to launch the notebook in parallel using an MPI parallel IPython cluster. In addition, you will need to configure the notebook to hook into IPython's built-in parallelism.
I've uploaded a notebook that uses parallel IPython here:
http://nbviewer.ipython.org/gist/ngoldbaum/32ca07cf4f5b3dd06add
Note that all the yt operations aren't really important, but the first few cells where I check to make sure IPython's parallelism works and that MPI parllelism is working are quite important.
Note also that you will need to launch the IPython cluster separately from the notebook server using the "ipcluster" command.
If you want to learn more about parallel IPython, I'd encourage you to take a look at Min Ranger-Kelley's tutorial from scipy 2014: http://pyvideo.org/video/2738/interactive-parallel-computing-with-ipython-pa...
All that said, it does add a significant amount of semantic overhead to use the IPython notebook in parallel. It's generally much more straightforward to work with yt in parallel using regular python scripts.
Hope that helps,
Nathan
On Wed, Jan 14, 2015 at 6:49 PM, Rick Sarmento rsarment@asu.edu wrote:
Hello,
I’m pretty new to yt… I’m trying to speed things up significantly since I have rather large RAMSES datasets (running 3Mpc sims with 2^9^3 particles). I’m using STAMPEDE at TACC, https://vis.tacc.utexas.edu/# , to startup a notebook on their visualization nodes (each has 16 cores). I’m trying to use
import yt yt.enable_parallelism() ds = yt.load("./output_00011/info_00011.txt”)
and then
# Looking through the "z" axis... Plot temp weighted by density
p = yt.ProjectionPlot(ds, "z", "temperature", weight_field="density") p.annotate_title(z) p.show()
but I only see one python process via top … and NO speedup. What do I need to do to take advantage of the 16 cores?
Cheers!
Rick Sarmento SESE Astronomy/Astrophysics Grad Student rsarment@asu.edu
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-
Nathan Goldbaum -
Rick Sarmento