Hi, Everybody!
I'm trying to run a parallel projection on a 1024^3 simulation, but I keep getting out-of-memory. Serial works fine. This is all on Stampede, on an interactive node. My script is direct from the website, and is below. Is there something dumb I'm doing?
Thanks!
I do
$ibrun -np 2 python parallel_test.py
and parallel_test.py contains
"""
from mpi4py import MPI
import yt
yt.enable_parallelism()
ds = yt.load("/scratch/00369/tg456484/Paper49d_moresims/ze01_M10_MA1_1024_quan/DD0095/data0095")
/DD%04d/data%04d"%(22,22))
p = yt.ProjectionPlot(ds, "x", "density")
p.save()
"""
my output then looks like:
000 yt : [INFO ] 2019-02-22 13:00:41,879 Gathering a field list (this may take a moment.)
File "parallel_test.py", line 13, in <module>
p = yt.ProjectionPlot(ds, "x", "density")
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/visualizati
on/plot_window.py", line 1480, in __init__
max_level=max_level)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_object
s/construction_data_containers.py", line 270, in __init__
self.get_data(field)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_object
s/construction_data_containers.py", line 334, in get_data
self._initialize_chunk(chunk, tree)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 271, in cached_func
tr = func(self)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 332, in icoords
ci = np.empty((self.data_size, 3), dtype='int64')
P006 yt : [ERROR ] 2019-02-22 13:03:36,145 MemoryError:
File "parallel_test.py", line 13, in <module>
p = yt.ProjectionPlot(ds, "x", "density")
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/visualization/plot_window.py", line 1480, in __init__
max_level=max_level)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 270, in __init__
self.get_data(field)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 334, in get_data
self._initialize_chunk(chunk, tree)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 401, in _initialize_chunk
icoords = chunk.icoords
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1555, in icoords
return self._current_chunk.icoords
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 271, in cached_func
tr = func(self)
File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 332, in icoords
ci = np.empty((self.data_size, 3), dtype='int64')
P007 yt : [ERROR ] 2019-02-22 13:03:36,145 MemoryError:
P002 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 2.
P004 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 4.
P000 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 0.
P001 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 1.
P003 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 3.
P005 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 5.
P006 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 6.
P007 yt : [ERROR ] 2019-02-22 13:03:36,200 Error occured on rank 7.
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 5
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 6
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 7
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 4
TACC: MPI job exited with code: 1
TACC: Shutdown complete. Exiting.