Grand, thanks, Matt. 
If you're on stampede, any of the data in /scratch/00369/tg456484/Paper49d_moresims/ze01_M10_MA1_1024_quan should be readable. 
I've made a ticket, as well.  The set was made with the StochasticForcing parameter file in the run directory with DualEnergy off.

Thanks, and let me know if there's anything else you need from me!
d.

On Fri, Feb 22, 2019 at 4:38 PM Matthew Turk <matthewturk@gmail.com> wrote:
I will try to make a mock dataset.  My *guess* is that this is related to the issue Andrew Myers was having, and his hotfix might work for this.

On Fri, Feb 22, 2019 at 3:30 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:
OK, it would be great if you could file an issue about this. If possible please include a link to a copy of the dataset so that one of us can try to reproduce the issue.

On Fri, Feb 22, 2019 at 3:26 PM David Collins <dcollins4096@gmail.com> wrote:
Hi--

It's a Enzo run, unigrid at 1024^3, and I ran on 4096 cores, so 4096 patches.  No particles, MHD with Dedner and Stochastic forcing, no other physics.
Thanks,
d.

On Fri, Feb 22, 2019 at 2:37 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:
Can you give more detail about the data? Enzo I'm guessing? Unigrid or AMR? How many grid patches in the output?

My first thought is that this is probably a bug, especially if it runs within memory bounds on one core, there's probably some unnecessary double-allocation going on.

On Fri, Feb 22, 2019 at 1:31 PM David Collins <dcollins4096@gmail.com> wrote:
Hi, Everybody!

I'm trying to run a parallel projection on a 1024^3 simulation, but I keep getting out-of-memory.  Serial works fine.  This is all on Stampede, on an interactive node.  My script is direct from the website, and is below.  Is there something dumb I'm doing?

Thanks!

I do

$ibrun -np 2 python parallel_test.py

and parallel_test.py contains
"""
from mpi4py import MPI
import yt
yt.enable_parallelism()

ds = yt.load("/scratch/00369/tg456484/Paper49d_moresims/ze01_M10_MA1_1024_quan/DD0095/data0095")
/DD%04d/data%04d"%(22,22))
p = yt.ProjectionPlot(ds, "x", "density")
p.save()
"""

my output then looks like:

000 yt : [INFO     ] 2019-02-22 13:00:41,879 Gathering a field list (this may take a moment.)
  File "parallel_test.py", line 13, in <module>
    p = yt.ProjectionPlot(ds, "x", "density")
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/visualizati
on/plot_window.py", line 1480, in __init__
    max_level=max_level)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_object
s/construction_data_containers.py", line 270, in __init__
    self.get_data(field)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_object
s/construction_data_containers.py", line 334, in get_data
    self._initialize_chunk(chunk, tree)

  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 271, in cached_func
    tr = func(self)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 332, in icoords
    ci = np.empty((self.data_size, 3), dtype='int64')
P006 yt : [ERROR    ] 2019-02-22 13:03:36,145 MemoryError:
  File "parallel_test.py", line 13, in <module>
    p = yt.ProjectionPlot(ds, "x", "density")
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/visualization/plot_window.py", line 1480, in __init__
    max_level=max_level)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 270, in __init__
    self.get_data(field)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 334, in get_data
    self._initialize_chunk(chunk, tree)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/construction_data_containers.py", line 401, in _initialize_chunk
    icoords = chunk.icoords
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1555, in icoords
    return self._current_chunk.icoords
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 271, in cached_func
    tr = func(self)
  File "/home1/00369/tg456484/local-yt-2019-02-22-py3/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 332, in icoords
    ci = np.empty((self.data_size, 3), dtype='int64')
P007 yt : [ERROR    ] 2019-02-22 13:03:36,145 MemoryError:
P002 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 2.
P004 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 4.
P000 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 0.
P001 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 1.
P003 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 3.
P005 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 5.
P006 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 6.
P007 yt : [ERROR    ] 2019-02-22 13:03:36,200 Error occured on rank 7.
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 5
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 6
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 7
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 4
TACC:  MPI job exited with code: 1
TACC:  Shutdown complete. Exiting.


--
-- Sent from a computer.
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org


--
-- Sent from a computer.
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org


--
-- Sent from a computer.