On Wed, Dec 14, 2016 at 5:30 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:
Hi all,
I'm working on a project where I'm comparing different RAMSES simulations. However I'm running into a problem with memory. It seems that yt loads the whole simulation into memory even though I'm only interested in a number of dark matter halos which I have the coordinates for.
Yes, this is a limitation of our current approach for octree data, we must build a global octree for indexing purposes. For large simulations this can be a prohibitive memory cost. We have long-term plans to fix this but as far as I know no one is actively working on it.
It also seems like it doesn't drop the first simulation out of memory when I'm done making the figures I want for that one.
This quickly fills up the memory.
You can find the relevant parts of my code at the pastebin at http://paste.yt/p4946.html I'm running yt version 3.3.2
What am I doing wrong?
In principle this shouldn't happen but it's entirely possible there is a bug somewhere. In the python memory model, an object is retained in memory until no other object have a reference to it, at which point it is garbage collected. I don't see anything in your script that would cause references to old datasets to be retained, so it might be the case that yt is inadvertantly retaining a reference to datasets in a ramses time series when it shouldn't be or if we need the reference that we should be using a weakref. Unfortunately it looks like we don't have any Ramses time series datasets available in our repository of public test datasets ( http://yt-project.org/data/) so I don't have a way of reproducing this behavior. Is there any chance you can make a ramses time series available for testing? You might find the yt curldrop to be useful for sharing a large dataset:
https://docs.hub.yt/services.html#curldrop
A test that this is isolated to the time series object would be to instead use a loop over filenames, loading a single filename in each loop iteration, and then explicitly deleting the dataset object using the `del` statement at the end of the loop. In principle this shouldn't be necessary but it might help reveal where the issue is localized to.
Also, is there a way to change the axis names? I'm making some dark matter density profiles using the ('deposit','io_density') field and this gives IO Density on the y axis, but I just want it to say Dark matter density so my readers actually understand what I'm talking about.
For the ProjectionPlot you can use the set_colorbar_label method:
http://yt-project.org/doc/reference/api/generated/yt.visualization.plot_wind...
Thanks for helping
Best, Andreas Ellewsen
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org