On Wed, Dec 14, 2016 at 5:30 AM, Thor Andreas Seiff Ellewsen <tellewsen@gmail.com> wrote:
Hi all,

I'm working on a project where I'm comparing different RAMSES simulations.
However I'm running into a problem with memory.
It seems that yt loads the whole simulation into memory even though I'm only interested in a number of dark matter halos which I have the coordinates for.

Yes, this is a limitation of our current approach for octree data, we must build a global octree for indexing purposes. For large simulations this can be a prohibitive memory cost. We have long-term plans to fix this but as far as I know no one is actively working on it.

It also seems like it doesn't drop the first simulation out of memory when I'm done making the figures I want for that one. 

This quickly fills up the memory.

You can find the relevant parts of my code at the pastebin at http://paste.yt/p4946.html
I'm running yt version 3.3.2

What am I doing wrong?

In principle this shouldn't happen but it's entirely possible there is a bug somewhere. In the python memory model, an object is retained in memory until no other object have a reference to it, at which point it is garbage collected. I don't see anything in your script that would cause references to old datasets to be retained, so it might be the case that yt is inadvertantly retaining a reference to datasets in a ramses time series when it shouldn't be or if we need the reference that we should be using a weakref. Unfortunately it looks like we don't have any Ramses time series datasets available in our repository of public test datasets (http://yt-project.org/data/) so I don't have a way of reproducing this behavior. Is there any chance you can make a ramses time series available for testing? You might find the yt curldrop to be useful for sharing a large dataset:


A test that this is isolated to the time series object would be to instead use a loop over filenames, loading a single filename in each loop iteration, and then explicitly deleting the dataset object using the `del` statement at the end of the loop. In principle this shouldn't be necessary but it might help reveal where the issue is localized to.

Also, is there a way to change the axis names?
I'm making some dark matter density profiles using the ('deposit','io_density') field and this gives IO Density on the y axis, but I just want it to say Dark matter density so my readers actually understand what I'm talking about.

For the ProjectionPlot you can use the set_colorbar_label method:


Thanks for helping

Andreas Ellewsen

yt-users mailing list