MEMORY ERROR ACCESSING HIGHER AMR LEVELS
Hi, I'm encountering a memory issue when I try to access higher AMR levels of my simulation using arbitrary_grid. My simulation has a minimum level of 7 and maximum level 15. I'm accessing level 9 (corresponding to 512).
DIM=512
dd=ds.arbitrary_grid(left_edge= ds.arr([(com_x_1.value-30), (com_y_1.value-30), (com_z_1.value-30)], 'kpc'), right_edge= ds.arr([(com_x_1.value+30), (com_y_1.value+30), (com_z_1.value+30)], 'kpc'), dims=[DIM,DIM,DIM])
m=dd["gas", "cell_mass"].in_units('Msun') and I get:
Traceback (most recent call last):
File "vel_disp.py", line 52, in <module> m=dd["gas", "cell_mass"].in_units('Msun') File
"/home/samvad/yt-conda/lib/python3.6/site-packages/yt/units/yt_array.py", line 610, in in_units
new_array = type(self)(self.ndview * conversion_factor, new_units) MemoryError Could someone please shed light on this? Am I accessing the levels in a wrong format? Is "dims" not corresponding to the absolute levels but the level differences? (as in the level 7 is considered to be level 0 in which case my maximum level would be 8)
How much RAM do you have access to on that machine? On Mon, Nov 5, 2018 at 4:01 AM Vadlamani Samhitha < vadlamani.samhitha@gmail.com> wrote:
Hi,
I'm encountering a memory issue when I try to access higher AMR levels of my simulation using arbitrary_grid. My simulation has a minimum level of 7 and maximum level 15. I'm accessing level 9 (corresponding to 512).
DIM=512
dd=ds.arbitrary_grid(left_edge= ds.arr([(com_x_1.value-30), (com_y_1.value-30), (com_z_1.value-30)], 'kpc'), right_edge= ds.arr([(com_x_1.value+30), (com_y_1.value+30), (com_z_1.value+30)], 'kpc'), dims=[DIM,DIM,DIM])
m=dd["gas", "cell_mass"].in_units('Msun')
and I get:
Traceback (most recent call last):
File "vel_disp.py", line 52, in <module>
m=dd["gas", "cell_mass"].in_units('Msun')
File
"/home/samvad/yt-conda/lib/python3.6/site-packages/yt/units/yt_array.py", line 610, in in_units
new_array = type(self)(self.ndview * conversion_factor, new_units)
MemoryError
Could someone please shed light on this? Am I accessing the levels in a wrong format? Is "dims" not corresponding to the absolute levels but the level differences? (as in the level 7 is considered to be level 0 in which case my maximum level would be 8) _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
64GB is what I have access to On Mon, 5 Nov 2018 at 2:34 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
How much RAM do you have access to on that machine?
On Mon, Nov 5, 2018 at 4:01 AM Vadlamani Samhitha < vadlamani.samhitha@gmail.com> wrote:
Hi,
I'm encountering a memory issue when I try to access higher AMR levels of my simulation using arbitrary_grid. My simulation has a minimum level of 7 and maximum level 15. I'm accessing level 9 (corresponding to 512).
DIM=512
dd=ds.arbitrary_grid(left_edge= ds.arr([(com_x_1.value-30), (com_y_1.value-30), (com_z_1.value-30)], 'kpc'), right_edge= ds.arr([(com_x_1.value+30), (com_y_1.value+30), (com_z_1.value+30)], 'kpc'), dims=[DIM,DIM,DIM])
m=dd["gas", "cell_mass"].in_units('Msun')
and I get:
Traceback (most recent call last):
File "vel_disp.py", line 52, in <module>
m=dd["gas", "cell_mass"].in_units('Msun')
File
"/home/samvad/yt-conda/lib/python3.6/site-packages/yt/units/yt_array.py", line 610, in in_units
new_array = type(self)(self.ndview * conversion_factor, new_units)
MemoryError
Could someone please shed light on this? Am I accessing the levels in a wrong format? Is "dims" not corresponding to the absolute levels but the level differences? (as in the level 7 is considered to be level 0 in which case my maximum level would be 8)
_______________________________________________
yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
I don't know offhand why you're hitting a memory error, I don't think this should need more than a few gigabytes of RAM to finish. Are you sure you have access to the full 64 GB of RAM? A quick way to check is to watch the python process executing your yt script in e.g. top to see how much memory it's using as it executes. There's also a memory_checker context manager you can use in yt.funcs which will print out the memory used by a script as it executes: with yt.funcs.memory_checker(interval=1): my_analysis_code() If yt is really using more than 64 GB of RAM here it would be great if you could share a reproducible example I can run locally. I'm not sure what the size of your RAMSES datasets are but if they're not prohibitively huge I'm happy to work with you to share the dataset off-list if it's proprietary and can't be shared publicly for some reason. You could also see if using one of the test ramses datasets on yt-project.org/data uses a ton of RAM for a similar workflow. On Mon, Nov 5, 2018 at 8:04 AM Vadlamani Samhitha < vadlamani.samhitha@gmail.com> wrote:
64GB is what I have access to
On Mon, 5 Nov 2018 at 2:34 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
How much RAM do you have access to on that machine?
On Mon, Nov 5, 2018 at 4:01 AM Vadlamani Samhitha < vadlamani.samhitha@gmail.com> wrote:
Hi,
I'm encountering a memory issue when I try to access higher AMR levels of my simulation using arbitrary_grid. My simulation has a minimum level of 7 and maximum level 15. I'm accessing level 9 (corresponding to 512).
DIM=512
dd=ds.arbitrary_grid(left_edge= ds.arr([(com_x_1.value-30), (com_y_1.value-30), (com_z_1.value-30)], 'kpc'), right_edge= ds.arr([(com_x_1.value+30), (com_y_1.value+30), (com_z_1.value+30)], 'kpc'), dims=[DIM,DIM,DIM])
m=dd["gas", "cell_mass"].in_units('Msun')
and I get:
Traceback (most recent call last):
File "vel_disp.py", line 52, in <module>
m=dd["gas", "cell_mass"].in_units('Msun')
File
"/home/samvad/yt-conda/lib/python3.6/site-packages/yt/units/yt_array.py", line 610, in in_units
new_array = type(self)(self.ndview * conversion_factor, new_units)
MemoryError
Could someone please shed light on this? Am I accessing the levels in a wrong format? Is "dims" not corresponding to the absolute levels but the level differences? (as in the level 7 is considered to be level 0 in which case my maximum level would be 8)
_______________________________________________
yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
participants (2)
-
Nathan Goldbaum -
Vadlamani Samhitha