I'm getting a memory error running parallelHF on Ranger. Is this a bug or is there some sort of way to request more memory in my job script?
Here is the full traceback:
Traceback (most recent call last): File "sim_halo_profiler.py", line 17, in <module> rearrange=True, safety=1.5, fancy_padding=True) File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py", line 1921, in __init__ threshold=threshold, dm_only=dm_only, rearrange=rearrange, premerge=premerge) File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py", line 1368, in __init__ HaloList.__init__(self, data_source, dm_only) File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py", line 992, in _ _init__ self._run_finder() File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/halo_objects.py", line 1389, in _run_finder self.threshold, rearrange=self.rearrange, premerge=self.premerge) File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/parallel_hop/parallel_hop_interf ace.py", line 68, in __init__ yt : [INFO ] 2011-05-13 11:42:17,599 Building kd tree for 9519616 particles... self._chain_hop() File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/parallel_hop/parallel_hop_interf ace.py", line 1428, in _chain_hop self._chain_hop() File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/parallel_hop/parallel_hop_interf ace.py", line 1428, in _chain_hop self._init_kd_tree() File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/parallel_hop/parallel_hop_interf ace.py", line 347, in _init_kd_tree fKD.pos = na.asfortranarray(na.empty((3, self.size), dtype='float64')) self._init_kd_tree() File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/yt-2.1stable-py2.6-linux-x86_64.egg/yt/analysis_modules/halo_finding/parallel_hop/parallel_hop_interf ace.py", line 347, in _init_kd_tree MemoryError fKD.pos = na.asfortranarray(na.empty((3, self.size), dtype='float64'))
File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/numpy/core/numeric.py", line 408, in asfortranarray return array(a, dtype, copy=False, order='F', ndmin=1) MemoryError
Hi Carolyn,
File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/numpy/core/numeric.py", line 408, in asfortranarray return array(a, dtype, copy=False, order='F', ndmin=1) MemoryError
You are suspecting the right thing, I think. You will need to run this with more memory. Are you running this in parallel? If so, ask for twice the nodes, with the same number of MPI tasks, putting half the tasks on each node. This will effectively double the amount of RAM per MPI task. I think you should only need to change '16way' to '8way'.
Good luck!
Success! Thanks
On May 13, 2011, at 1:03 PM, Stephen Skory wrote:
Hi Carolyn,
File "/share/home/01078/tg803421/yt-x86_64/lib/python2.6/site-packages/numpy/core/numeric.py", line 408, in asfortranarray return array(a, dtype, copy=False, order='F', ndmin=1) MemoryError
You are suspecting the right thing, I think. You will need to run this with more memory. Are you running this in parallel? If so, ask for twice the nodes, with the same number of MPI tasks, putting half the tasks on each node. This will effectively double the amount of RAM per MPI task. I think you should only need to change '16way' to '8way'.
Good luck!
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org