Hi all, If we are able to track this down to an item that's not in yt -- i.e., an interplay between h5py, numpy, python 2.7, etc -- we should try to report these issues upstream. In the past there have been indications that interplay between Python 2.7 and h5py 1.3.1 have caused issues, although this particular issue is unfamiliar to me. h5py 1.4.0 is coming out soon (a beta was just released) so perhaps there will be relief on that end. Anyway, if you can figure out how to get this problem to be reproduced without importing yt directly, it might help track it down. Also, Geoffrey, I think I have run on up to 64 processors on Nautilus, but maybe not since the latest upgrade. Thanks for the pointer about mpt and the neighborhood issue. Best, Matt On Wed, May 25, 2011 at 3:06 PM, wrote:

I'm not sure if this is relevant to Kraken users, but I have gone through a lot of trouble to make YT work on Nautilus, which is similar machine to Kraken.  After going back and forth with the sys admins, I was told to do the following:

module swap mpt mpt/2.03 export MPI_SHARED_NEIGHBORHOOD=HOST

and use no more than 8 cores due to some bug in the drivers of a particular hardware.

Once I had YT working on Nautilus I haven't tried to use YT on Kraken.

From G.S.

...

I encountered a similar error trying to run python 2.7 in parallel on kraken.  Has anyone on this list been able to the Kraken YT install in parallel?  If anyone has, it would be vary helpful to know.

Britton

On Wed, May 25, 2011 at 12:57 PM, Anthony Harness < anthony.harness@colorado.edu> wrote:

...

Using the "unstable" version of yt on Kraken, I get the following error while simply trying to import yt while running in parallel. I haven't done anything memory-intensive, so I suspect this is a bug. I have the PrgEnv-gnu module loaded.

Traceback (most recent call last):   File "/lustre/scratch/harnessa/analysis/Clumping/test2.py", line 1, in <module>     from yt.mods import *   File "/lustre/scratch/proj/yt_common/unstable/lib/python2.7/site-packages/yt-2.2dev-py2.7-linux-x86_64.egg/yt/mods.py", line 35, in <module>     import numpy as na # For historical reasons   File "/lustre/scratch/proj/yt_common/unstable/lib/python2.7/site-packages/numpy/__init__.py", line 154, in <module>     import ctypeslib   File "/lustre/scratch/proj/yt_common/unstable/lib/python2.7/site-packages/numpy/ctypeslib.py", line 60, in <module>     import ctypes   File "/lustre/scratch/proj/yt_common/unstable/lib/python2.7/ctypes/__init__.py", line 549, in <module>     CFUNCTYPE(c_int)(lambda: None) MemoryError

Thanks, Anthony

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Anthony, I just tried a very simple parallel python script using the yt module and had success. I have these two lines in my job script: module swap PrgEnv-pgi PrgEnv-gnu ; module load yt ; Can you try adding those in and giving that a shot? Britton On Wed, May 25, 2011 at 9:55 PM, Stephen Skory wrote:

Hi All,

I'm pretty certain that the install of yt that Anthony was trying to use (/lustre/scratch/proj/yt_common/unstable), which is the one I maintain, will not work in parallel. It has to do with various shared library issues that I never quite sorted out. However, I've successfully run things on Kraken in parallel using the yt module that Harinarayan (Hari) Krishnan has set up. If it's not updated to include the fixes you need, perhaps Hari can update it.

Anyway, I've updated my install, which should work as long as you're running from a login node.

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org