Mailman 3 July 2011 - yt-users

h5py problem
by John ZuHone 07 Jul '11

07 Jul '11

Hi all, I have a loop that goes over a list of filenames I've generated, loads parameter files for them, passes each pf object to a procedure which grabs data from the file, and then loops back over. After the first iteration, I get this crash: Traceback (most recent call last): File "/local/data/duce2/zuhone/src/ctools/filter_particle_velocities.py", line 91, in <module> main() File "/local/data/duce2/zuhone/src/ctools/filter_particle_velocities.py", line 68, in main compute_turb_vels(pf, numVelCells, xp, yp, zp, vxp, vyp, vzp) File "/local/data/duce2/zuhone/src/ctools/compute_turb_vels.py", line 15, in compute_turb_vels leaf_grids = [g for g in pf.h.grids if len(g.Children) == 0] File "/local/data/duce2/zuhone/opt/yt-x86_64/src/yt-hg/yt/data_objects/static_output.py", line 163, in hierarchy self, data_style=self.data_style) File "/local/data/duce2/zuhone/opt/yt-x86_64/src/yt-hg/yt/frontends/flash/data_structures.py", line 73, in __init__ self._handle = h5py.File(self.hierarchy_filename) File "/local/data/duce2/zuhone/opt/yt-x86_64/lib/python2.7/site-packages/h5py-1.3.1-py2.7-linux-x86_64.egg/h5py/highlevel.py", line 797, in __init__ self.fid = self._generate_fid(name, mode, plist) File "/local/data/duce2/zuhone/opt/yt-x86_64/lib/python2.7/site-packages/h5py-1.3.1-py2.7-linux-x86_64.egg/h5py/highlevel.py", line 842, in _generate_fid fid = h5f.create(name, h5f.ACC_EXCL, fapl=plist) File "h5f.pyx", line 87, in h5py.h5f.create (h5py/h5f.c:1378) h5py._stub.IOError: File exists (File accessability: Unable to open file) It does not matter where in the list of files I start, so it's not due to a single bad file. It's like it doesn't detect the file it's trying to read somehow, so it tries to create it, but then it finds out it already exists, and then aborts. Very odd. Best, John ZuHone

4 7

pHOP child halos
by Brian Crosby 05 Jul '11

05 Jul '11

Hi everyone, I'm using Nautilus to run pHOP on a 1024^3 dataset and have encountered a strange problem. All of the halos are found and written to the SQL database successfully, but then when pHOP attempts to go back and identify the parent/child relationships, it hangs and gives message: Traceback (most recent call last): File "halofinding.py", line 14, in <module> MergerTree(restart_files=files, database='/gpfs/medusa/oshea/PopIII_1024/halos1024.db', halo_finder_function=parallelHF) File "/nics/e/sw/analysis/yt/2.1/sles11.1_gnu4.3.4/src/yt-hg/yt/analysis_modules/halo_merger_tree/merger_tree.py", line 201, in __init__ last = self._compute_child_fraction(pair[0], pair[1], last) File "/nics/e/sw/analysis/yt/2.1/sles11.1_gnu4.3.4/src/yt-hg/yt/analysis_modules/halo_merger_tree/merger_tree.py", line 489, in _compute_child_fraction for group in h5fp: File "/sw/analysis/yt/2.1/sles11.1_gnu4.3.4/lib/python2.7/site-packages/h5py-1.2.0-py2.7-linux-x86_64.egg/h5py/highlevel.py", line 300, in __iter__ return self.id.__iter__() File "/sw/analysis/yt/2.1/sles11.1_gnu4.3.4/lib/python2.7/site-packages/h5py-1.2.0-py2.7-linux-x86_64.egg/h5py/_sync.py", line 94, in wrap return func(*args, **kwds) File "h5g.pyx", line 449, in h5py.h5g.GroupID.__iter__ (h5py/h5g.c:3441) File "h5g.pyx", line 103, in h5py.h5g.GroupIter.__init__ (h5py/h5g.c:1235) File "/sw/analysis/yt/2.1/sles11.1_gnu4.3.4/lib/python2.7/site-packages/h5py-1.2.0-py2.7-linux-x86_64.egg/h5py/_sync.py", line 88, in wrap return func(*args, **kwds) File "h5g.pyx", line 316, in h5py.h5g.GroupID.get_num_objs (h5py/h5g.c:2469) h5py._stub.ValueError: Invalid group id (Invalid arguments to routine: Bad value) Does anyone have an idea what might be going wrong here, or how I should go about fixing it? Thanks, Brian

4 7

SciPy
by John ZuHone 02 Jul '11

02 Jul '11

Hi all, Has anyone successfully installed SciPy on a 64bit Linux machine with the ATLAS/LAPACK libraries? I'm having lots of trouble with errors like these: ld: liblapack.a(second_INT_CPU_TIME.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC liblapack.a(second_INT_CPU_TIME.o): could not read symbols: Bad value The forums I've browsed say that some files in LAPACK don't get compiled with fPIC, even when specified in the Makefile, and must be compiled manually, which I have done, but this does not change anything. I need to build it from source so I can use it with yt. Best, John ZuHone

1 0

Unpickling phase plots
by david collins 01 Jul '11

01 Jul '11

Hi there-- I have a phase plot that I pickled on one machine, and moved to my laptop and want to re-plot. I pickled it like this: fptr=open(filename,'rb') cPickle.dump(pc.plots[-1].data,fptr,protocol=-1) I want to unpickle it like this: obj = cPickle.load( open(filename) ) but I get this parameter file error (full traceback below): return self._convert_pf(self._records[hash]) KeyError: (('32aa7d1d4941760ffd8fdcab4ea97e1f',), <function _reconstruct_pf at 0x101f9d050>, ('32aa7d1d4941760ffd8fdcab4ea97e1f',)) I suspect that the key error is because I don't have the original dataset on this machine (due to disk space) so the pf store on this machine doesn't have that key. Does that sound right, and is there a work around for that? This is a technique I've used in the past, so in principle it's sound. Thanks, d. Full trace: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "p8_theta_coarse.py", line 6, in <module> phase = fPickle.load(infile) File "/Users/dcollins/Sites/Research/CodeGames/yt2.0/fPickle.py", line 9, in load output = cPickle.load(inputfile,*args,**kwargs) File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/static_output.py", line 192, in _reconstruct_pf pf = pfs.get_pf_hash(*args) File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/utilities/parameter_file_storage.py", line 104, in get_pf_hash return self._convert_pf(self._records[hash]) KeyError: (('32aa7d1d4941760ffd8fdcab4ea97e1f',), <function _reconstruct_pf at 0x101f9d050>, ('32aa7d1d4941760ffd8fdcab4ea97e1f',)) -- Sent from my computer.

3 6