Re: [yt-users] yt-users Digest, Vol 56, Issue 12
On Tue, Oct 9, 2012 at 3:24 AM,
Send yt-users mailing list submissions to yt-users@lists.spacepope.org
To subscribe or unsubscribe via the World Wide Web, visit http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org or, via email, send a message with subject or body 'help' to yt-users-request@lists.spacepope.org
You can reach the person managing the list at yt-users-owner@lists.spacepope.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of yt-users digest..."
Today's Topics:
1. Error when open hdf5 file in yt (Chao Shi) 2. Re: Error when open hdf5 file in yt (Geoffrey So)
----------------------------------------------------------------------
Message: 1 Date: Mon, 8 Oct 2012 23:23:14 -0400 From: Chao Shi
To: yt-users@lists.spacepope.org Subject: [yt-users] Error when open hdf5 file in yt Message-ID: < CA+P3+n_qhLFa-dwUcOYt+WWp_RcqAvjZj_BCYhAJP3HDpZUZYA@mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi all,
I'm trying to modify the particle_type of some particles in a hdf5 data file extract0062.cpu0000 in yt, typing the following command:
for g in pf.h.grids: lbhs =
np.logical_and(np.logical_and(g['particle_type']==1,g['particle_index']>=256**3),g['ParticleMassMsun']>1) # These two lines are to pick out bhs = np.where(lbhs)[0]
# particles that I wanted to modify if(np.any(g['particle_type'])): g['particle_type'][bhs]=9 f=h5py.File(g.filename) ff = f['/Grid%08i'%g.id]['particle_type'] ff.write_direct(g['particle_type']) f.close()
It spited out following error information:
ReadingError Traceback (most recent call last) /home/cshi31/yt-x86_64/src/yt-hg/yt/mods.pyc in <module>() 1 for g in pf.h.grids: ----> 2 lbhs =
np.logical_and(np.logical_and(g['particle_type']==1,g['particle_index']>=256**3),g['ParticleMassMsun']>1) 3 bhs = np.where(lbhs)[0] 4 if(np.any(g['particle_type'])): 5 g['particle_type'][bhs]=9
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in __getitem__(self, key) 155 """ 156 if not self.field_data.has_key(key): --> 157 self.get_data(key) 158 return self.field_data[key] 159
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in get_data(self, field, convert) 188 return self.field_data[field] 189 try: --> 190 temp = self.hierarchy.io.pop(self, field) 191 self[field] = na.multiply(temp, conv_factor, temp) 192 except self.hierarchy.io._read_exception, exc:
/home/cshi31/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in pop(self, grid, field) 59 else: 60 # We only read the one set and do not store it if it isn't pre-loaded ---> 61 return self._read_data_set(grid, field) 62 63 def peek(self, grid, field):
/home/cshi31/yt-x86_64/src/yt-hg/yt/frontends/enzo/io.pyc in _read_data_set(self, grid, field) 183 def _read_data_set(self, grid, field): 184 return self.modify(hdf5_light_reader.ReadData(grid.filename, --> 185 "/Grid%08i/%s" % (grid.id, field))) 186 187 def _read_data_slice(self, grid, field, axis, coord):
ReadingError: ReadHDF5DataSet: Unable to open /home/cshi31/research/SG256-RP/extract0062
So I tried to look where the loop stopped, typed g in the command:
In [37]: g Out[37]: EnzoGrid_0056
so it stopped at the 56th grid in that file. I tried to see what particles is contained in this grid, typing g['particle_type'], it turned out this is what caused the problem:
In [31]: g['particle_type'] --------------------------------------------------------------------------- ReadingError Traceback (most recent call last) /home/cshi31/yt-x86_64/src/yt-hg/yt/mods.pyc in <module>() ----> 1 g['particle_type']
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in __getitem__(self, key) 155 """ 156 if not self.field_data.has_key(key): --> 157 self.get_data(key) 158 return self.field_data[key] 159
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in get_data(self, field, convert) 188 return self.field_data[field] 189 try: --> 190 temp = self.hierarchy.io.pop(self, field) 191 self[field] = na.multiply(temp, conv_factor, temp) 192 except self.hierarchy.io._read_exception, exc:
/home/cshi31/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in pop(self, grid, field) 59 else: 60 # We only read the one set and do not store it if it isn't pre-loaded ---> 61 return self._read_data_set(grid, field) 62 63 def peek(self, grid, field):
/home/cshi31/yt-x86_64/src/yt-hg/yt/frontends/enzo/io.pyc in _read_data_set(self, grid, field) 183 def _read_data_set(self, grid, field): 184 return self.modify(hdf5_light_reader.ReadData(grid.filename, --> 185 "/Grid%08i/%s" % (grid.id, field))) 186 187 def _read_data_slice(self, grid, field, axis, coord):
ReadingError: ReadHDF5DataSet: Unable to open /home/cshi31/research/SG256-RP/extract0062.cpu0000
Can anyone help me on this? Thanks very much.
Chao -------------- next part -------------- An HTML attachment was scrubbed... URL: < http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/2012...
------------------------------
Message: 2 Date: Tue, 9 Oct 2012 00:19:02 -0700 From: Geoffrey So
To: Discussion of the yt analysis package Subject: Re: [yt-users] Error when open hdf5 file in yt Message-ID: Content-Type: text/plain; charset="iso-8859-1" Is this an Enzo sim? It is conceivable to have no particles in that grid. I've seen others encounter some grids where the hdf5 file is missing Dark_Matter_Density field because there were no DM particles in that particular grid.
You can confirm if the particle_type or Dark_Matter_Density field is missing by typing
h5ls extract0062.cpu####/Grid######## where the #'s correspond to the file number which has the EnzoGrid_0056 From G.S. On Mon, Oct 8, 2012 at 8:23 PM, Chao Shi
wrote:
Hi G.S., thanks for the reply. There's no Dark_Matter_Density field in this grid as well as all others. But also, there're two elements in the particle attributes in EnzoGrid_0056 when I used h5ls to inspect. It's wired why yt can't load it. And I have encountered grid that has no particles in it, in which case yt just returned an empty array but didn't give any error. Thanks, Chao
Hi all,
I'm trying to modify the particle_type of some particles in a hdf5 data file extract0062.cpu0000 in yt, typing the following command:
for g in pf.h.grids: lbhs =
np.logical_and(np.logical_and(g['particle_type']==1,g['particle_index']>=256**3),g['ParticleMassMsun']>1)
# These two lines are to pick out bhs = np.where(lbhs)[0]
# particles that I wanted to modify if(np.any(g['particle_type'])): g['particle_type'][bhs]=9 f=h5py.File(g.filename) ff = f['/Grid%08i'%g.id]['particle_type'] ff.write_direct(g['particle_type']) f.close()
It spited out following error information:
ReadingError Traceback (most recent call
last)
/home/cshi31/yt-x86_64/src/yt-hg/yt/mods.pyc in <module>() 1 for g in pf.h.grids: ----> 2 lbhs =
np.logical_and(np.logical_and(g['particle_type']==1,g['particle_index']>=256**3),g['ParticleMassMsun']>1)
3 bhs = np.where(lbhs)[0] 4 if(np.any(g['particle_type'])): 5 g['particle_type'][bhs]=9
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in __getitem__(self, key) 155 """ 156 if not self.field_data.has_key(key): --> 157 self.get_data(key) 158 return self.field_data[key] 159
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in get_data(self, field, convert) 188 return self.field_data[field] 189 try: --> 190 temp = self.hierarchy.io.pop(self, field) 191 self[field] = na.multiply(temp, conv_factor, temp) 192 except self.hierarchy.io._read_exception, exc:
/home/cshi31/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in pop(self, grid, field) 59 else: 60 # We only read the one set and do not store it if it isn't pre-loaded ---> 61 return self._read_data_set(grid, field) 62 63 def peek(self, grid, field):
/home/cshi31/yt-x86_64/src/yt-hg/yt/frontends/enzo/io.pyc in _read_data_set(self, grid, field) 183 def _read_data_set(self, grid, field): 184 return self.modify(hdf5_light_reader.ReadData(grid.filename, --> 185 "/Grid%08i/%s" % (grid.id, field))) 186 187 def _read_data_slice(self, grid, field, axis, coord):
ReadingError: ReadHDF5DataSet: Unable to open /home/cshi31/research/SG256-RP/extract0062
So I tried to look where the loop stopped, typed g in the command:
In [37]: g Out[37]: EnzoGrid_0056
so it stopped at the 56th grid in that file. I tried to see what
particles
is contained in this grid, typing g['particle_type'], it turned out this is what caused the problem:
In [31]: g['particle_type']
---------------------------------------------------------------------------
ReadingError Traceback (most recent call last) /home/cshi31/yt-x86_64/src/yt-hg/yt/mods.pyc in <module>() ----> 1 g['particle_type']
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in __getitem__(self, key) 155 """ 156 if not self.field_data.has_key(key): --> 157 self.get_data(key) 158 return self.field_data[key] 159
/home/cshi31/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.pyc in get_data(self, field, convert) 188 return self.field_data[field] 189 try: --> 190 temp = self.hierarchy.io.pop(self, field) 191 self[field] = na.multiply(temp, conv_factor, temp) 192 except self.hierarchy.io._read_exception, exc:
/home/cshi31/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in pop(self, grid, field) 59 else: 60 # We only read the one set and do not store it if it isn't pre-loaded ---> 61 return self._read_data_set(grid, field) 62 63 def peek(self, grid, field):
/home/cshi31/yt-x86_64/src/yt-hg/yt/frontends/enzo/io.pyc in _read_data_set(self, grid, field) 183 def _read_data_set(self, grid, field): 184 return self.modify(hdf5_light_reader.ReadData(grid.filename, --> 185 "/Grid%08i/%s" % (grid.id, field))) 186 187 def _read_data_slice(self, grid, field, axis, coord):
ReadingError: ReadHDF5DataSet: Unable to open /home/cshi31/research/SG256-RP/extract0062.cpu0000
Can anyone help me on this? Thanks very much.
Chao
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-------------- next part -------------- An HTML attachment was scrubbed... URL: < http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/2012...
------------------------------
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
End of yt-users Digest, Vol 56, Issue 12 ****************************************
Hi Chao,
Hi G.S., thanks for the reply. There's no Dark_Matter_Density field in this grid as well as all others. But also, there're two elements in the particle attributes in EnzoGrid_0056 when I used h5ls to inspect. It's wired why yt can't load it. And I have encountered grid that has no particles in it, in which case yt just returned an empty array but didn't give any error.
Thanks,
Chao
This does seem like odd behavior which I don't have an immediate understanding of or solution for, particularly since the problem does not seem to be in yt, but rather in the h5py code that opens a grid. What happens if you start your iteration at grid 50 or so, and continue from there? Reading the source code itself, this error: ReadingError: ReadHDF5DataSet: Unable to open /home/cshi31/research/SG256-RP/extract0062.cpu0000 will only be thrown if there is an error in how the HDF5 files are opened, specifically if the H5Fopen call fails. Are you able to open this using h5py? This code, run in isolation, should test that: import h5py f = h5py.File("/home/cshi31/research/SG256-RP/extract0062.cpu0000") print f["/Grid00000056/particle_type"][:] It would also be helpful to check that you can do: g = pf.h.grids[55] # indexed-by-1 in ids f = h5py.File(g.filename) print f["/Grid00000056/"].keys() print f["/Grid00000056/particle_type"][:] Thanks, -Matt PS It's much easier for us to keep up an email discussion if you change from digest to regular delivery when you post to the list. That way, we don't get the full digest, and email threading works both in email clients and in the archive view. It will also help people following along to track the progress of the conversation.
participants (2)
-
Chao Shi
-
Matthew Turk