Hi Basu, Sorry for the late reply, and thanks for reaching out. On Sun, Jul 18, 2021 at 9:08 AM basu <basu@mpa-garching.mpg.de> wrote:

Dear Sir/Ma'am,

To introduce myself first, I am going to start my PhD at Max Planck Institute of Astrophysics in the group of Dr. Benedetta Ciardi in few months. For my ongoing project, I need to use yt package for which I have installed yt 4.0.0 via pip pipeline. But, I am facing a difficulty in processing the file successfully loaded and read by "yt.load". I am sharing 2-3 lines of my code and the errors I get.

CODE: import yt import numpy as np import h5py

ad=ds.all_data() ##[ERROR 1 here] fields = ['density'] le = ds.domain_left_edge re = ds.domain_right_edge ag = ds.arbitrary_grid(le,re,[256]*3)

for f in fields: print(f) data[f] = ag[('gas', f)] bbox = np.array(list(zip(le.in_units('Mpccm/h'), re.in_units('Mpccm/h')))) ds_gridded = yt.load_uniform_grid(data, [256]*3, length_unit = ds.length_unit.to('Mpccm/h'), bbox=bbox) sp = yt.SlicePlot(ds_gridded,'z',('gas','density')) sp.save() ##[ERROR 2 here]

ERROR:1: RuntimeError Traceback (most recent call last) <ipython-input-4-007efe033704> in <module> ----> 1 ad=ds.all_data()

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in all_data(self, find_max, **kwargs) 1039 which covers the entire simulation domain. 1040 """ -> 1041 self.index 1042 if find_max: 1043 c = self.find_max("density")[1]

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in index(self) 526 if self._index_class is None: 527 raise RuntimeError("You should not instantiate Dataset.") --> 528 self._instantiated_index = self._index_class( 529 self, dataset_type=self.dataset_type 530 )

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in __init__(self, ds, dataset_type) 22 self.dataset = weakref.proxy(ds) 23 self.float_type = np.float64 ---> 24 super().__init__(ds, dataset_type) 25 self._initialize_index() 26

~/.local/lib/python3.8/site-packages/yt/geometry/geometry_handler.py in __init__(self, ds, dataset_type) 42 # potentially quite expensive, and should be done with the indexing. 43 mylog.debug("Detecting fields.") ---> 44 self._detect_output_fields() 45 46 @abc.abstractmethod

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in _detect_output_fields(self) 296 fl, _units = field_cache[dom.filename] 297 else: --> 298 fl, _units = self.io._identify_fields(dom) 299 field_cache[dom.filename] = fl, _units 300 units.update(_units)

~/.local/lib/python3.8/site-packages/yt/frontends/gadget/ioKeyError Traceback (most recent call last) <ipython-input-6-97fe8b4b4ff8> in <module> 1 for f in fields: 2 print(f) ----> 3 data[f] = ag[('gas', f)] 4 bbox = np.array(list(zip(le.in_units('Mpccm/h'), re.in_units('Mpccm/h')))) 5 ds_gridded = yt.load_uniform_grid(data, [256]*3, length_unit = ds.length_unit.to('Mpccm/h'), bbox=bbox)

~/.local/lib/python3.8/site-packages/yt/data_objects/data_containers.py in __getitem__(self, key) 250 Returns a single field. Will add if necessary. 251 """ --> 252 f = self._determine_fields([key])[0] 253 if f not in self.field_data and key not in self.field_data: 254 if f in self._container_fields:

~/.local/lib/python3.8/site-packages/yt/data_objects/data_containers.py in _determine_fields(self, fields) 1450 ftype, fname = self._tupleize_field(field) 1451 # print(field, " : ",ftype, fname) -> 1452 finfo = self.ds._get_field_info(ftype, fname) 1453 1454 # really ugly check to ensure that this field really does exist somewhere,

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in _get_field_info(self, ftype, fname) 817 818 def _get_field_info(self, ftype, fname=None): --> 819 field_info, is_ambiguous = self._get_field_info_helper(ftype, fname) 820 821 if is_ambiguous:

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in _get_field_info_helper(self, ftype, fname) 833 834 def _get_field_info_helper(self, ftype, fname=None): --> 835 self.index 836 837 # store the original inputs in case we need to raise an error

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in index(self) 526 if self._index_class is None: 527 raise RuntimeError("You should not instantiate Dataset.") --> 528 self._instantiated_index = self._index_class( 529 self, dataset_type=self.dataset_type 530 )

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in __init__(self, ds, dataset_type) 22 self.dataset = weakref.proxy(ds) 23 self.float_type = np.float64 ---> 24 super().__init__(ds, dataset_type) 25 self._initialize_index() 26

~/.local/lib/python3.8/site-packages/yt/geometry/geometry_handler.py in __init__(self, ds, dataset_type) 42 # potentially quite expensive, and should be done with the indexing. 43 mylog.debug("Detecting fields.") ---> 44 self._detect_output_fields() 45 46 @abc.abstractmethod

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in _detect_output_fields(self) 296 fl, _units = field_cache[dom.filename] 297 else: --> 298 fl, _units = self.io._identify_fields(dom) 299 field_cache[dom.filename] = fl, _units 300 units.update(_units)

~/.local/lib/python3.8/site-packages/yt/frontends/gadget/io.py in _identify_fields(self, data_file) 290 else: 291 kk = k --> 292 if not hasattr(g[kk], "shape"): 293 continue 294 if len(g[kk].shape) > 1:

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

~/.local/lib/python3.8/site-packages/h5py/_hl/group.py in __getitem__(self, name) 286 raise ValueError("Invalid HDF5 object reference") 287 else: --> 288 oid = h5o.open(self.id, self._e(name), lapl=self._lapl) 289 290 otype = h5i.get_type(oid)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5o.pyx in h5py.h5o.open()

KeyError: 'Unable to open object (bad object header version number)'.py in _identify_fields(self, data_file) 261 # -------------------------------------- 262 g = f[key] --> 263 if cname not in g: 264 continue 265

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

~/.local/lib/python3.8/site-packages/h5py/_hl/group.py in __contains__(self, name) 437 def __contains__(self, name): 438 """ Test if a member name exists """ --> 439 return self._e(name) in self.id 440 441 def copy(self, source, dest, name=None,

h5py/h5g.pyx in h5py.h5g.GroupID.__contains__()

h5py/h5g.pyx in h5py.h5g.GroupID.__contains__()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5g.pyx in h5py.h5g._path_valid()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5l.pyx in h5py.h5l.LinkProxy.exists()

RuntimeError: Unable to get link info (free block size is zero?)

ERROR:2: KeyError Traceback (most recent call last) <ipython-input-6-97fe8b4b4ff8> in <module> 1 for f in fields: 2 print(f) ----> 3 data[f] = ag[('gas', f)] 4 bbox = np.array(list(zip(le.in_units('Mpccm/h'), re.in_units('Mpccm/h')))) 5 ds_gridded = yt.load_uniform_grid(data, [256]*3, length_unit = ds.length_unit.to('Mpccm/h'), bbox=bbox)

~/.local/lib/python3.8/site-packages/yt/data_objects/data_containers.py in __getitem__(self, key) 250 Returns a single field. Will add if necessary. 251 """ --> 252 f = self._determine_fields([key])[0] 253 if f not in self.field_data and key not in self.field_data: 254 if f in self._container_fields:

~/.local/lib/python3.8/site-packages/yt/data_objects/data_containers.py in _determine_fields(self, fields) 1450 ftype, fname = self._tupleize_field(field) 1451 # print(field, " : ",ftype, fname) -> 1452 finfo = self.ds._get_field_info(ftype, fname) 1453 1454 # really ugly check to ensure that this field really does exist somewhere,

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in _get_field_info(self, ftype, fname) 817 818 def _get_field_info(self, ftype, fname=None): --> 819 field_info, is_ambiguous = self._get_field_info_helper(ftype, fname) 820 821 if is_ambiguous:

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in _get_field_info_helper(self, ftype, fname) 833 834 def _get_field_info_helper(self, ftype, fname=None): --> 835 self.index 836 837 # store the original inputs in case we need to raise an error

~/.local/lib/python3.8/site-packages/yt/data_objects/static_output.py in index(self) 526 if self._index_class is None: 527 raise RuntimeError("You should not instantiate Dataset.") --> 528 self._instantiated_index = self._index_class( 529 self, dataset_type=self.dataset_type 530 )

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in __init__(self, ds, dataset_type) 22 self.dataset = weakref.proxy(ds) 23 self.float_type = np.float64 ---> 24 super().__init__(ds, dataset_type) 25 self._initialize_index() 26

~/.local/lib/python3.8/site-packages/yt/geometry/geometry_handler.py in __init__(self, ds, dataset_type) 42 # potentially quite expensive, and should be done with the indexing. 43 mylog.debug("Detecting fields.") ---> 44 self._detect_output_fields() 45 46 @abc.abstractmethod

~/.local/lib/python3.8/site-packages/yt/geometry/particle_geometry_handler.py in _detect_output_fields(self) 296 fl, _units = field_cache[dom.filename] 297 else: --> 298 fl, _units = self.io._identify_fields(dom) 299 field_cache[dom.filename] = fl, _units 300 units.update(_units)

~/.local/lib/python3.8/site-packages/yt/frontends/gadget/io.py in _identify_fields(self, data_file) 290 else: 291 kk = k --> 292 if not hasattr(g[kk], "shape"): 293 continue 294 if len(g[kk].shape) > 1:

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

~/.local/lib/python3.8/site-packages/h5py/_hl/group.py in __getitem__(self, name) 286 raise ValueError("Invalid HDF5 object reference") 287 else: --> 288 oid = h5o.open(self.id, self._e(name), lapl=self._lapl) 289 290 otype = h5i.get_type(oid)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5o.pyx in h5py.h5o.open()

KeyError: 'Unable to open object (bad object header version number)'

What I feel is that there is a mismatch in the h5py version. What is that exact problem here? I have h5py 3.0.0, matplotlib 3.4.1,python 3.8,yt 4.0.0 in my machine. Please make me understand and suggest me a reasonable solution so that I can proceed smoothly to my next step. The length of this email seems large although most of it contains some outcomes of the codes.

This agrees with my assessment -- either there's an issue with the h5py/HDF5 version, *or* there's a problem with the specific HDF5 file that is being loaded. It's not immediately obvious to me *which* file is being accessed, but you can check two different things to narrow it down. The first would be to run yt by itself, and to utilize it to load a dataset from the yt website that uses HDF5. I'd recommend these two different datasets, which should work (if you have pooch installed): ds1 = yt.load_sample("IsolatedGalaxy") print(ds1.r[:].max(("gas","density"))) ds2 = yt.load_sample("snapshot_033") print(ds2.r[:].max(("gas","density"))) Try this to see if it is able to run. The second would be to use something like the h5debug utility to determine if the file is valid; on all of the hdf5 files in the specific dataset you're loading, run h5debug filename where "filename" is the hdf5 file. This should output some information, but the principle thing we want to do is verify that hdf5 can indeed open the files. Again, sorry for the delay, and I heop this helps. -Matt

Waiting for your early reply.

Regards, Arghyadeep Basu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com