Hello, This is probably a simple thing to fix, but I've been trying to use the volume rendering package on my data and am running into an error. If I follow the steps at http://paste.enzotools.org/show/404/ when I get to mi,ma = dd.quantities["Extrema"]("Density")[0] I get the error message at the bottom of this email. It seems that it is looking for files ending in .cpu000# rather than .grid.cpu000# like the default enzo output. I've tried doing a rename but then I get the same error but the other way around. Making a copy of the file with each filename works if I'm just doing a single image, but I'm trying to make a movie so doing that for 100 files for 8 cpus each would be a bit tedious. If anyone knows a way around this or I'm doing something wrong, I'd appreciate any suggestions. Thanks -Mike In [8]: mi,ma=dd.quantities["Extrema"]("Density")[0] ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (105, 0)) ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (72, 0)) ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (119, 0)) --------------------------------------------------------------------------- ReadingError Traceback (most recent call last) /home/butler85/yt/src/yt-trunk-svn/scripts/iyt in <module>() ----> 1 2 3 4 5 /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DerivedQuantities.pyc in __call__(self, *args, **kwargs) 70 self._data_source.pf.h.io) 71 if lazy_reader and not self.force_unlazy: ---> 72 return self._call_func_lazy(args, kwargs) 73 else: 74 return self._call_func_unlazy(args, kwargs) /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DerivedQuantities.pyc in _call_func_lazy(self, args, kwargs) 77 self.retvals = [ [] for i in range(self.n_ret)] 78 for gi,g in enumerate(self._get_grids()): ---> 79 rv = self.func(GridChildMaskWrapper(g, self._data_source), *args, **kwargs) 80 for i in range(self.n_ret): self.retvals[i].append(rv[i]) 81 g.clear_data() /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DerivedQuantities.pyc in _Extrema(data, fields, filter) 385 mins, maxs = [], [] 386 for field in fields: --> 387 if data[field].size < 1: 388 mins.append(1e90) 389 maxs.append(-1e90) /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DerivedQuantities.pyc in __getitem__(self, item) 41 return getattr(self.grid, attr) 42 def __getitem__(self, item): ---> 43 return self.data_source._get_data_from_grid(self.grid, item) 44 45 class DerivedQuantity(ParallelAnalysisInterface): /home/butler85/yt/src/yt-trunk-svn/yt/lagos/BaseDataTypes.pyc in save_state(self, grid, field) 41 old_keys = grid.data.keys() 42 grid.field_parameters = self.field_parameters ---> 43 tr = func(self, grid, field) 44 grid.field_parameters = old_params 45 grid.data = dict( [(k, grid.data[k]) for k in old_keys] ) /home/butler85/yt/src/yt-trunk-svn/yt/lagos/BaseDataTypes.pyc in _get_data_from_grid(self, grid, field) 1660 else: 1661 pointI = self._get_point_indices(grid) -> 1662 if grid[field].size == 1: # dx, dy, dz, cellvolume 1663 t = grid[field] * na.ones(grid.ActiveDimensions, dtype='float64') 1664 return t[pointI].ravel() /home/butler85/yt/src/yt-trunk-svn/yt/lagos/BaseGridType.pyc in __getitem__(self, key) 133 """ 134 if not self.data.has_key(key): --> 135 self.get_data(key) 136 return self.data[key] 137 /home/butler85/yt/src/yt-trunk-svn/yt/lagos/BaseGridType.pyc in get_data(self, field) 166 return self.data[field] 167 try: --> 168 temp = self.hierarchy.io.pop(self, field) 169 self[field] = na.multiply(temp, conv_factor, temp) 170 except self.hierarchy.io._read_exception, exc: /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DataReadingFuncs.pyc in pop(self, grid, field) 55 else: 56 # We only read the one set and do not store it if it isn't pre-loaded ---> 57 return self._read_data_set(grid, field) 58 59 def peek(self, grid, field): /home/butler85/yt/src/yt-trunk-svn/yt/lagos/DataReadingFuncs.pyc in _read_data_set(self, grid, field) 232 def _read_data_set(self, grid, field): 233 return HDF5LightReader.ReadData(grid.filename, --> 234 "/Grid%08i/%s" % (grid.id, field)).swapaxes(0,2) 235 236 def _read_data_slice(self, grid, field, axis, coord): ReadingError: ReadHDF5DataSet: /home/butler85/sim/starsfuvamrrec/Cloud_128AMR4StarsFUV_0101.cpu0000 does not exist, or no read permissions
Hi Mike,
I get the error message at the bottom of this email. It seems that it is looking for files ending in .cpu000# rather than .grid.cpu000# like the default enzo output. I've tried doing a rename but then I get the same error but the other way around. Making a copy of the file with each filename works if I'm just doing a single image, but I'm trying to make a movie so doing that for 100 files for 8 cpus each would be a bit tedious.
Here's the quick fix, and then some backstory. Put this at the top of your imports: http://paste.enzotools.org/show/446/ As to the backstory: the history and family tree of Enzo is a complicated flow, and I'm not even convinced it's acyclic. There have been at least three different primary modes of IO: HDF4, HDF5, and "packed" HDF5. (This is ignoring for a moment things like AMRLib, which was also supported at one point, as well as anything that may have come before.) There have been two implementations of the "packed" HDF5 format, where each grid is an HDF5 group output by its owning processor. Unfortunately, while the internal naming scheme is the same, the naming scheme for the files is different. To my understanding, this difference in naming scheme -- including ".grid." and not including ".grid." mostly stems from the implementations from the East Coast and West Coast, respectively. When I initially implemented the packed HDF5, yt had no preconceptions about the naming scheme, and it worked independent of the inclusion of ".grid.", by simply reading the FileName parameters in the .hierarchy. However, last Summer and Fall I worked on optimizing the hierarchy for a much faster instantiation time and to clean up a lot of the cruft -- which ended up actually including a file name guessing mechanism. By then I had actually switched code bases and not even realizes that the ".grid." difference existed, so I never saw any problems until I revisited my thesis data a few months ago. That's why the change above has been added. Because there's a substantial amount of data out there that still includes ".grid." in its filenames, I will see if I can fix this behavior in the long term; when I initially applied this "hotfix" I thought it wouldn't be a problem for many or any other people. I guess that's not true, as I spoke with some people last week that also required it. However, for now I believe it should work for you and your use case. Let me know if it does not! Best, Matt
participants (2)
-
Matthew Turk -
Mike Butler