Hi Matt,
Great -- that now works for me and I've approved the pull (but only tested it on new pickles).
For a testing script, is there a specific format? If so (and I should know about it!) could I be pointed to the appropriate thread / example for writing one?
Elizabeth
On Mar 15, 2013, at 1:01 PM, Matthew Turk matthewturk@gmail.com wrote:
Hi Elizabeth,
Okay, I have a fix. I think. Unfortunately, pickle files are not currently tested -- they were, once upon a time, but are not now. Would you be willing to help out and convert this script to a testing script? I think it draws out some important things.
The code I came up with is not terribly pretty. Unfortunately, keeping backwards compatibility for pickle files means making a few compromises in the niceness of the code. So hopefully this will work, and not break a bunch of pre-existing pickles. If it does (paging David Collins) then it's a bug and we cannot accept the PR.
Here's the pull request:
https://bitbucket.org/yt_analysis/yt/pull-request/463/attempting-to-create-a...
Let me know if it works for you. The changeset hash to pull:
hg pull -r 15d72caf0ea6 https://bitbucket.org/MatthewTurk/yt/
-Matt
On Fri, Mar 15, 2013 at 11:59 AM, Matthew Turk matthewturk@gmail.com wrote:
Hi Elizabeth,
Okay -- cool. I think I will work around the h5py thing for now. But it is worrisome; I think save_object is currently broken as h5py changed how it deals with VLEN strings between 2.0 and 2.1.
I've reproduced it by using Pickle by itself. More as I get it.
-Matt
On Fri, Mar 15, 2013 at 11:25 AM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Hi Matt,
According to the directory I have in yt, an older version:
/Users/Elizabeth/yt/src/h5py-2.0.1/
Elizabeth
On Mar 15, 2013, at 10:04 AM, Matthew Turk matthewturk@gmail.com wrote:
Hi Elizabeth,
On Wed, Mar 13, 2013 at 4:45 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Hi Matt,
So ironically, it took me a long time to find the right combination to make this break in the test script! The fatal combination seems to be an extract region from a boolean object.
This is the script I used:
http://paste.yt-project.org/show/3228/
It should work with any data set.
It creates 6 saved objects. Objects 1-4 work fine but 5 and 6 have the attribute error.
I'm getting h5py errors when I even run the script. Which version of h5py are you on?
> h5py.version.version
'2.1.0'
Elizabeth
On Mar 13, 2013, at 3:08 PM, Matthew Turk matthewturk@gmail.com wrote:
Hi Elizabeth,
Okay, this seems like it's bumped up against a problem with the way yt handles pickles. At some point in the reconstruction, data objects aren't getting separated from their parameter files.
Any chance you could make something that reproduces this on one of the workshop datasets?
-Matt
On Wed, Mar 13, 2013 at 3:03 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote: > Hi, > > I'm struggling to unpickle / load_object a previously pickled / save_object. > > I've done: > > pf.h.save_object(allclouds, "clouds_pyfindclouds2_yt") > > but when I do: > > clouds = pf.h.load_object("clouds_pyfindclouds2_yt") > > I get: > > AttributeError Traceback (most recent call last) > /Users/Elizabeth/yt/src/yt-hg/scripts/iyt in <module>() > ----> 1 execfile("cloud_surfacedensity_b.py") > > /Users/Elizabeth/yt/src/yt-hg/scripts/iyt in <module>() > 13 #file.close() > 14 > ---> 15 clouds = pf.h.load_object("clouds_pyfindclouds2_yt") > > /Users/Elizabeth/yt/src/yt-hg/yt/data_objects/hierarchy.pyc in load_object(self, name) > 328 if obj is None: > 329 return > --> 330 obj = cPickle.loads(obj.value) > 331 if iterable(obj) and len(obj) == 2: > 332 obj = obj[1] # Just the object, not the pf > > /Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in _reconstruct_object(*args, **kwargs) > 4623 pf = pfs.get_pf_hash(pfid) > 4624 cls = getattr(pf.h, dtype) > -> 4625 obj = cls(*new_args) > 4626 obj.field_parameters.update(field_parameters) > 4627 return pf, obj > > /Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in __init__(self, regions, fields, pf, **kwargs) > 4046 self._all_overlap = [] > 4047 self._cut_masks = {} > -> 4048 self._get_all_regions() > 4049 self._make_overlaps() > 4050 self._get_list_of_grids() > > /Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in _get_all_regions(self) > 4057 self._all_regions.append(item) > 4058 # So cut_masks don't get messed up. > -> 4059 item._boolean_touched = True > 4060 self._all_regions = np.unique(self._all_regions) > 4061 > > AttributeError: ("'tuple' object has no attribute '_boolean_touched'", <function _reconstruct_object at 0x101dcd578>, ('73850deee907adf5217f43156c5057cd', 'boolean', [(GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240, AMRCylinder (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240): center=[ 16. 16. 16.], _norm_vec=[ 0. 0. 1.], _radius=7.5, _height=0.100000208264), 'NOT', (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240, AMRCylinder (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240): center=[ 16. 16. 16.], _norm_vec=[ 0. 0. 1.], _radius=4.5, _height=0.100000208264)], {'bulk_velocity': array([ 0., 0., 0.]), 'center': array([ 0.5, 0.5, 0.5]), 'normal': array([ 0., 0., 1.])})) > > > > This is all on the same computer. > > Elizabeth > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org