Hi Elizabeth,
Okay, I have a fix. I think. Unfortunately, pickle files are not
currently tested -- they were, once upon a time, but are not now.
Would you be willing to help out and convert this script to a testing
script? I think it draws out some important things.
The code I came up with is not terribly pretty. Unfortunately,
keeping backwards compatibility for pickle files means making a few
compromises in the niceness of the code. So hopefully this will work,
and not break a bunch of pre-existing pickles. If it does (paging
David Collins) then it's a bug and we cannot accept the PR.
Here's the pull request:
https://bitbucket.org/yt_analysis/yt/pull-request/463/attempting-to-create-a...
Let me know if it works for you. The changeset hash to pull:
hg pull -r 15d72caf0ea6 https://bitbucket.org/MatthewTurk/yt/
-Matt
On Fri, Mar 15, 2013 at 11:59 AM, Matthew Turk
Hi Elizabeth,
Okay -- cool. I think I will work around the h5py thing for now. But it is worrisome; I think save_object is currently broken as h5py changed how it deals with VLEN strings between 2.0 and 2.1.
I've reproduced it by using Pickle by itself. More as I get it.
-Matt
On Fri, Mar 15, 2013 at 11:25 AM, Elizabeth Tasker
wrote: Hi Matt,
According to the directory I have in yt, an older version:
/Users/Elizabeth/yt/src/h5py-2.0.1/
Elizabeth
On Mar 15, 2013, at 10:04 AM, Matthew Turk
wrote: Hi Elizabeth,
On Wed, Mar 13, 2013 at 4:45 PM, Elizabeth Tasker
wrote: Hi Matt,
So ironically, it took me a long time to find the right combination to make this break in the test script! The fatal combination seems to be an extract region from a boolean object.
This is the script I used:
http://paste.yt-project.org/show/3228/
It should work with any data set.
It creates 6 saved objects. Objects 1-4 work fine but 5 and 6 have the attribute error.
I'm getting h5py errors when I even run the script. Which version of h5py are you on?
h5py.version.version '2.1.0'
Elizabeth
On Mar 13, 2013, at 3:08 PM, Matthew Turk
wrote: Hi Elizabeth,
Okay, this seems like it's bumped up against a problem with the way yt handles pickles. At some point in the reconstruction, data objects aren't getting separated from their parameter files.
Any chance you could make something that reproduces this on one of the workshop datasets?
-Matt
On Wed, Mar 13, 2013 at 3:03 PM, Elizabeth Tasker
wrote: Hi,
I'm struggling to unpickle / load_object a previously pickled / save_object.
I've done:
pf.h.save_object(allclouds, "clouds_pyfindclouds2_yt")
but when I do:
clouds = pf.h.load_object("clouds_pyfindclouds2_yt")
I get:
AttributeError Traceback (most recent call last) /Users/Elizabeth/yt/src/yt-hg/scripts/iyt in <module>() ----> 1 execfile("cloud_surfacedensity_b.py")
/Users/Elizabeth/yt/src/yt-hg/scripts/iyt in <module>() 13 #file.close() 14 ---> 15 clouds = pf.h.load_object("clouds_pyfindclouds2_yt")
/Users/Elizabeth/yt/src/yt-hg/yt/data_objects/hierarchy.pyc in load_object(self, name) 328 if obj is None: 329 return --> 330 obj = cPickle.loads(obj.value) 331 if iterable(obj) and len(obj) == 2: 332 obj = obj[1] # Just the object, not the pf
/Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in _reconstruct_object(*args, **kwargs) 4623 pf = pfs.get_pf_hash(pfid) 4624 cls = getattr(pf.h, dtype) -> 4625 obj = cls(*new_args) 4626 obj.field_parameters.update(field_parameters) 4627 return pf, obj
/Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in __init__(self, regions, fields, pf, **kwargs) 4046 self._all_overlap = [] 4047 self._cut_masks = {} -> 4048 self._get_all_regions() 4049 self._make_overlaps() 4050 self._get_list_of_grids()
/Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in _get_all_regions(self) 4057 self._all_regions.append(item) 4058 # So cut_masks don't get messed up. -> 4059 item._boolean_touched = True 4060 self._all_regions = np.unique(self._all_regions) 4061
AttributeError: ("'tuple' object has no attribute '_boolean_touched'",
, ('73850deee907adf5217f43156c5057cd', 'boolean', [(GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240, AMRCylinder (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240): center=[ 16. 16. 16.], _norm_vec=[ 0. 0. 1.], _radius=7.5, _height=0.100000208264), 'NOT', (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240, AMRCylinder (GR_Enzo2_128amr5_zeus_turb_psupp_rr_0240): center=[ 16. 16. 16.], _norm_vec=[ 0. 0. 1.], _radius=4.5, _height=0.100000208264)], {'bulk_velocity': array([ 0., 0., 0.]), 'center': array([ 0.5, 0.5, 0.5]), 'normal': array([ 0., 0., 1.])})) This is all on the same computer.
Elizabeth _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org