Problems with pickle
Hi,
I'm getting a problem with pickle that I've not seen before.
I have an extracted_region that I'm trying to pickle via:
file = open(('cloud%d_pyfindclouds2.pickle' % p), 'wb') cPickle.dump(cloud, file, protocol=-1) file.close()
but I'm getting:
In [16]: cPickle.dump(cloud, file, protocol=-1) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /Users/Elizabeth/yt/src/yt-hg/scripts/iyt in <module>() ----> 1 cPickle.dump(cloud, file, protocol=-1)
/Users/Elizabeth/yt/src/yt-hg/yt/data_objects/data_containers.pyc in __reduce__(self) 403 def __reduce__(self): 404 args = tuple([self.pf._hash(), self._type_name] + --> 405 [getattr(self, n) for n in self._con_args] + 406 [self.field_parameters]) 407 return (_reconstruct_object, args)
AttributeError: 'AMRBooleanRegion' object has no attribute 'r'
Does anyone know what might be causing this?
Elizabeth
Hi Elizabeth,
I'm getting a problem with pickle that I've not seen before.
Have you tried save_object / load_object?
http://yt-project.org/doc/analyzing/objects.html#storing-and-loading-objects
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Stephen,
That method gives me exactly the same error.
Either I'm doing something silly (quite probable) or perhaps there's been a change to this machinery since the last update?
Elizabeth
On Mar 4, 2013, at 10:18 PM, Stephen Skory s@skory.us wrote:
Hi Elizabeth,
I'm getting a problem with pickle that I've not seen before.
Have you tried save_object / load_object?
http://yt-project.org/doc/analyzing/objects.html#storing-and-loading-objects
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Just as an update to this problem:
It does stem from the fact that the data was a boolean object in its dim and distant past:
diskouter = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 7.5, 100e-3/pf['kpc']) diskinner = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 4.5, 100e-3/pf['kpc']) diskdata = pf.h.boolean([diskouter, "NOT", diskinner]) . . . contours = diskdata.extract_connected_sets("Density", 1, cloud_threshold, maxv+1, log_space=True, cache=True) this_agglomeration = contours[1][0][a] . . . cloud = this_agglomeration.extract_region(agglomeration_peak_assign == p) . . . cPickle.dump(cloud, file, protocol=-1)
I also did a very old roll back and confirmed this isn't a new problem; I must have never tried to save an object that was at one point a boolean. Is there any way around this? (Apart from not using the boolean?)
Thanks!
Elizabeth
On Mar 4, 2013, at 10:29 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Hi Stephen,
That method gives me exactly the same error.
Either I'm doing something silly (quite probable) or perhaps there's been a change to this machinery since the last update?
Elizabeth
On Mar 4, 2013, at 10:18 PM, Stephen Skory s@skory.us wrote:
Hi Elizabeth,
I'm getting a problem with pickle that I've not seen before.
Have you tried save_object / load_object?
http://yt-project.org/doc/analyzing/objects.html#storing-and-loading-objects
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
(So e.g. trying to pickle disk data fails, but disk outer is fine)
On Mar 4, 2013, at 11:39 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Just as an update to this problem:
It does stem from the fact that the data was a boolean object in its dim and distant past:
diskouter = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 7.5, 100e-3/pf['kpc']) diskinner = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 4.5, 100e-3/pf['kpc']) diskdata = pf.h.boolean([diskouter, "NOT", diskinner]) . . . contours = diskdata.extract_connected_sets("Density", 1, cloud_threshold, maxv+1, log_space=True, cache=True) this_agglomeration = contours[1][0][a] . . . cloud = this_agglomeration.extract_region(agglomeration_peak_assign == p) . . . cPickle.dump(cloud, file, protocol=-1)
I also did a very old roll back and confirmed this isn't a new problem; I must have never tried to save an object that was at one point a boolean. Is there any way around this? (Apart from not using the boolean?)
Thanks!
Elizabeth
On Mar 4, 2013, at 10:29 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Hi Stephen,
That method gives me exactly the same error.
Either I'm doing something silly (quite probable) or perhaps there's been a change to this machinery since the last update?
Elizabeth
On Mar 4, 2013, at 10:18 PM, Stephen Skory s@skory.us wrote:
Hi Elizabeth,
I'm getting a problem with pickle that I've not seen before.
Have you tried save_object / load_object?
http://yt-project.org/doc/analyzing/objects.html#storing-and-loading-objects
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Elizabeth,
I've taken a look and I think I see what the issue is.
_con_args is a property of AMRData objects that describes the "construction arguments." These are the arguments necessary to rebuild a data object from its basic components -- so for a sphere, this would be the center and the radius. This is what yt sends to Pickle. It looks like con_args is somehow broken for AMRBooleanRegion.
I believe there is a typo in the AMRBooleanRegion object. On line 4039 (wow this file needs to be split up) of yt/data_objects/data_containers.py, there is a line that looks like this:
_con_args = ("regions")
I believe this should be:
_con_args = ("regions",)
This will change iteration behavior from:
for i in ("regions"): print i
... r e g i o n s
to:
for i in ("regions",): print i
... regions
If that works, would you mind submitting a quick pull request?
-Matt
On Mon, Mar 4, 2013 at 11:40 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
(So e.g. trying to pickle disk data fails, but disk outer is fine)
On Mar 4, 2013, at 11:39 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Just as an update to this problem:
It does stem from the fact that the data was a boolean object in its dim and distant past:
diskouter = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 7.5, 100e-3/pf['kpc']) diskinner = pf.h.disk(na.array([16.0, 16.0, 16.0]), na.array([0.0,0.0,1.0]), 4.5, 100e-3/pf['kpc']) diskdata = pf.h.boolean([diskouter, "NOT", diskinner]) . . . contours = diskdata.extract_connected_sets("Density", 1, cloud_threshold, maxv+1, log_space=True, cache=True) this_agglomeration = contours[1][0][a] . . . cloud = this_agglomeration.extract_region(agglomeration_peak_assign == p) . . . cPickle.dump(cloud, file, protocol=-1)
I also did a very old roll back and confirmed this isn't a new problem; I must have never tried to save an object that was at one point a boolean. Is there any way around this? (Apart from not using the boolean?)
Thanks!
Elizabeth
On Mar 4, 2013, at 10:29 PM, Elizabeth Tasker tasker@astro1.sci.hokudai.ac.jp wrote:
Hi Stephen,
That method gives me exactly the same error.
Either I'm doing something silly (quite probable) or perhaps there's been a change to this machinery since the last update?
Elizabeth
On Mar 4, 2013, at 10:18 PM, Stephen Skory s@skory.us wrote:
Hi Elizabeth,
I'm getting a problem with pickle that I've not seen before.
Have you tried save_object / load_object?
http://yt-project.org/doc/analyzing/objects.html#storing-and-loading-objects
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Elizabeth Tasker -
Matthew Turk -
Stephen Skory