Tis: In [1]: pf._hash() Out[1]: '283b7c4d88671dbff7acf083098da6ae' E. On 2011-12-01, at 11:57 PM, Matthew Turk wrote:
Hi Elizabeth,
What is the output of pf._hash() ?
-Matt
On Thu, Dec 1, 2011 at 9:32 AM, Elizabeth Tasker <tasker@astro1.sci.hokudai.ac.jp> wrote:
Sorry, still no luck!
The current version of the code is:
--- c1a446be3a7a (yt) tip ---
unpickle core data --------------------------------------------------------------------------- KeyError Traceback (most recent call last)
/home/taskere/yt-2/src/yt-hg/scripts/iyt in <module>() 199 200 file = open('cores.pickle','rb') --> 201 allcloudcores = cPickle.load(file) 202 file.close() 203
/home/taskere/yt-2/src/yt-hg/yt/data_objects/data_containers.pyc in _reconstruct_object(*args, **kwargs) 3640 else: new_args.append(arg) 3641 pfs = ParameterFileStore() -> 3642 pf = pfs.get_pf_hash(pfid) 3643 cls = getattr(pf.h, dtype) 3644 obj = cls(*new_args)
/home/taskere/yt-2/src/yt-hg/yt/utilities/parameter_file_storage.pyc in get_pf_hash(self, hash) 109 def get_pf_hash(self, hash): 110 """ This returns a parameter file based on a hash. """ --> 111 return self._convert_pf(self._records[hash]) 112 113 def get_pf_ctid(self, ctid):
KeyError: (('283b7c4d88671dbff7acf083098da6ae',), <function _reconstruct_object at 0xa8cd848>, ('283b7c4d88671dbff7acf083098da6ae', 'region', array([ 16., 16., 16.]), array([ 0., 0., 0.]), array([ 32., 32., 32.]), {'disk_center': array([16, 16, 16]), 'center': array([ 16., 16., 16.]), 'bulk_velocity': array([ 0., 0., 0.]), 'disk_vector': array([0, 0, 1]), 'disk_radius': array([ 0.1 , 0.1238, 0.1476, 0.1714, 0.1952, 0.219 , 0.2428, 0.2666, 0.2904, 0.3142, 0.33
Unless I need to re-pickle first?
Elizabeth
On 2011-12-01, at 8:20 PM, Matthew Turk wrote:
Thanks, Elizabeth. The move to in-memory brought up a funny bug where the initialization process was reseting the list of known parameter hashes. This is fixed in c1a446be3a7a.
Also, save_object won't work except on yt objects; the result of extract_connected_sets is a tuple of values and python objects. Pickle will, tho.
On Thu, Dec 1, 2011 at 4:33 AM, Elizabeth Tasker <tasker@astro1.sci.hokudai.ac.jp> wrote:
I also tried switching out pickle for the yt method of storing objects. Unfortunately, it doesn't seem to save the object I want. I tried:
contours = dd.extract_connected_sets("NegEscapeVelocity", 1, 30.0, maxv, log_space=False) contours.save_object("cloudcores", "storedcores.cpkl")
In [1]: contours.save_object("cloudcores", "storedcores.cpkl") --------------------------------------------------------------------------- AttributeError Traceback (most recent call last)
/home/tasker/yt/src/yt-hg/scripts/iyt in <module>() ----> 1 2 3 4 5
AttributeError: 'tuple' object has no attribute 'save_object'
The alternative method --of saving it via the hierarchy-- is slightly less good since I'm running multiple yt scripts on the same data set. (As least, when I tried to load the values back in, it produced an empty set which I presume was because I'd inadvertently overwritten the .yt file?).
If I could get one of these working again, then I'd be super happy. I don't need the all :)
Elizabeth
On 1 December 2011 12:35, Elizabeth Tasker <tasker@astro1.sci.hokudai.ac.jp> wrote:
Hi Matt,
Sorry, that still doesn't work.... unless I've misunderstood your suggestion? The code is here:
http://paste.yt-project.org/show/1974/
Pickle + unpickle on a pile of properties I calculate from the connected sets is fine, but pickling the connected sets themselves is a problem. Could it be the later pickle doesn't require the parameter file? Or is it possible there is an unrelated write error happening?
Elizabeth
On 30 November 2011 23:43, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Elizabeth,
We recently changed the mechanism of storing parameter files to be off-by-default, which means that rather than dumping them to a .csv file, they only stay in memory. This functionality can be turned back on (this change is documented in the development documentation.) Can you try:
1) Loading your parameter file, *then* loading your pickle. I am optimistic this will fix it. 2) If it doesn't, at the very top of your python script, put this:
from yt.config import ytcfg; ytcfg["yt","storeparameterfiles"] = "True"
-Matt
On Wed, Nov 30, 2011 at 4:46 AM, Elizabeth Tasker <tasker@astro1.sci.hokudai.ac.jp> wrote: > Hi, > > A bigger problem.... > > The lastest version of yt > > The current version of the code is: > > --- > a519b8754ba8 (yt) tip > --- > > > has problems with pickle. If I pickle a data set and then try and > unpickle it, I see: > > unpickle core data > --------------------------------------------------------------------------- > KeyError Traceback (most recent call last) > > /home/tasker/yt/src/yt-hg/scripts/iyt in <module>() > 338 > 339 #file = open('cores.pickle','rb') > --> 340 allcloudcores = cPickle.load(file('cores.pickle', 'rb')) > 341 file.close() > 342 > > /home/tasker/yt/src/yt-hg/yt/data_objects/data_containers.pyc in > _reconstruct_object(*args, **kwargs) > 3676 else: new_args.append(arg) > 3677 pfs = ParameterFileStore() > -> 3678 pf = pfs.get_pf_hash(pfid) > 3679 cls = getattr(pf.h, dtype) > 3680 obj = cls(*new_args) > > /home/tasker/yt/src/yt-hg/yt/utilities/parameter_file_storage.pyc in > get_pf_hash(self, hash) > 106 def get_pf_hash(self, hash): > 107 """ This returns a parameter file based on a hash. """ > --> 108 return self._convert_pf(self._records[hash]) > 109 > 110 def get_pf_ctid(self, ctid): > > KeyError: (('283b7c4d88671dbff7acf083098da6ae',), <function > _reconstruct_object at 0x272c938>, > ('283b7c4d88671dbff7acf083098da6ae', 'region', array([ 16., 16., > 16.]), array([ 0., 0., 0.]), array([ 32., 32., 32.]), > {'disk_center': array([16, 16, 16]), 'center': array([ 16., 16., > 16.]), 'bulk_velocity': array([ 0., 0., 0.]), 'disk_vector': > array([0, 0, 1]), 'disk_radius': array([ 0.1 , 0.1358, 0.1716, > 0.2074, 0.2432, 0.279 , > > > > > The fact it mentions the field parameters in the last line, might mean > it's an error introduced because of the corrections to them (that I > have been demanding!)? > > The script I have works fine (pickles and all) on yt version: > > --- > 16e8d749a806 (yt) tip > --- > > > Elizabeth > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org