[AstroPy] new pyfits version deletes NP_pyfits, breaking pickle
cohen at lpta.in2p3.fr
Thu Nov 11 18:13:20 EST 2010
well there is ROOT : http://root.cern.ch
but it might represent too large a stretch.....
On 11/11/2010 11:57 PM, Joe Harrington wrote:
> Erik, thanks for the unpickler code. We saw the note about those but
> don't offhand know how to use it to fix the problem.
> Perry, thanks for looking into it.
> Also, if you know of *any* other way to save an object, please say.
> It seems pretty clear to me that to load objects from any kind of save
> file, you have to import the classes of the object and any objects it
> contains that are not standard Python objects. So even if we had
> other methods for saving, they would have the same problem as pickle.
> But we have to be able to save objects! Perhaps saving the definitions
> of the types rather than importing them would be the way to go? I bet
> there's a long thread about this somewhere.
> On Thu, 11 Nov 2010 14:38:38 -0500, Perry Greenfield<perry at stsci.edu> wrote:
>> We'll look into it. This is a general problem with pickles (and one
>> reason I've been hesitant to avoid using them like save files). I
>> wonder if there is a better solution than that. In this case we had to
>> clean out the previous numarray interface.
>> On Nov 10, 2010, at 7:24 PM, Joe Harrington wrote:
>>> My research group uses Python pickles to save data as it goes through
>>> our pipeline (.npy and .npz do not save objects, and neither does HDF,
>>> etc.). These need to be loadable forever, as we often compare work to
>>> work done much earlier. Some of the objects we save contain pyfits
>>> header objects. Pickles have to import all classes used in the
>>> pickled objects before they load, and we are getting an ImportError
>>> about NP_pyfits. The file NP_pyfits.py existed in stsci_python 2.8
>>> but is gone in 2.10. The pickles refer to this object explicitly:
>>> Is there any way to make our pickles readable again, other than
>>> running the old version of pyfits forever? Can you provide a pickle
>>> converter that replaces the old names in the file with whatever is
>>> Please (everyone, not just STScI) be aware of this issue going
>>> forward. Pickles are the only way we know of to save objects. You
>>> can add things to your classes, but if you change what they import (or
>>> otherwise break pickle), nobody can restore your class across
> AstroPy mailing list
> AstroPy at scipy.org
More information about the AstroPy