[SciPy-user] future-safe saving of numpy arrays?

[Travis Oliphant]

Cory Davis wrote:

> Hi All,
>
> I have had some trouble with my data and changes to numpy over time.  
> I often want to save both single arrays and complicated objects with 
> arrays as data members.  Until now I have almost always used cPickle. 
> But this can cause problems when I upgrade numpy/scipy, when I can no 
> longer unpickle data saved using older versions.  

Unfortunately, there were some bugs in the NumPy reduce implementation 
that required small changes.   Post 1.0, there will not be significant 
changes made to pickle that cause old pickles not to load (I don't seen 
any changes happening from now on, frankly).  This is definitely a 
growing pain of the pre-1.0 release.

> Does anyone have any suggestions on avoiding this problem?

A problem with Pickle generally, is that if you pickle objects requiring 
specific modules, then any name changes in those modules will cause 
difficulties with loading (most of these problems can be worked around 
--- often trivially), but it does get to be a pain for long-term 
persistence with Pickle.   Using PyTables is probably a better idea.

-Travis