[Python-Dev] new features for 2.3?

[Guido van Rossum]

> > We can do that in Python 2.3.  Because this is backwards incompatible,
> > I propose that you have to request this protocol explicitly.  I
> > propose to "upgrade' the binary flag to a general "protocol version"
> > flag, with values:
> > 
> > 0 - original protocol
> > 1 - binary protocol
> > 2 - new protocol
> 
> If you're going to do this, why not go all the way, and encode the
> python version in the protocol? The value to use could be encoded as
> constants in pickle.py (and cPickle), with a CURRENT_PROTOCOL or
> some such to make things easy.
> 
> It might also be an idea to allow passing `sys.version_info`.
> 
> The thing I'm trying to avoid is a proliferation of magic numbers
> when pickling ... what happens when we get another
> backwards-incompatible feature ... do we bump the number up to 3 and
> call it 'newer protocol'?
> 
> I think using pickle constants (be they version numbers or
> otherwise) is the way to go here.

No way.  If any kind of version number is encoded in a pickle, it
should be the pickle protocol version.  Unlike marshalled data,
pickles are guraranteed future-proof, and also backwards compatible,
unless the pickled data itself depends on Python features (for
example, an instance of a list subclass pickled in Python 2.2 can't be
unpickled in older Python versions).  Each subsequent pickle protocol
is a superset of the previous protocol.

There are use cases that have many pickles containing very small
amounts of data, where adding a version header to each pickle would
amount to a serious increase of data size.  Maybe we can afford adding
one byte to indicate the new version number (so that unpicklers that
don't speak the new protocol will die cleanly), but I think that's
about it.

--Guido van Rossum (home page: http://www.python.org/~guido/)