[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data

[Terry Reedy]

On 3/28/2018 9:15 PM, Nathaniel Smith wrote:

> There's obviously some tension here between pickle's use as a
> persistent storage format, and its use as a transient wire format. For
> the former, you definitely can't store code objects because there's no
> forwards- or backwards-compatibility guarantee for bytecode. But for
> the latter, transmitting bytecode is totally fine, because all you
> care about is whether it can be decoded once, right now, by some peer
> process whose python version you can control -- that's why cloudpickle
> exists.

An interesting observation.  IDLE compiles user code in the user process 
to check for syntax errors.  idlelib.rpc subclasses Pickler to pickle 
the resulting code objects via marshal.dumps so it can send them to the 
user code execution subprocess.

> Would it make sense to have a special pickle version that the
> transient wire format users could opt into, that only promises
> compatibility within a given 3.X release cycle? Like version=-2 or
> version=pickle.NONPORTABLE or something?
> 
> (This is orthogonal to Antoine's PEP.)

-- 
Terry Jan Reedy