[Python-Dev] Accepting PEP 3154 for 3.4?

Tim Peters tim.peters at gmail.com
Tue Nov 19 22:41:51 CET 2013

>> ...
>> But if some _other_ implementation of unpickling didn't give a hoot
>> about framing, having an explicit opcode means that implementation
>> could ignore the whole scheme very easily:  just implement the FRAME
>> opcode in *its* opcode-decoding loop to consume the FRAME argument,
>> ignore it, and move on.  As-is, all other implementations _have_ to
>> know everything about the buffering scheme because it's all implicit
>> low-level magic.

> Ahah, ok, I see where you're going. But how many other implementations
> of unpickling are there?

That's something you should have researched when writing the PEP ;-)
How many implementations of Python aren't CPython?  That's probably
the answer.  I'm not an expert on that, but there's more than one.

>> Initially, all I desperately ;-) want changed here is for the
>> _buffering layer_, on the writing end, to write 9 bytes instead of 8
>> (1 new one for a FRAME opcode), and on the reading end to consume 9
>> bytes instead of 8 (extra credit if it checked the first byte to
>> verify it really is a FRAME opcode - there's nothing wrong with sanity
>> checks).
>> Then it becomes _possible_ to optimize "small pickles" later (in the
>> sense of not bothering to frame them at all).

> So the CPython unpickler must be able to work with and without framing
> by detecting the FRAME opcode?

Not at first, no.  At first the buffering layer could raise an
exception if there's no FRAME opcode when it expected one.  Or just
read up garbage bytes and assume it's a frame size, which is
effectively what it's doing now anyway ;-)

> Otherwise the "later optimization" can't work.

Right.  _If_ reducing framing overhead to "nothing" for small pickles
turns out to be sufficiently desirable, then the buffering layer would
need to learn how to turn itself off in the absence of a FRAME opcode
immediately following the current frame.  Perhaps the opcode decoding
loop would also need to learn how to turn the buffering layer back on
again too (next time a FRAME opcode showed up).  Sounds annoying, but
not impossible.

More information about the Python-Dev mailing list