[Python-Dev] Accepting PEP 3154 for 3.4?

[Antoine Pitrou]

On Tue, 19 Nov 2013 13:22:52 -0600
Tim Peters <tim.peters at gmail.com> wrote:
> [Guido]
> > So using an opcode for framing is out? (Sorry, I've lost track of the
> > back-and-forth.)
> 
> It was never in ;-)  I'd *prefer* one, but not enough to try to block
> the PEP.  As is, framing is done at a "lower level" than opcode
> decoding.  I fear this is brittle, for all the usual "explicit is
> better than implicit" kinds of reasons.  The only way now to know that
> you're looking at a frame size is to keep a running count of bytes
> processed and realize you've reached a byte offset where a frame size
> "is expected".

That's integrated to the built-in buffering. It's not really an
additional constraint: the frame sizes simply dictate how buffering
happens in practice. The main point of framing is to *simplify* the
buffering logic (of course, the old buffering logic is still there for
protocols <= 3, unfortunately).

Note some drawbacks of frame opcodes:
- the decoder has to sanity check the frame opcodes (what if a frame
  opcode is encountered when already inside a frame?)
- a pickle-mutating function such as pickletools.optimize() may naively
  ignore the frame opcodes while rearranging the pickle stream, only to
  emit a new pickle with invalid frame sizes

Regards

Antoine.