[Python-Dev] Accepting PEP 3154 for 3.4?

[Antoine Pitrou]

On Mon, 18 Nov 2013 18:50:01 -0600
Tim Peters <tim.peters at gmail.com> wrote:
> [Antoine]
> > Yet another possibility: keep framing but use a variable-length
> > encoding for the frame size:
> >
> > - first byte: bits 7-5: N (= frame size bytes length - 1)
> > - first byte: bits 4-0: first 5 bits of frame size
> > - remaning N bytes: remaining bits of frame size
> >
> > With this scheme, very small pickles have a one byte overhead; small
> > ones a two byte overhead; and the max frame size is 2**61 rather than
> > 2**64, which should still be sufficient.
> >
> > And the frame size is read using either one or two read() calls, which
> > is efficient.
> 
> That would be a happy compromise :-)
> 
> I'm unclear on how that would work for, e.g., encoding 40 =
> 0b000101000.  That has 6 significant bits.  Would you store 0 in the
> leading byte and 40 in the second byte?  That would work.

Yeah, I haven't decided whether it would be big-endian or
little-endian. It doesn't matter much. big-endian sounds a bit easier
to decode and encode (no bit shifts needed), it's also less consistent
with the rest of the pickle opcodes.

> 2**61 = 2,305,843,009,213,693,952 is a lot of bytes, especially for a pickle ;-)

Let's call it a biggle :-)

Regards

Antoine.