[Python-Dev] Accepting PEP 3154 for 3.4?

[Guido van Rossum]

On Mon, Nov 18, 2013 at 4:30 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 19 November 2013 09:57, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Mon, 18 Nov 2013 16:44:59 -0600
> > Tim Peters <tim.peters at gmail.com> wrote:
> >> [Tim]
> >> >> But it has a different kind of advantage:  PREFETCH was optional.  As
> >> >> Guido said, it's annoying to bloat the size of small pickles (which
> >> >> may, although individually small, occur in great numbers) by 8 bytes
> >> >> each.  There's really no point to framing small chunks of data,
> right?
> >>
> >> [Antoine]
> >> > You can't know how much space the pickle will take until the pickling
> >> > ends, though, which makes it difficult to decide whether you want to
> >> > emit a PREFETCH opcode or not.
> >>
> >> Ah, of course.  Presumably the outgoing pickle stream is first stored
> >> in some memory buffer, right?  If pickling completes before the buffer
> >> is first flushed, then you know exactly how large the entire pickle
> >> is.  If "it's small" (say, < 100 bytes), don't write out the PREFETCH
> >> part.  Else do.
> >
> > Yet another possibility: keep framing but use a variable-length
> > encoding for the frame size:
> >
> > - first byte: bits 7-5: N (= frame size bytes length - 1)
> > - first byte: bits 4-0: first 5 bits of frame size
> > - remaning N bytes: remaining bits of frame size
> >
> > With this scheme, very small pickles have a one byte overhead; small
> > ones a two byte overhead; and the max frame size is 2**61 rather than
> > 2**64, which should still be sufficient.
> >
> > And the frame size is read using either one or two read() calls, which
> > is efficient.
>
> And it's only a minimal change from the current patch. Sounds good to me.
>

Food for thought: maybe we should have variable-encoding lengths for all
opcodes, rather than the current cumbersome scheme?

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131118/113e6bcb/attachment.html>