[Python-Dev] Accepting PEP 3154 for 3.4?
Nick Coghlan
ncoghlan at gmail.com
Tue Nov 19 01:30:45 CET 2013
On 19 November 2013 09:57, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 18 Nov 2013 16:44:59 -0600
> Tim Peters <tim.peters at gmail.com> wrote:
>> [Tim]
>> >> But it has a different kind of advantage: PREFETCH was optional. As
>> >> Guido said, it's annoying to bloat the size of small pickles (which
>> >> may, although individually small, occur in great numbers) by 8 bytes
>> >> each. There's really no point to framing small chunks of data, right?
>>
>> [Antoine]
>> > You can't know how much space the pickle will take until the pickling
>> > ends, though, which makes it difficult to decide whether you want to
>> > emit a PREFETCH opcode or not.
>>
>> Ah, of course. Presumably the outgoing pickle stream is first stored
>> in some memory buffer, right? If pickling completes before the buffer
>> is first flushed, then you know exactly how large the entire pickle
>> is. If "it's small" (say, < 100 bytes), don't write out the PREFETCH
>> part. Else do.
>
> Yet another possibility: keep framing but use a variable-length
> encoding for the frame size:
>
> - first byte: bits 7-5: N (= frame size bytes length - 1)
> - first byte: bits 4-0: first 5 bits of frame size
> - remaning N bytes: remaining bits of frame size
>
> With this scheme, very small pickles have a one byte overhead; small
> ones a two byte overhead; and the max frame size is 2**61 rather than
> 2**64, which should still be sufficient.
>
> And the frame size is read using either one or two read() calls, which
> is efficient.
And it's only a minimal change from the current patch. Sounds good to me.
Cheers,
Nick.
--
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Python-Dev
mailing list