[New-bugs-announce] [issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Pickle protocol 4 uses framing for reducing the overhead of calling the read() method for small chunks of data. Most read chunks are small -- opcodes, small integers, short strings, etc, and calling read() for every 1 or 4 bytes is too expensive. But using framing itself adds an overhead. It increases the size of pickled data by 9 bytes. A frame  itself needs 3 reads -- the opcode, the frame size, and a payload. Thus it doesn't make sense to create a frame containing less than 3 chunks of data.

For example after issue31993 pickling the list [b'a'*70000, b'b'*70000] with the Python implementation produces a data containing 3 frames of sizes 3, 1 and 3. Using frames here is completely redundant.

----------
components: Library (Lib)
messages: 309564
nosy: Olivier.Grisel, alexandre.vassalotti, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Avoid creating small frames in pickle protocol 4
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32503>
_______________________________________