[New-bugs-announce] [issue32503] Avoid creating small frames in pickle protocol 4
report at bugs.python.org
Sat Jan 6 13:36:38 EST 2018
New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:
Pickle protocol 4 uses framing for reducing the overhead of calling the read() method for small chunks of data. Most read chunks are small -- opcodes, small integers, short strings, etc, and calling read() for every 1 or 4 bytes is too expensive. But using framing itself adds an overhead. It increases the size of pickled data by 9 bytes. A frame itself needs 3 reads -- the opcode, the frame size, and a payload. Thus it doesn't make sense to create a frame containing less than 3 chunks of data.
For example after issue31993 pickling the list [b'a'*70000, b'b'*70000] with the Python implementation produces a data containing 3 frames of sizes 3, 1 and 3. Using frames here is completely redundant.
components: Library (Lib)
nosy: Olivier.Grisel, alexandre.vassalotti, pitrou, serhiy.storchaka
title: Avoid creating small frames in pickle protocol 4
versions: Python 3.7
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce