Hi, Unless I'm mistaken, pickle's documentation doesn't mention that the pickle wire-format is self-delimiting. Is there any reason why it's not documented? The reason I'm asking is because I've seen some code out there doing its own ad-hoc length-prefix framing. Cheers, cf
On Tue, 1 Apr 2014 19:29:38 +0100
Charles-François Natali
Hi,
Unless I'm mistaken, pickle's documentation doesn't mention that the pickle wire-format is self-delimiting. Is there any reason why it's not documented?
No reason AFAIK. However, the fact that it is self-delimited is implicit in the fact that "Bytes past the pickled object’s representation are ignored": https://docs.python.org/dev/library/pickle.html#pickle.load Also, note that protocol 4 now features a length-prefix framing to improve buffering performance with arbitrary streams. Regards Antoine.
No reason AFAIK. However, the fact that it is self-delimited is implicit in the fact that "Bytes past the pickled object's representation are ignored": https://docs.python.org/dev/library/pickle.html#pickle.load
I find this sentence worrying: it could lead one to think that load() could read more bytes than the expected object representation size: this would make pickle actually non self-delimiting, and could lead to problems when reading e.g. from a socket, since an extraneous read() could block. I think it's worth making it clear in the doc, I'll open an issue on the tracker.
participants (2)
-
Antoine Pitrou
-
Charles-François Natali