[issue3873] Unpickling is really slow

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

alexandre.vassalotti wrote:
> The solution is to add a read buffer to Unpickler (...) 
> would mitigate much of the (quite large) Python function 
> call overhead. (...) cPickle has a performance hack to make it 
> uses cStringIO and PyFile directly (via C function calls). In 
> Python 3, the hack was removed (...)

Yes, unpickler_read() calls Buffered_read() through 
PyObject_Call+PyCFunction_Call which is expensive. And unpickle main 
loop starts with unpickler_read(self, &s, 1): just read *one* byte 
(the "opcode"). If Buffered_read() call is expensive, a solution is to 
avoid calling it (eg. read more bytes and... go backward at the end?).

----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3873>
_______________________________________