[Python-bugs-list] [ python-Bugs-592112 ] Hint for speeding up cPickle

[noreply@sourceforge.net]

Bugs item #592112, was opened at 2002-08-07 15:31
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=592112&group_id=5470

Category: Python Library
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Jiba (jiba)
Assigned to: Nobody/Anonymous (nobody)
Summary: Hint for speeding up cPickle

Initial Comment:
There is a strange behaviour in cPickle (in Python 2.2.1).
For big serialized file, the following:

        cPickle.loads(open("my_file").read())

can be much (e.g. 7 times) faster than:

        cPickle.load(open("my_file"))

It seems that cPickle use unbuffered reading or what ?
The pickle implementation doesn't have this behaviour,
and can even be
faster than the C one!

----------------------------------------------------------------------

>Comment By: Jeremy Hylton (jhylton)
Date: 2002-08-08 17:15

Message:
Logged In: YES 
user_id=31392

I expect this has to do with extra overhead in Python to
read from the file, independent of buffering.  The cPickle
code is trying to read small numbers of bytes at a time. 
Each read used fread().  This is slow, because fread is slow
for small numbers of bytes.  It's also slow because cPickle
releases and acquires the global interpeter lock around the
fread() calls.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=592112&group_id=5470