pickle vs .pyc

Michael Vezie mlv at pobox.com
Wed Jun 2 22:22:15 CEST 1999


I need to be able to read a couple very complex (dictionary of arrays 
of dictionaries, and array of dictionaries of array of dictionaries) 
data structures into python.  To generate it by hand takes too long, 
so I want to generate it once, and read it each time (the data doesn't 
change).

The obvious choice is, of course pickle, or some flavor thereof.
But can someone tell me why this wouldn't be faster:

In the code that does the "pickling", simply do:
f = open("cache.py", "w")
f.write("# cache file for fast,slow\n")
f.write("fast = "+`fast`+'\n')
f.write("slow = "+`slow'+'\n')
f.close()
import cache

Then, later, when I want the data, I just do:

from cache import fast,slow

and it's right there.  It's compiled, and seems really fast (loading a 
50k file in .12 seconds).  I just tried the same data using cPickle, and 
it took 1.4 seconds.  It's also not as portable.  There is a space savings 
with pickle, but it's only 5% (well, 56% if you count both the .py and 
.pyc files), but that doesn't really matter to me.

Am I missing something here?  This sounds like an obvious, and fast, 
way to do things.  True, the caching part may take longer.  But I 
really don't care about that, since it's done only once, and in the 
background.  

Michael




More information about the Python-list mailing list