pickle vs .pyc
Michael Vezie
mlv at pobox.com
Wed Jun 2 16:22:15 EDT 1999
I need to be able to read a couple very complex (dictionary of arrays
of dictionaries, and array of dictionaries of array of dictionaries)
data structures into python. To generate it by hand takes too long,
so I want to generate it once, and read it each time (the data doesn't
change).
The obvious choice is, of course pickle, or some flavor thereof.
But can someone tell me why this wouldn't be faster:
In the code that does the "pickling", simply do:
f = open("cache.py", "w")
f.write("# cache file for fast,slow\n")
f.write("fast = "+`fast`+'\n')
f.write("slow = "+`slow'+'\n')
f.close()
import cache
Then, later, when I want the data, I just do:
from cache import fast,slow
and it's right there. It's compiled, and seems really fast (loading a
50k file in .12 seconds). I just tried the same data using cPickle, and
it took 1.4 seconds. It's also not as portable. There is a space savings
with pickle, but it's only 5% (well, 56% if you count both the .py and
.pyc files), but that doesn't really matter to me.
Am I missing something here? This sounds like an obvious, and fast,
way to do things. True, the caching part may take longer. But I
really don't care about that, since it's done only once, and in the
background.
Michael
More information about the Python-list
mailing list