Problem with cPickle and cElementTree
stefan_ml at behnel.de
Fri Oct 10 07:27:35 CEST 2008
> I recently tried switching from ElementTree to cElementTree. My
> application parses a collection of large XML files and creates indexes
> based on certain attributes. This entire collection is saved as an
> instance of my Database class. Using ElementTree and cPickle has
> allowed me to save these instances and use them later.
> Using cElementTree significantly reduces parse time (~50%) and memory
> ~(15%) but cPickle refuses to pickle the database object. I receive:
> TypeError: expecting string or Unicode object, NoneType found
> The offending line of code simple shows my invocation of cPickle,
> which is not helpful.
> Doing exactly the same thing with ElementTree works fine.
> It appears that the objects returned by cElementTree do not pickle
> correctly. Is this a know issue? I was unable to find any reports of
> this problem.
Pickling is (almost always) built-in for pure Python classes, but it needs to
be implemented explicitly for C classes. cElementTree simply doesn't support this.
If you need a fast and memory friendly XML engine *and* want to pickle
elements, take a look at lxml.objectify. Note that it's only partially
compatible with ElementTree, so you will have to change your code (lxml.etree
is mostly compatible, but it doesn't support pickling). It's a very
easy-to-use XML library, though.
More information about the Python-list