[Chicago] Out of Memory: Killed Process: on CentOS

David Beazley d-beazley at sbcglobal.net
Mon Apr 27 18:51:20 CEST 2009


>
>
> If it's simply that cElementTree is keeping huge structures in memory
> and you're getting too many concurrent requests for your RAM, move to
> a SAX style API (state machine).
>

Also look at the ElementTree.iterparse() function.  If you use that in  
a clever way, you get all of the benefit of ElementTree plus the  
memory savings of SAX (basically you can iteratively rip through XML  
data and throw away the parts you're done with as you  go).

Cheers,
Dave



More information about the Chicago mailing list