[Chicago] Out of Memory: Killed Process: on CentOS

[David Beazley]

>
>
> If it's simply that cElementTree is keeping huge structures in memory
> and you're getting too many concurrent requests for your RAM, move to
> a SAX style API (state machine).
>

Also look at the ElementTree.iterparse() function.  If you use that in  
a clever way, you get all of the benefit of ElementTree plus the  
memory savings of SAX (basically you can iteratively rip through XML  
data and throw away the parts you're done with as you  go).

Cheers,
Dave