10GB XML Blows out Memory, Suggestions?

fuzzylollipop jarrod.roberson at gmail.com
Tue Jun 6 22:51:50 EDT 2006


axwack at gmail.com wrote:
> Paul,
>
> This is interesting. Unfortunately, I have no control over the XML
> output. The file is from Goldmine. However, you have given me an
> idea...
>
> Is it possible to read an XML document in compressed format?

compressing the footprint on disk won't matter, you  still have 10GB of
data that you need to process and it can only be processed
uncompressed.

I would just export the data in smaller batches, there should not be
any reason you can't export subsets and process them that way.




More information about the Python-list mailing list