Trying to parse a HUGE(1gb) xml file

Tim Harig usernet at ilthio.net
Sat Dec 25 12:27:15 EST 2010


On 2010-12-25, Steve Holden <steve at holdenweb.com> wrote:
> On 12/23/2010 4:34 PM, Stefan Sonnenberg-Carstens wrote:
>> For large datasets I always have huge question marks if one says "xml".
>> But I don't want to start a flame war.

I would agree; but, you don't always have the choice over the data format
that you have to work with.  You just have to do the best you can with what
they give you.

> I agree people abuse the "spirit of XML" using it to transfer gigabytes
> of data, but what else are they to use?

Something with an index so that you don't have to parse the entire file
would be nice.  SQLite comes to mind.  It is not standardized; but, the
implementation is free with bindings for most languages.



More information about the Python-list mailing list