[Tutor] ElementTree - reading large XML files as file handles

[Kent Johnson]

Srinivas Iyyer wrote:
> Dear tutors, 
> 
> I use ElementTree for XML works. I have a 1.3GB file
> to parse. 
> 
> 
> I takes a lot of time to open my input XML file. 
> 
> Is that because of my hardware limitation or am I
> using a blunt method to load the file.
> 
> my computer config:
> Inte(R)
> Pentium(R)4 CPU 2.80GHz
> 2.79GHz, 0.99GB of RAM
> 
> from elementtree import ElementTree
> myfile = open('myXML.out','r')

Reading a 1.3 GB file on a machine with .99 GB RAM is certainly pushing 
things. To parse it into an ElementTree will probably double or triple 
your memory requirements.
> 
> Do you suggest any tip to circumvent the file opening
> problem. 

Do you need the whole parsed tree at once or can you process it a little 
at a time? If not, maybe this will help:
http://effbot.org/zone/element-iterparse.htm

Kent