Extra content at the end of the document
29 Jun
29 Jun
3:51 p.m.
junkmail@visumpoint.com, 29.06.2014 17:35:
I am attempting to use lxml to work with a very large file (over 1G). I have tested the code on a smaller data set and it works fine. However when I attempt to process the larger file I get the following error. I have visually examined the file and it looks structurally correct. Can anyone provide some insight on this issue?
Exception MemoryError: MemoryError() in 'lxml.etree._BaseErrorLog._receive' ignored
You're running out of memory. You didn't show your code, but my guess is that you're reading the entire tree into memory, which can be multiple times larger than the serialised file size. Use iterparse() instead and delete tree content when you're done with it. Stefan
3583
Age (days ago)
3583
Last active (days ago)
1 comments
2 participants
participants (2)
-
junkmail@visumpoint.com
-
Stefan Behnel