What means exactly "Memory error"?
staschuk at telusplanet.net
Fri Apr 25 17:21:52 CEST 2003
Quoth Bo M. Maryniuck:
> On Friday 25 April 2003 12:56, Albert Hofkamp wrote:
> > Do you need all data DOMmed at once?
> That what I doing right now. Well, it not so easy to make an app with high
> perfomance where to parse it *twice* and re-collect data from the disk
> storage. But I see there is no other way...
Others have suggested using a streaming API (such as SAX), and
that seems like the obvious way to reduce your program's memory
Certainly breaking the document up at <hr>s is easy with SAX.
What are you doing to the parts afterwards? For example, when
you're transforming one part, do you need access to all the other
If you do need access to all the other parts, one alternative
approach would be to replace your dict parsedBranches with a
shelf. This would move the unneeded parts out to disk, saving
memory (at the cost of time, of course), providing a crude kind of
Steven Taschuk staschuk at telusplanet.net
"I'm always serious, never more so than when I'm being flippant."
-- _Look to Windward_, Iain M. Banks
More information about the Python-list