What means exactly "Memory error"?

[Oren Tirosh]

On Thu, Apr 24, 2003 at 06:36:38PM +0200, Bo M. Maryniuck wrote:
> On Thursday 24 April 2003 17:19, Skip Montanaro wrote:
> >     Bo> Somebody knows what "Memory error" might appear and how to avoid
> > (if Bo> possible) it?
> > It means a memory allocation using malloc or realloc failed.  In short,
> > virtual memory was exhausted.
> 
> Mmm... OK, but what else I can to do? Yes, Python process grows up to ~1.2G in 
> the memory. The main code where it crashes it's like:
> 
> parsedBranches = {}
> for onePart in aLotOfParts:
> 	parsedBranches[generateId()] = HtmlLib.FromHtml(onePart)
> 
> ...and after 600-700 parts HtmlLib (4DOM) just crashes.
> 
> Any ideas?

Try to use a streaming XML API like SAX instead of DOM. For the splitting
operation you describe there shouldn't be any need to store the entire file
in memory. You may not even need and XML parser at all - looking for the 
HR markers could probably be done by a few regular expressions without full 
parsing of the XML stream.

    Oren