High memory usage - program mistake or Python feature?

[Delaney, Timothy C (Timothy)]

> From: Ben S [mailto:bens at replytothegroupplease.com.avaya.com]
> 
> If I use xreadlines (and drop the string stripping) I suppose I would
> gain a lot on memory but lose out on speed (due to multiple 
> passes over
> the file), right? This isn't necessarily a bad thing since 
> it's running
> as a CGI script, and therefore the bottleneck is usually 
> going to be the
> network, I expect.

You will always have to trade off various types of resources against each other - in this case, memory vs. performance.

I suspect you will find though that the performance loss is fairly inconsequential compared to the memory savings. Indeed, you may even have a performance improvement since most machines are still 256MB or less, and as a result you are probably hitting virtual memory with your Python instance ...

Tim Delaney