multythreading app memory consumption

[Roman Petrichev]

Dennis Lee Bieber wrote:
> On Mon, 23 Oct 2006 03:31:28 +0400, Roman Petrichev <tier at inbox.ru>
> declaimed the following in comp.lang.python:
> 
>> Hi folks.
>> I've just faced with very nasty memory consumption problem.
>> I have a multythreaded app with 150 threads which use the only and the 
>> same function - through urllib2 it just gets the web page's html code 
>> and assigns it to local variable. On the next turn the variable is 
>> overritten with another page's code. At every moment the summary of 
>> values of the variables containig code is not more than 15Mb (I've just 
>> invented a tricky way to measure this). But during the first 30 minutes 
>> all the system memory (512Mb) is consumed and 'MemoryError's is arising.
>> Why is it so and how can I limit the memory consumption in borders, say, 
>> 400Mb? Maybe there is a memory leak there?
>> Thnx
>>
> 	How much stack space gets allocated for 150 threads?
Actually I don't know. How can I get to know this?
>> Q = Queue.Queue()
>> for i in rez: #rez length - 5000
> 
> 	Can't be the "test code" as you don't show the imports or where
> "rez" is defined.
Isn't it clear that "rez" is just a list of 5000 urls? I cannot place it 
here, but believe me all of them are not big - "At every moment the 
summary of values of the variables containig code is not more than 15Mb"

Regards