Queue not releasing memory
Robin Becker
robin at jessikat.demon.co.uk
Sat Sep 11 23:56:26 EDT 1999
In article <19990912032623.13533.qmail at islanddata.com>,
nathan at islanddata.com writes
>After filling a Queue class instance with large amounts of data, the
>memory footprint of python grows dramatically. I would expect, then,
>that this footprint would decrease in size when the queue is emptied,
>but it does not. The memory use drops instantly when the empty queue
>is delete, however. I've examined the Queue.py code and cannot see
>where this data may be cached, etc. Executing:
>
>sys.getrefcount(q.queue[0])
>
>for example, yields 2: the temporary reference and a single one for
>the queue entry. It stands to reason that after get'ing an item from
>the queue that it should be lost forever. Test program:
>
>q = Queue.Queue(20000)
>for x in string.uppercase:
> for y in string.uppercase:
> for z in string.uppercase:
> m = (x + y + z) * 1024
> q.put(m)
>
>for x in string.uppercase:
> for y in string.uppercase:
> for z in string.uppercase:
> q.get()
>
>The memory taken up by this program does not decrease until q is
>deleted. Can anyone tell me why this happens and how I can fix it
>*without* deleting the queue and creating a new one? Thanks...
>
>
it may be that your sequence of queue pushes and pops has left you with
a sparsely populated lump of memory. Some allocators recognize when
blocks are completely free and then return them to the system. I believe
the Win95 allocator works a bit like that.
--
Robin Becker
More information about the Python-list
mailing list