Queue not releasing memory

Nathan Clegg nathan at islanddata.com
Mon Sep 13 12:13:49 EDT 1999


On 12-Sep-99 Tim Peters wrote:
> Python has no control over when your OS decides that released memory is
> available to something else.  If you name your OS, perhaps someone will
> know
> how to influence it reliably.

The results I described were under Linux, glibc 2.1.1.  The target
platform for this application, however, is Solaris 2.7, which is worse. 
Under Solaris, the memory is not reclaimed when the queue is deleted.  The
entire python process must terminated.  Not excited about this.  I just
did a simple malloc/free test with the exact same results.  Bad Solaris.

> By the way, look at that code carefully:  the default Queue
> implementation
> wasn't meant to handle memory-busting amounts of data!  That "del
> self.queue[0]" line makes a physical copy of the entire data remaining
> every
> time.

I do need the thread-safety of the Queue class.  The large size
requirement is only an extreme situation, which is why it is important
that it backs down afterward.  Perhaps I need to take some pressure off
Queue and move more quickly to my other lines of defense.

Is there no way to remove an element from a list without making a copy? 
append() doesn't need a copy.  I was under the impression that lists were
internally represented by a more dynamic form.

Thanks for the information.  I will reevaluate my plan of attack.

----------------------------------
Nathan Clegg
 nathan at islanddata.com






More information about the Python-list mailing list