Why deleting large variables doesn't free up any memory?

[Andrew MacIntyre]

On Sat, 5 Apr 2003, Nick Hilliard wrote:

> sdieselil wrote:
>
> > I'm using Python 2.2.2 under FreeBSD. When I create a large variable
> > (using for example "a = range(1000000)") 'top' command indicates that
> > Python ate 12MB of swap space. But when I execute "del a" nothing
> > happens, it doesn't release these 12MB! Using "gc.collect()" also
> > doesn't help. What's the problem? What happened to Python's garbage
> > collector?
>
> Regardless of the other answers in this thread, I suspect that this problem is
> probably less important than you think.  If FreeBSD thinks that the memory isn't
> being used, it will be swapped out if there's a memory shortfall at any stage.
> The important factor is amount of memory which is being actively used at a
> particular time.

There's an additional subtlety with the OP's example: a only references
the list.

Each individual element in the list is an integer object in its own right.
As a performance optimisation, the memory for all integer objects created
is kept in a single list, and never free()ed.  Integer object memory
released by garbage collection of unreferenced integers (except -5..100,
which are interned) is reused for new integer references.

So when a is derefenced, only about 4MB (the size of the list vector) is
actually free()ed, which may or may not be returned to the OS by the
malloc implementation.

xrange() may help the OP, depending on the actual usage of a.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  | Snail: PO Box 370
        andymac at pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia