[python-win32] gc does not free memory [SEC=PERSONAL]

[Andrew MacIntyre]

> I am trying to understand Python's garbage collection mechanism. I 
> have a long running program (actually a shell
> extension) that needs to allocate large arrays occasionally, which I'd

> del after use, then call
> gc.collect() to collect them. But although gc.get_objects() reported 
> the arrays have been collected (also confirmed they didn't end up in 
> gc.garbage), but the memory footprint (VM
> size) remain the same, as if the memory was never freed and return to 
> heap.
> 
> Appreciate if anyone can help shed some light on this. Google didn't 
> hit much useful websites on this topic. FYI, I am using ActivePython 
> 2.5 on XP SP2. Thanks.

You need to be a bit more specific about what you're doing to get
detailed information.

Some things to remember about memory consumption, taking into account
you're using Python 2.5:
- make sure that you aren't keeping unnecessary references (only cyclic
  references show up in the gc as potential garbage)
- if by array you mean list or tuple, be aware that the members of these
  structures are independent objects in their own right.  arrays created
  using the array module store all members internally.  this difference
  matters if your members are floats or ints.
- int and float objects use a free list structure for deleted objects
  which is never pruned, so allocating large numbers of these objects
  and deleting them will leave memory in the free list that is not
  returned to the OS until the interpreter exits.  additionally, only
  integers in the range -5..256 (IIRC) are shared; for all other integer
  values, multiple objects can exist with the same value.
- while most small Python objects (those occupying <=256 bytes, except
  ints and floats) are managed through Python's internal memory
allocator
  this allocator uses 256kB arenas which are only returned when
completely
  unused.  this allocator only moves objects in memory when resizing an
  object, so usage patterns exist which can result in no arenas being
  free'ed even with most memory unused.

-------------------------> "These thoughts are mine alone!" <---------
Andrew MacIntyre           National Licensing and Allocations Branch
tel:   +61 2 6219 5356     Inputs to Industry Division
fax:   +61 2 6253 3277     Australian Communications & Media Authority
email: andrew.macintyre at acma.gov.au 

If you have received this email in error, please notify the sender immediately and erase all copies of the email and any attachments to it. The information contained in this email and any attachments may be private, confidential and legally privileged or the subject of copyright. If you are not the addressee it may be illegal to review, disclose, use, forward, or distribute this email and/or its contents.
 
Unless otherwise specified, the information in the email and any attachments is intended as a guide only and should not be relied upon as legal or technical advice or regarded as a substitute for legal or technical advice in individual cases. Opinions contained in this email or any of its attachments do not necessarily reflect the opinions of ACMA.