[Python-Dev] When do sets shrink?

[Raymond Hettinger]

> > It's very difficult to do something useful about it. Even if
> > you manage to determine how much memory you want to release,
> > it's nearly impossible to actually release the memory to the
> > operating system, because of the many layers of memory
> > management software that all need to agree that the memory
> > should be reused somewhere else (instead of keeping it on
> > that layer, just in case somebody using that layer wants
> > some memory).
> >
> I checked - when doing the same thing with lists, all the memory was
> released for use by other Python objects, and most of it was released
> for use by the operating system.

Put another way, it is difficult to assure that the memory is released
back to the operating system.  We don't control that part.

What could be done is to add a test for excess dummy entries and trigger
a periodic resize operation.  That would make the memory available for
other parts of the currently running script and possibly available for
the O/S.

The downside is slowing down a fine-grained operation like pop().  For
dictionaries, this wasn't considered worth it.  For sets, I made the
same design decision.  It wasn't an accident.  I don't plan on changing
that decision unless we find a body of real world code that would be
better-off with more frequent re-sizing.


Raymond