[Python-Dev] Problems with the Python Memory Manager

[skip@pobox.com]

    Travis> More to the point, however, these scalar objects were allocated
    Travis> using the standard PyObject_New and PyObject_Del functions which
    Travis> of course use the Python memory manager.  One user ported his
    Travis> (long-running) code to the new scipy core and found much to his
    Travis> dismay that what used to consume around 100MB now completely
    Travis> dominated his machine consuming up to 2GB of memory after only a
    Travis> few iterations.  After searching many hours for memory leaks in
    Travis> scipy core (not a bad exercise anyway as some were found), the
    Travis> real problem was tracked to the fact that his code ended up
    Travis> creating and destroying many of these new array scalars.

What Python object were his array elements a subclass of?

    Travis> In the long term, what is the status of plans to re-work the
    Travis> Python Memory manager to free memory that it acquires (or
    Travis> improve the detection of already freed memory locations).  

None that I'm aware of.  It's seen a great deal of work in the past and
generally doesn't cause problems.  Maybe your user's usage patterns were
a bad corner case.  It's hard to tell without more details.

Skip