[Python-Dev] Problems with the Python Memory Manager
skip at pobox.com
Wed Nov 16 11:59:03 CET 2005
Travis> More to the point, however, these scalar objects were allocated
Travis> using the standard PyObject_New and PyObject_Del functions which
Travis> of course use the Python memory manager. One user ported his
Travis> (long-running) code to the new scipy core and found much to his
Travis> dismay that what used to consume around 100MB now completely
Travis> dominated his machine consuming up to 2GB of memory after only a
Travis> few iterations. After searching many hours for memory leaks in
Travis> scipy core (not a bad exercise anyway as some were found), the
Travis> real problem was tracked to the fact that his code ended up
Travis> creating and destroying many of these new array scalars.
What Python object were his array elements a subclass of?
Travis> In the long term, what is the status of plans to re-work the
Travis> Python Memory manager to free memory that it acquires (or
Travis> improve the detection of already freed memory locations).
None that I'm aware of. It's seen a great deal of work in the past and
generally doesn't cause problems. Maybe your user's usage patterns were
a bad corner case. It's hard to tell without more details.
More information about the Python-Dev