[Numpy-discussion] Python memory management issues using Linux. Maybe Numpy related.

Robert Kern robert.kern at gmail.com
Sun May 22 13:32:43 EDT 2011


On Sat, May 21, 2011 at 23:27, Jeffrey Spencer <jeffspencerd at gmail.com> wrote:
> I have had issues with Linux (Ubuntu 10.04 x86_64) not releasing memory
> back to the OS. I have code here the exhibits the issue:
>
> http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-in-linux-even-after-g
>
> Then I read at http://pushingtheweb.com/2010/06/python-and-tcmalloc/
> that this was a problem fixed by compiling your own version of Python
> using the TCMalloc library which I have now done. The issue is still
> exhibited so I was keen to know if possibly it was because numpy is
> compiled and linked to the default memory management library (not sure
> if this is true or how this works but would I also need to compile numpy
> linked to the TCMalloc library). Let me know if anyone has any insight
> on this issue. Or if it isn't related to numpy at all.

There are a few places where we (improperly) directly call malloc()
instead of PyMem_Malloc(), so yes, you should rebuild numpy against
TCMalloc in addition to the Python interpreter.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco



More information about the NumPy-Discussion mailing list