<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#ffffff" text="#000000">
<pre wrap=""><div class="moz-txt-sig"><pre wrap="">There are a few places where we (improperly) directly call malloc()
instead of PyMem_Malloc(), so yes, you should rebuild numpy against
TCMalloc in addition to the Python interpreter.
</pre><p wrap=""><font color="#000000">How do I build against the -ltcmalloc library? Just not sure how to input compile time arguments with the build script.
</font></p><pre wrap="">from numpy import arange, sum
<span class="moz-txt-citetags">></span>
<span class="moz-txt-citetags">> </span>for x in range(10000000):
<span class="moz-txt-citetags">> </span> inhibVal = sum(arange(15))
Memory usage stays constant with Ubuntu 11.04, 64-bit, using the numpy
1.5.1 package from ubuntu, and using 1.6.1.dev-a265004.
efiring@manini:~$ uname -a
Linux manini 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC
2011 x86_64 x86_64 x86_64 GNU/Linux
Eric
</pre><p wrap=""><font color="#000000">Also noticed that the same issue occurs as described with using the prod instead of sum. Other commands like copy and reshape have no issues. I have a machine with 11.04 on it so I will give that a go later to see what goes.
</font></p><p wrap=""><font color="#000000">
Thanks,
Jeff
</font></p>
Message: 2
Date: Sun, 22 May 2011 12:32:43 -0500
From: Robert Kern <a class="moz-txt-link-rfc2396E" href="mailto:robert.kern@gmail.com"><robert.kern@gmail.com></a>
Subject: Re: [Numpy-discussion] Python memory management issues using
Linux. Maybe Numpy related.
To: Discussion of Numerical Python <a class="moz-txt-link-rfc2396E" href="mailto:numpy-discussion@scipy.org"><numpy-discussion@scipy.org></a>
Message-ID: <a class="moz-txt-link-rfc2396E" href="mailto:BANLkTikUgYm8COoZEhNP2tJGYRb_PBH1og@mail.gmail.com"><BANLkTikUgYm8COoZEhNP2tJGYRb_PBH1og@mail.gmail.com></a>
Content-Type: text/plain; charset=UTF-8
On Sat, May 21, 2011 at 23:27, Jeffrey Spencer <a class="moz-txt-link-rfc2396E" href="mailto:jeffspencerd@gmail.com"><jeffspencerd@gmail.com></a> wrote:
</div></pre>
<blockquote type="cite" style="color: rgb(0, 0, 0);">
<pre wrap=""><span class="moz-txt-citetags">> </span>I have had issues with Linux (Ubuntu 10.04 x86_64) not releasing memory
<span class="moz-txt-citetags">> </span>back to the OS. I have code here the exhibits the issue:
<span class="moz-txt-citetags">></span>
<span class="moz-txt-citetags">> </span><a class="moz-txt-link-freetext" href="http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-in-linux-even-after-g">http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-in-linux-even-after-g</a>
<span class="moz-txt-citetags">></span>
<span class="moz-txt-citetags">> </span>Then I read at <a class="moz-txt-link-freetext" href="http://pushingtheweb.com/2010/06/python-and-tcmalloc/">http://pushingtheweb.com/2010/06/python-and-tcmalloc/</a>
<span class="moz-txt-citetags">> </span>that this was a problem fixed by compiling your own version of Python
<span class="moz-txt-citetags">> </span>using the TCMalloc library which I have now done. The issue is still
<span class="moz-txt-citetags">> </span>exhibited so I was keen to know if possibly it was because numpy is
<span class="moz-txt-citetags">> </span>compiled and linked to the default memory management library (not sure
<span class="moz-txt-citetags">> </span>if this is true or how this works but would I also need to compile numpy
<span class="moz-txt-citetags">> </span>linked to the TCMalloc library). Let me know if anyone has any insight
<span class="moz-txt-citetags">> </span>on this issue. Or if it isn't related to numpy at all.
</pre>
</blockquote>
<pre wrap="">There are a few places where we (improperly) directly call malloc()
instead of PyMem_Malloc(), so yes, you should rebuild numpy against
TCMalloc in addition to the Python interpreter.
<div class="moz-txt-sig">--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
? -- Umberto Eco
</div></pre>
<pre class="moz-signature" cols="72">--
________________________
Jeffrey Spencer
<a class="moz-txt-link-abbreviated" href="mailto:jeffspencerd@gmail.com">jeffspencerd@gmail.com</a>
</pre>
</body>
</html>