Hi all I recently posted about a memory leak in numpy and failed to mention the version. The leak manifests itself in numpy-1.0.3.1 but is not present in numpy-1.0.2 The following code reproduces the bug: import numpy as np a = np.array([1.0, 2.0, 3.0]) while True: b = str(a) What happens above is that is repeatedly converted to a string. The process size grow quite rapidly. Has anyone else come across this? Where do I look to try to correct it? Thanks Robert
Which version Python are you using ? Matthieu 2007/10/26, Robert Crida <robert.crida@ska.ac.za>:
Hi all
I recently posted about a memory leak in numpy and failed to mention the version. The leak manifests itself in numpy-1.0.3.1 but is not present in numpy-1.0.2
The following code reproduces the bug:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
What happens above is that is repeatedly converted to a string. The process size grow quite rapidly.
Has anyone else come across this? Where do I look to try to correct it?
Thanks Robert
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- French PhD student Website : http://miles.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
On 10/26/07, Robert Crida <robert.crida@ska.ac.za> wrote:
Hi all
I recently posted about a memory leak in numpy and failed to mention the version. The leak manifests itself in numpy-1.0.3.1 but is not present in numpy-1.0.2
The following code reproduces the bug:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
What happens above is that is repeatedly converted to a string. The process size grow quite rapidly.
Has anyone else come across this? Where do I look to try to correct it?
I find the same leak: Python version: 2.5.1 numpy version: 1.0.4.dev4081 Regards, Kurt
Thanks Robert
On Friday 26 October 2007 05:39, Robert Crida wrote:
Hi all
I recently posted about a memory leak in numpy and failed to mention the version. The leak manifests itself in numpy-1.0.3.1 but is not present in numpy-1.0.2
The following code reproduces the bug:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
What happens above is that is repeatedly converted to a string. The process size grow quite rapidly.
Has anyone else come across this? Where do I look to try to correct it?
Thanks Robert
I can reproduce this leak: Python 2.5 (r25:51908, Apr 30 2007, 15:03:13) [GCC 3.4.6 (Debian 3.4.6-5)] on linux2
import numpy numpy.version.version '1.0.4.dev4317' a = numpy.array([1.0]) while True: ... b = a.__str__() ...
which causes the resident size of the process to grow by about 1MB/s as mentioned earlier. Interestingly, using non-float dtypes does not cause the loop to leak. Karol -- written by Karol Langner Sun Oct 28 00:47:31 EDT 2007
On Friday 26 October 2007 05:39, Robert Crida wrote:
I recently posted about a memory leak in numpy and failed to mention the version. The leak manifests itself in numpy-1.0.3.1 but is not present in numpy-1.0.2
The following code reproduces the bug:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
What happens above is that is repeatedly converted to a string. The process size grow quite rapidly.
Has anyone else come across this? Where do I look to try to correct it?
Debugging exposes numpy.core.umath.seterrobj as the source, so you can get even faster memory growth with this (around 30MB/s on my machine): import numpy pyvals = numpy.core.umath.geterrobj() while True: numpy.core.umath.seterrobj(pyvals) That is actually ufunc_seterr in trunk/numpy/core/src/ufuncobject.c... which calls ufunc_update_use_defaults as of r3040. A call to Py_DECREF(errobj) is missing there after calling PyUFunc_GetPyValues. So using the following patch on the current svn revision of ufuncobject.c should fix this leak: 3206a3207
Py_DECREF(errobj);
Cheers, Karol -- written by Karol Langner Sun Oct 28 20:00:47 EDT 2007
I opened a ticket for this (#602). Hopefully someone will confirm that adding that Py_DECREF call fixes the leak and someone with write access patches it in svn. - Karol -- written by Karol Langner Sun Oct 28 23:29:18 EDT 2007
Hi Karol Thank you very much for the sleuth work here. We are in the midst of software ATP so it helps a lot that I will be able to resolve this bug properly. Robert On 10/29/07, Karol Langner <karol.langner@kn.pl> wrote:
I opened a ticket for this (#602). Hopefully someone will confirm that adding that Py_DECREF call fixes the leak and someone with write access patches it in svn.
- Karol
-- written by Karol Langner Sun Oct 28 23:29:18 EDT 2007 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
participants (5)
-
Karol Langner
-
Kurt Smith
-
Matthieu Brucher
-
Robert Crida
-
Travis E. Oliphant