Memory leak in ndarray
Hi all I seem to have tracked down a memory leak in the string conversion mechanism of numpy. It is demonstrated using the following code: import numpy as np a = np.array([1.0, 2.0, 3.0]) while True: b = str(a) What happens above is that is repeatedly converted to a string. The process size grow quite rapidly. Has anyone else come across this? Where do I look to try to correct it? Thanks Robert
Robert Crida wrote:
Hi all
I seem to have tracked down a memory leak in the string conversion mechanism of numpy. It is demonstrated using the following code:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
What happens above is that is repeatedly converted to a string. The process size grow quite rapidly.
Has anyone else come across this? Where do I look to try to correct it? Hi Robert,
I cannot reproduce this on my machine. Could you give more details (which numpy version, etc...) ? cheers, David
Hi,
I seem to have tracked down a memory leak in the string conversion mechanism of numpy. It is demonstrated using the following code:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
Would you not expect python rather than numpy to be dealing with the memory here though? Matthew
Hi I don't think it is a python issue because if you change the line b = str(a) to just read str(a) then the problem still occurs. Also, if you change a to be a list instead of ndarray then the problem does not occur. Cheers Robert On 10/26/07, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
I seem to have tracked down a memory leak in the string conversion mechanism of numpy. It is demonstrated using the following code:
import numpy as np
a = np.array([1.0, 2.0, 3.0]) while True: b = str(a)
Would you not expect python rather than numpy to be dealing with the memory here though?
Matthew _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Robert Crida wrote:
Hi
I don't think it is a python issue because if you change the line b = str(a) to just read str(a) then the problem still occurs.
Also, if you change a to be a list instead of ndarray then the problem does not occur. How do you know there is a memory leak ?
cheers, David
Hi again I watch the VmSize of the process using eg top or ps If a is a list then it remains constant. If a is an ndarray as shown in the example, then the VmSize grows quite rapidly. Cheers Robert On 10/26/07, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
Robert Crida wrote:
Hi
I don't think it is a python issue because if you change the line b = str(a) to just read str(a) then the problem still occurs.
Also, if you change a to be a list instead of ndarray then the problem does not occur. How do you know there is a memory leak ?
cheers,
David _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
I can confirm the same behaviour with numpy '1.0.4.dev4271' on OS X 10.4with python 2.5.1 (installer from python.org). For me the memory used by the python process grows at about 1MB/sec. The memory isn't released when the loop is canceled.
On 10/26/07, Robert Crida <robert.crida@ska.ac.za> wrote:
Hi again
I watch the VmSize of the process using eg top or ps
If a is a list then it remains constant. If a is an ndarray as shown in the example, then the VmSize grows quite rapidly.
Actually, I did a typo while copying your example. I can confirm the memory leak (happen in PyObject_Malloc). cheers, David
On 10/26/07, David Cournapeau <cournape@gmail.com> wrote:
On 10/26/07, Robert Crida <robert.crida@ska.ac.za> wrote:
Hi again
I watch the VmSize of the process using eg top or ps
If a is a list then it remains constant. If a is an ndarray as shown in the example, then the VmSize grows quite rapidly.
Actually, I did a typo while copying your example. I can confirm the memory leak (happen in PyObject_Malloc).
The problem *may* be within the ufunc machinery. I am still investigating, but since we can reproduce the problem with such a simple code, it should not be difficult to track down the problem with high level tools such as valgrind. David
participants (5)
-
David Cournapeau -
David Cournapeau -
Matthew Brett -
Robert Crida -
Robin