[Numpy-discussion] ndarray subclass causes memory leak

[Jeremy Mayes]

I've narrowed this down to a change between 1.0.4 and 1.1.0.  Valgrind (of
v1.3.0) shows the following result.  The change was in setArrayFromSequence
where PyArray_EnsureArray gets invoked in v1.1.0 where it did not in v1.0.4.

==10132== 4,474,768 (3,197,200 direct, 1,277,568 indirect) bytes in 39,965
blocks are definitely lost in loss record 36 of 36
==10132==    at 0x4905B65: malloc (vg_replace_malloc.c:149)
==10132==    by 0x7FFFDAA: array_alloc (arrayobject.c:7387)
==10132==    by 0x8003465: PyArray_NewFromDescr (arrayobject.c:5900)
==10132==    by 0x802434D: PyArray_EnsureArray (multiarraymodule.c:226)
==10132==    by 0x8025766: setArrayFromSequence (arrayobject.c:7938)
==10132==    by 0x80256B8: setArrayFromSequence (arrayobject.c:7957)
==10132==    by 0x800F5FD: PyArray_FromAny (arrayobject.c:7984)
==10132==    by 0x802CC21: PyArray_CheckFromAny (arrayobject.c:9530)
==10132==    by 0x8037F64: _array_fromobject (multiarraymodule.c:6329)
==10132==    by 0x4AC9501: PyEval_EvalFrameEx (ceval.c:3612)
==10132==    by 0x4ACA894: PyEval_EvalCodeEx (ceval.c:2875)
==10132==    by 0x4ACAA11: PyEval_EvalCode (ceval.c:514)
==10132==    by 0x4AEC98B: PyRun_FileExFlags (pythonrun.c:1273)
==10132==    by 0x4AED612: PyRun_SimpleFileExFlags (pythonrun.c:879)
==10132==    by 0x4AF88C7: Py_Main (main.c:532)
==10132==    by 0x38C821C3FA: (below main) (in /lib64/tls/libc-2.3.4.so)
==10132==


On Fri, Aug 14, 2009 at 6:34 AM, Jeremy Mayes <jeremy.mayes at gmail.com>wrote:

> This simple example causes a massive leak in v1.3.0 which didn't exist in
> v1.0.1.  What am I doing wrong?  If I replace
> arr = [Array((2,2)), Array((2,2))]
>
> with
>
> arr = [numpy.ndarray((2,2,)), numpy.ndarray((2,2))]
>
> then I don't have the leak
>
>
>
> import numpy
> import gc
>
>  class Array(numpy.ndarray):
> def __new__(subtype, shape, dtype=float, buffer=None, offset=0,
> strides=None, order=None, info=None):
> return numpy.ndarray.__new__(subtype, shape, dtype, buffer, offset,
> strides, order)
>
> def __array_finalize__(self, obj):
> print 'called array_finalize'
>
> if __name__=='__main__':
> arr = [Array((2,2)), Array((2,2))]
>
> nbytesAllocated = 0
>  for i in xrange(1000000000):
> a = numpy.array(arr)
> nbytesAllocated += a.nbytes
>  if i%1000 == 0:
> print 'allocted %s'%nbytesAllocated
> gc.collect()
>
>
> --
> --jlm
>
>
>
>
> --
> --jlm
>



-- 
--jlm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090814/02ce05d3/attachment.html>