I've narrowed this down to a change between 1.0.4 and 1.1.0. Valgrind (of v1.3.0) shows the following result. The change was in setArrayFromSequence where PyArray_EnsureArray gets invoked in v1.1.0 where it did not in v1.0.4. ==10132== 4,474,768 (3,197,200 direct, 1,277,568 indirect) bytes in 39,965 blocks are definitely lost in loss record 36 of 36 ==10132== at 0x4905B65: malloc (vg_replace_malloc.c:149) ==10132== by 0x7FFFDAA: array_alloc (arrayobject.c:7387) ==10132== by 0x8003465: PyArray_NewFromDescr (arrayobject.c:5900) ==10132== by 0x802434D: PyArray_EnsureArray (multiarraymodule.c:226) ==10132== by 0x8025766: setArrayFromSequence (arrayobject.c:7938) ==10132== by 0x80256B8: setArrayFromSequence (arrayobject.c:7957) ==10132== by 0x800F5FD: PyArray_FromAny (arrayobject.c:7984) ==10132== by 0x802CC21: PyArray_CheckFromAny (arrayobject.c:9530) ==10132== by 0x8037F64: _array_fromobject (multiarraymodule.c:6329) ==10132== by 0x4AC9501: PyEval_EvalFrameEx (ceval.c:3612) ==10132== by 0x4ACA894: PyEval_EvalCodeEx (ceval.c:2875) ==10132== by 0x4ACAA11: PyEval_EvalCode (ceval.c:514) ==10132== by 0x4AEC98B: PyRun_FileExFlags (pythonrun.c:1273) ==10132== by 0x4AED612: PyRun_SimpleFileExFlags (pythonrun.c:879) ==10132== by 0x4AF88C7: Py_Main (main.c:532) ==10132== by 0x38C821C3FA: (below main) (in /lib64/tls/libc-2.3.4.so) ==10132== On Fri, Aug 14, 2009 at 6:34 AM, Jeremy Mayes <jeremy.mayes@gmail.com>wrote:
This simple example causes a massive leak in v1.3.0 which didn't exist in v1.0.1. What am I doing wrong? If I replace arr = [Array((2,2)), Array((2,2))]
with
arr = [numpy.ndarray((2,2,)), numpy.ndarray((2,2))]
then I don't have the leak
import numpy import gc
class Array(numpy.ndarray): def __new__(subtype, shape, dtype=float, buffer=None, offset=0, strides=None, order=None, info=None): return numpy.ndarray.__new__(subtype, shape, dtype, buffer, offset, strides, order)
def __array_finalize__(self, obj): print 'called array_finalize'
if __name__=='__main__': arr = [Array((2,2)), Array((2,2))]
nbytesAllocated = 0 for i in xrange(1000000000): a = numpy.array(arr) nbytesAllocated += a.nbytes if i%1000 == 0: print 'allocted %s'%nbytesAllocated gc.collect()
-- --jlm
-- --jlm
-- --jlm