[Numpy-discussion] reference leacks in numpy.asarray
Lisandro Dalcin
dalcinl at gmail.com
Thu Aug 2 18:20:22 EDT 2007
I think the problem is in _array_fromobject (seen as numpy.array in Python)
This function parses its arguments by using the convertor
PyArray_DescrConverter2. which RETURNS A NEW REFERENCE!!! This
reference is never DECREF'ed.
BTW, A lesson I've learned of the pattern
if (!PyArg_ParseXXX(....)) return NULL
is that convertor functions should NEVER return new references to
PyObject*'s, because if the conversion fails (because of latter wrong
argument), you leak a reference to the 'converted' object.
If this pattern is used everywhere in numpy, well, there are big
chances of leaking references in the case of bad args to C functions.
Regards,
On 8/2/07, Timothy Hochberg <tim.hochberg at ieee.org> wrote:
>
>
>
> On 8/2/07, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> > using numpy-1.0.3, I believe there are a reference leak somewhere.
> > Using a debug build of Python 2.5.1 (--with-pydebug), I get the
> > following
> >
> > import sys, gc
> > import numpy
> >
> > def testleaks(func, args=(), kargs={}, repeats=5):
> > for i in xrange(repeats):
> > r1 = sys.gettotalrefcount()
> > func(*args,**kargs)
> > r2 = sys.gettotalrefcount()
> > rd = r2-r1
> > print 'before: %d, after: %d, diff: [%d]' % (r1, r2, rd)
> >
> > def npy_asarray_1():
> > a = numpy.zeros(5, dtype=int)
> > b = numpy.asarray(a, dtype=float)
> > del a, b
> >
> > def npy_asarray_2():
> > a = numpy.zeros(5, dtype=float)
> > b = numpy.asarray(a, dtype=float)
> > del a, b
> >
> > if __name__ == '__main__':
> > testleaks(npy_asarray_1)
> > testleaks(npy_asarray_2)
> >
> >
> > $ python npyleaktest.py
> > before: 84531, after: 84532, diff: [1]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84534, after: 84534, diff: [0]
> > before: 84531, after: 84533, diff: [2]
> > before: 84535, after: 84536, diff: [1]
> > before: 84536, after: 84537, diff: [1]
> > before: 84537, after: 84538, diff: [1]
> > before: 84538, after: 84539, diff: [1]
> >
> > It seems npy_asarray_2() is leaking a reference. I am missing
> > something here?. The same problem is found in C, using
> > PyArray_FROM_OTF (no time to go inside to see what's going on, sorry)
> >
> > If this is know and solved in SVN, please forget me.
>
> I don't have a debug build handy to test this on, but this might not be a
> reference leak. Since you are checking the count before and after each
> cycle, it could be that there are cycles being created that are subsequently
> cleaned up by the garbage collector.
>
> Can you try instead to look at the difference between the reference count at
> the end of each cycle with the reference count before the first cycle? If
> that goes up indefinitely, then it's probably a leak. If it bounces around
> or levels off, then probably not. You'd probably want to run a bunch of
> repeats just to be sure.
> regards,
> -tim
>
> > Regards,
> >
> >
> > --
> > Lisandro Dalcín
> > ---------------
> > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> > PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> > Tel/Fax: +54-(0)342-451.1594
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at scipy.org
> >
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
>
> --
> . __
> . |-\
> .
> . tim.hochberg at ieee.org
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
More information about the NumPy-Discussion
mailing list