[Numpy-discussion] Another reference count leak: ticket #848

[Robert Kern]

On Tue, Jul 8, 2008 at 16:23, Michael Abbott <michael at araneidae.co.uk> wrote:
> On Tue, 8 Jul 2008, Travis E. Oliphant wrote:
>> Michael Abbott wrote:
>> > The attached patch fixes another reference count leak in the use of
>> > PyArray_DescrFromType.
>> >
>> The first part of this patch is good.  The second is not needed.
> I don't see that.  The second part of the patch addresses the case of an
> early return: this means that the DECREF that occurs later on in the code
> is bypassed, and so a reference leak will still occur if this early return
> case occurs.  Don't forget that PyArray_DescrFromType returns an
> incremented reference that has to be decremented, returned or explicitly
> assigned -- the DECREF obligation has to be met somewhere.
>
>> Also, it would be useful if you could write a test case that shows what
>> is leaking and how you determined that it is leaking.
>
> Roughly
>
>        r = range(n)
>        i = 0
>        refs = 0
>        refs = sys.gettotalrefcount()
>        for i in r: float32()
>        print refs - sys.gettotalrefcount()
>
> in debug mode python.  This isn't quite the whole story (reference counts
> can be annoyingly fluid), but that's the most of it.  In trunk this leaks
> 2 refs per n, with the attached patch there remains one leak I haven't
> chased down yet.
>
> Is there a framework for writing test cases?  I'm constructing tests just
> to pin down leaks that I find in my application (uses numpy and ctypes
> extensively), so they're terribly ad-hoc at the moment.

If you can measure the leak in-process with sys.getrefcount() and
friends on a standard non-debug build of Python, then it might be
useful for you to write a unit test for us. For example, in
numpy/core/tests/test_regression.py, you can see several tests
involving reference counts.

If the leak isn't particularly measurable without resorting to top(1),
then don't bother trying to make a unit test, but a small, complete
example that demonstrates the leak is very useful for the rest of us
to see if the problem exists on our systems and so we can try our
hands at fixing it, too.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco