[Numpy-discussion] Python ref count leak in numpy

[Michael Abbott]

I hope this is going to the right place.  I've tried to submit a Trac 
ticket at http://projects.scipy.org/scipy/numpy/ but unfortunately it 
won't let me log in, even though I've registered and successfully logged 
in as a SciPy user!  Grr.

The bug itself is very easy to see in Python debug mode: adding arrays of 
differing shapes causes the reference count to increase on each operation.  
For example:

	array(1) + 1            # Leaks one ref count per call
	1 + array(1)            # Leaks 12 ref counts per call(!)

It turns out that PyArray_CanCoerceScalar is being called rather a lot of 
times (some of the lower level numpy code is really not nice, which is a 
pity as there's some really clean code there too) ... and it has a ref 
count leak.  The patch below fixes this problem.

This patch was made against version 5331 from subversion.  The leak was 
introduced in revision 2575 in June '06, over two years ago -- I'm a 
little surprised this hasn't been discovered already.



commit ea66a7ee65e1ae855bbc432a48eb1f48176a85d9
Author: Michael Abbott <michael.abbott at diamond.ac.uk>
Date:   Fri Jul 4 12:08:24 2008 +0100

    Fix memory leak in PyArray_CanCoerceScalar
    
    This routine (in multiarraymodule.c) was calling PyArray_DescrFromType
    without properly decrementing the reference count afterwards.  As this
    is use widely, this results in memory leaks in the most basic of
    operations.

diff --git a/numpy/core/src/multiarraymodule.c b/numpy/core/src/multiarraymodule.c
index 1b752d6..e44ebac 100644
--- a/numpy/core/src/multiarraymodule.c
+++ b/numpy/core/src/multiarraymodule.c
@@ -2143,8 +2143,13 @@ PyArray_CanCoerceScalar(int thistype, int neededtype,
     if (from->f->cancastscalarkindto &&
         (castlist = from->f->cancastscalarkindto[scalar])) {
         while (*castlist != PyArray_NOTYPE)
-            if (*castlist++ == neededtype) return 1;
+            if (*castlist++ == neededtype) {
+                Py_DECREF(from);
+                return 1;
+            }
     }
+    Py_DECREF(from);
+    
     switch(scalar) {
     case PyArray_BOOL_SCALAR:
     case PyArray_OBJECT_SCALAR: