Ref counts: bug? feature? gotcha?

[John Fisher]

Hey folks,

I just tracked down a bug in a program of mine and, realizing what
caused it, I've begun to wonder if it really should have been an error.

In a nutshell, a C extension did some work and packed the results into a
NumPy array to return to Python.  It went something like this:
	return Py_BuildValue("OOOlld", PyArray_Return(array1),
			     .... );
The other two arrays were returned the same way as the first; the three
other arguments are inconsequential.  The error came once back in the
interpreter: any attempts to access any of the arrays resulted in a seg
fault.

My solution was to manually Py_INCREF() each array, and then change the
return to:
	return Py_BuildValue("NNNlld", PyArray_Return(array1),
			     .... );

This worked fine.  But I'm unclear on some things, and would love a bit
of insight from those more experienced than I.

First... why was the change necessary?  Py_BuildValue() increments the
ref count for objects declared "O" in the format string.  The arrays
going into PyArray_Return() were not improper in any form; presumably
PyArray_Return() simply spat back a duplicate pointer.  I fail to see
why Py_BuildValue() did not increment the desired object's ref count.

Secondly, was it a correct, or safe, change to turn the "O"s into "N"s
in the format string?  I'm assuming that the pointers given by
PyArray_Return() were simply duplicates, and thus I don't want to
increment their reference counts again -- but assuming the pointers were
duplicates is what got me into trouble in the first place.  Yet it seems
to work here.

Anyway, has anyone had similar problems before?  It seems like a bit of
a gotcha in the API that might bear documenting.  Or maybe I was just
slow.  Heh.

Thanks!
John


Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.