[Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

[Michael Abbott]

Travis, thank you for your encouraging words.

On Sat, 19 Jul 2008, Travis E. Oliphant wrote:
> Really this code "grew" from a simple thing into a complicated thing as 
> more "features" were added.  This is a common issue that happens all 
> over the place.  
Aye.

> The reason I say I'm not sure is that I don't remember seeing a DECREF 
> after the PyArray_Scalar in the obj = NULL part of the code in my 
> looking at your patches.    But, I could have just missed it.   
There wasn't -- instead, I was trying to retain the typecode (and paying 
the price of releasing it on all the early returns!)

>  From a "generic" reference counting point of view you did correctly 
> emphasize the problem of having a reference count creation occur in an 
> if-statement but a DECREF occur all the time in the finish: section of 
> the code. 
Yah -- I think the core idea I was trying to get over is that of an 
"invariant" property at each point in the code to capture what needs to be 
true for the code to be correct.

> It was really that statement: "the fantasy that PyArray_Scalar steals a 
> reference" that tipped me off to what I consider one of the real 
> problems to be.    The fact that it was masked at the end of a long 
> discussion about other reference counting and a "stealing" discussion 
> that were not the core problem was distracting and ultimately not very 
> helpful. 
That was the really hard bit for me.  To me the issue was actually very 
obvious (though I didn't realise that typecode could be regenerated, which 
simplifies things enormously), so the problem was trying to figure out 
what you and Charles were not seeing.  I think that's why I ended up 
throwing everything into the pot!

> I'm very impressed with your ability to follow these reference count 
> issues.  Especially given that you only started learning about the 
> Python C-API a few months ago (if I remember correctly).   
Alas no.  I'm a bit of an old lag really, I did dabble with the Python C 
API quite a few years ago (2001ish maybe?).  Myy roots are in computer 
science and then assembler (graduated 1980) before Pascal (seriously) then 
C, then C++ (which I now regard as a serious mistake) and finally shell 
script plus Python, all largely on embedded applications.  I'd love the 
opportunity to learn and use Haskell now.

> I'm also very glad you are checking these corner cases which have not 
> received the testing that they deserve.  I hope we have not discouraged 
> you too much from continuing to help.  Your input is highly valued.

Maybe I'll have a further play.  The memory leak issue was a direct 
consequence of using numpy in an embedded application, and that issue's 
closed now, but I ought to see if this painful code can be revisited.

I'm learning my way around git and have just used `git svn` to grab (and 
update) the numpy repository.  I'm hugely impressed by it, though it is 
very expensive to run the first time -- it fetches every single svn 
revision!  Hopefully that didn't overload the web server...  This will 
make working on patches much easier.