[Python-Dev] Problems with the Python Memory Manager

[Travis E. Oliphant]

Armin Rigo wrote:
> Hi,
> 
> Ok, here is the reason for the leak...
> 
> There is in scipy a type called 'int32_arrtype' which inherits from both
> another scipy type called 'signedinteger_arrtype', and from 'int'.
> Obscure!  This is not 100% officially allowed: you are inheriting from
> two C types.  You're living dangerously!
> 
> Now in this case it mostly works as expected, because the parent scipy
> type has no field at all, so it's mostly like inheriting from both
> 'object' and 'int' -- which is allowed, or would be if the bases were
> written in the opposite order.  But still, something confuses the
> fragile logic of typeobject.c.  (I'll leave this bit to scipy people to
> debug :-)
> 
> The net result is that unless you force your own tp_free as in revision
> 1490, the type 'int32_arrtype' has tp_free set to int_free(), which is
> the normal tp_free of 'int' objects.  This causes all deallocated
> int32_arrtype instances to be added to the CPython free list of integers
> instead of being freed!

I can confirm that indeed the int32_arrtype object gets the tp_free slot 
from it's second parent (the python integer type) instead of its first 
parent (the new, empty signed integer type).  I just did a printf after 
PyType_Ready was called to see what the tp_free slot contained, and 
indeed it contained the wrong thing.

I suspect this may also be true of the float64_arrtype as well (which 
inherits from Python's float type).

What I don't understand is why the tp_free slot from the second base 
type got copied over into the tp_free slot of the child.  It should have 
received the tp_free slot of the first parent, right?

I'm still looking for why that would be the case.  I think, though, 
Armin has identified the real culprit of the problem.  I apologize for 
any consternation over the memory manager that may have taken place. 
This problem is obviously an issue of dual inheritance in C.

I understand this is not well tested code, but in principle it should 
work correctly, right?  I'll keep looking to see if I made a mistake in 
believing that the int32_arrtype should have inherited its tp_free slot 
from the first parent and not the second.

-Travis