[Python-Dev] Problems with the Python Memory Manager
Travis E. Oliphant
oliphant.travis at ieee.org
Thu Nov 24 18:02:59 CET 2005
Armin Rigo wrote:
> Ok, here is the reason for the leak...
> There is in scipy a type called 'int32_arrtype' which inherits from both
> another scipy type called 'signedinteger_arrtype', and from 'int'.
> Obscure! This is not 100% officially allowed: you are inheriting from
> two C types. You're living dangerously!
> Now in this case it mostly works as expected, because the parent scipy
> type has no field at all, so it's mostly like inheriting from both
> 'object' and 'int' -- which is allowed, or would be if the bases were
> written in the opposite order. But still, something confuses the
> fragile logic of typeobject.c. (I'll leave this bit to scipy people to
> debug :-)
> The net result is that unless you force your own tp_free as in revision
> 1490, the type 'int32_arrtype' has tp_free set to int_free(), which is
> the normal tp_free of 'int' objects. This causes all deallocated
> int32_arrtype instances to be added to the CPython free list of integers
> instead of being freed!
I can confirm that indeed the int32_arrtype object gets the tp_free slot
from it's second parent (the python integer type) instead of its first
parent (the new, empty signed integer type). I just did a printf after
PyType_Ready was called to see what the tp_free slot contained, and
indeed it contained the wrong thing.
I suspect this may also be true of the float64_arrtype as well (which
inherits from Python's float type).
What I don't understand is why the tp_free slot from the second base
type got copied over into the tp_free slot of the child. It should have
received the tp_free slot of the first parent, right?
I'm still looking for why that would be the case. I think, though,
Armin has identified the real culprit of the problem. I apologize for
any consternation over the memory manager that may have taken place.
This problem is obviously an issue of dual inheritance in C.
I understand this is not well tested code, but in principle it should
work correctly, right? I'll keep looking to see if I made a mistake in
believing that the int32_arrtype should have inherited its tp_free slot
from the first parent and not the second.
More information about the Python-Dev