inheritance, multiple inheritance and the weaklist and instance dictionaries

[Rouslan Korneychuk]

On 02/09/2011 04:58 PM, Carl Banks wrote:
> On Feb 9, 1:14 pm, Rouslan Korneychuk<rousl... at msn.com>  wrote:
>> On 02/09/2011 02:42 PM, Carl Banks wrote:
>>> This is the only case I can think of where the
>>> layout conflict would be caused by a type setting tp_dictoffset.
>>
>> No, actually I have code that is roughly equivalent to the following
>> pseudocode:
>>
>> class _internal_class(object):
>>       __slots__ = ()
>>
>> class BaseA(_internal_class):
>>       __slots__ = (some_data,...,__weaklist__,__dict__)
>>
>> class BaseB(BaseA):
>>       __slots__ = (some_data,...,__weaklist__,__dict__)
>>
>> class BaseC(_internal_class):
>>       __slots__ = (some_other_data,...,__weaklist__,__dict__)
>>
>> class Derived(BaseB,BaseC):
>>       __slots__ = (combined_data,...,__weaklist__,__dict__)
>
> Ah, ok.  So BaseA sticks weaklist and dict into a certain layout
> location.  BaseB gets the same location because it has the same
> layout.
> BaseC sticks weaklist and dict into a different location.  Derived
> then can't reconcile it because dict and weakref are in different
> locations.

That doesn't explain why changing _internal_class to:
     class _internal_class(object):
         __slots__ = (__weaklist__,__dict__)
makes it work. Also, why does Derived need to reconcile anything if I'm 
telling it where the dictionaries are in the new class? Doesn't CPython 
calculate where, for example, weaklist is with what is essentially:
     obj_instance + obj_instance->ob_type->tp_weaklistoffset
(at least when tp_weaklistoffset is non-negative)? If so, then it 
wouldn't matter what the base classes look like.


> "some_data" a proper subset of "some_other_data", right?  (If it isn't
> you have worse problems than dict and weakreflist.)

Not at all. The point of this, is to allow C++ multiple inheritance to 
be mapped to Python multiple inheritance. I solve this issue as follows: 
All attributes of base classes of multiply-inheriting classes are turned 
into properties. Any time the wrapped C++ object needs to be accessed, 
the PyObject self variable's type is compared to every derived type that 
also inherits from another type. If it matches a derived type, a 
reinterpret_cast is performed on the C++ object to the derived C++ type, 
and then an implicit cast is performed to the intended type, which 
allows the proper pointer fix-up to happen. Here is the actual function 
that does that for BaseA:

BaseA &get_base_BaseA(PyObject *x,bool safe = true) {
 
if(reinterpret_cast<long>(static_cast<BaseA*>(reinterpret_cast<Derived*>(1))) 
!= 1 &&
 
PyObject_IsInstance(x,reinterpret_cast<PyObject*>(get_obj_DerivedType())))
         return cast_base_Derived(x);

     if(UNLIKELY(safe && 
!PyObject_IsInstance(x,reinterpret_cast<PyObject*>(get_obj_BaseAType())))) {
         PyErr_SetString(PyExc_TypeError,"object is not an instance of 
BaseA");
         throw py_error_set();
     }
 
assert(PyObject_IsInstance(x,reinterpret_cast<PyObject*>(get_obj_BaseAType())));
     return cast_base_BaseA(x);
}

(cast_base_X checks how the C++ object is stored and retrieves a 
reference to it. The 
"reinterpret_cast<long>(static_cast<BaseA*>(reinterpret_cast<Derived*>(1))) 
!= 1" part is an optimization trick. It checks to see if a pointer 
fix-up is even necessary. If not, that 'if' statement will be subject to 
dead code removal.)

Of course, if this was hand-written code, such a method would be very 
brittle and would be a nightmare to maintain, but it's generated by a 
Python program and only needs to be correct.


> Don't set tp_dictoffset and tp_weakrefoffset in any of the bases.  Set
> them only in Derived.  If you need to instantiate a Base, make a
> trivial dervied class for it.

That would mean calling isinstance(derived_instance,BaseType) would 
return false. In any case, I'm writing a general-purpose tool and not 
trying to solve a specific problem. If I can't do what I'm trying to do 
here, I can just make the program not allow multiply-inheriting classes 
to derive (directly or indirectly) from classes that have a different 
combination of weaklist and dict (having both will be the default 
anyway, so this is a special case).