inheritance, multiple inheritance and the weaklist and instance dictionaries

[Carl Banks]

On Feb 9, 3:11 pm, Rouslan Korneychuk <rousl... at msn.com> wrote:
> On 02/09/2011 04:58 PM, Carl Banks wrote:
> > On Feb 9, 1:14 pm, Rouslan Korneychuk<rousl... at msn.com>  wrote:
> >> On 02/09/2011 02:42 PM, Carl Banks wrote:
> >>> This is the only case I can think of where the
> >>> layout conflict would be caused by a type setting tp_dictoffset.
>
> >> No, actually I have code that is roughly equivalent to the following
> >> pseudocode:
>
> >> class _internal_class(object):
> >>       __slots__ = ()
>
> >> class BaseA(_internal_class):
> >>       __slots__ = (some_data,...,__weaklist__,__dict__)
>
> >> class BaseB(BaseA):
> >>       __slots__ = (some_data,...,__weaklist__,__dict__)
>
> >> class BaseC(_internal_class):
> >>       __slots__ = (some_other_data,...,__weaklist__,__dict__)
>
> >> class Derived(BaseB,BaseC):
> >>       __slots__ = (combined_data,...,__weaklist__,__dict__)
>
> > Ah, ok.  So BaseA sticks weaklist and dict into a certain layout
> > location.  BaseB gets the same location because it has the same
> > layout.
> > BaseC sticks weaklist and dict into a different location.  Derived
> > then can't reconcile it because dict and weakref are in different
> > locations.
>
> That doesn't explain why changing _internal_class to:
>      class _internal_class(object):
>          __slots__ = (__weaklist__,__dict__)
> makes it work.

Yes it does.  When weaklist and dict are slots in the base class, then
all the derived classes inherit those slots.  So tp_dictoffset doesn't
end up with one value in BaseB and a different value in BaseC.  For
the layouts to be compatible the dicts have to be in the same
location.


> Also, why does Derived need to reconcile anything if I'm
> telling it where the dictionaries are in the new class? Doesn't CPython
> calculate where, for example, weaklist is with what is essentially:
>      obj_instance + obj_instance->ob_type->tp_weaklistoffset
> (at least when tp_weaklistoffset is non-negative)? If so, then it
> wouldn't matter what the base classes look like.

I explained why in my last post; there's a bunch of reasons.
Generally you can't assume someone's going to go through the type
structure to find the object's dict, nor can you expect inherited
methods to always use the derived class's type structure (some methods
might use their own type's tp_dictoffset or tp_weakreflist, which
would be wrong if called from a superclass that changes those
values).  Even if you are careful to avoid such usage, the Python
interpreter can't be sure.  So it has to check for layout conflicts,
and these checks would become very complex if it allowed dict and
weakreflist to appear in different locations in the layout (it's have
to check a lot more).


> > "some_data" a proper subset of "some_other_data", right?  (If it isn't
> > you have worse problems than dict and weakreflist.)
>
> Not at all. The point of this, is to allow C++ multiple inheritance to
> be mapped to Python multiple inheritance.

I would say you do.  Python's type system specifies that a derived
type's layout is a superset of its base types' layout.  You seem to
have found a way to derive a type without a common layout, perhaps by
exploiting a bug, and you claim to be able to keep data access
straight.  But Python types are not intended to work that way, and you
are asking for trouble if you try to do it.

I guess there's also no point in arguing that tp_dictoffset and
tp_weakreflist need to have the same value for base and derived types,
since you're rejecting the premise that layouts need to be
compatible.  Therefore, I'll only point out that the layout checking
code is based on this premise, so that's why you're running afoul of
it.


[snip complicated variable access code]


> > Don't set tp_dictoffset and tp_weakrefoffset in any of the bases.  Set
> > them only in Derived.  If you need to instantiate a Base, make a
> > trivial dervied class for it.
>
> That would mean calling isinstance(derived_instance,BaseType) would
> return false.

You claimed in another post you weren't trying to mimic the C++ type
hierarchy in Python, but this line suggests you are.

My suggestion was that you shouldn't; just don't create a type
hierarchy at all in Python.  Even more so if you're using Python 3,
where isinstance() is customizable.


Carl Banks