[Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[PJ Eby]

On Mon, Apr 28, 2014 at 7:26 PM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

> Well, sure I did, as I mentioned, but as that's first time I see that
> code (that specific piece is in typeobject.c:extra_ivars()), it would
> take quite some time be sure I understand all aspects of it. Thanks for
> confirming that it's governed essentially by CPython implementation
> details and not some language-level semantics like metaclasses (I
> mentioned them because error message in Python2 did so, though Python3
> doesn't refer to metaclasses).
>
> An example would really help me to get a feel of the issue, but I
> assume lack of them means that there's no well-known idiom where such
> inheritance is used, and that's good enough on its own. I also tried to
> figure how it's important to support such multi-base cases, so the code
> I write didn't require complete rewrite if it hits one day, but
> everything seems to turn out to be pretty extensible.
>

>From memory of the last time I dealt with this, the rules were that you
could mix two classes only if their __slots__ differed from their common
__base__ by *at most* __dict__ and/or __weakref__.  The dict and weakref
slots are special, in that the type structure contains their offsets, which
makes them relocatable in subclasses.  But any other __slots__ aren't
relocatable in subclasses, because the type structure doesn't directly keep
track of the offsets.  (The slot descriptors do.)

But I don't think there's anything in principle that requires this, it's
just the implementation.  You could in theory relocate __slots__ defined
from Python code in order to make a merged subclass.  It's just that the
effective "__slots__" of C code can't be moved, because C code is expecting
to find them at specific offsets.  Therefore, if two types define their own
struct fields, they can't be inherited from unless one is a subtype of the
other.

In the C code (again if I recall correctly), this is done using the
__base__ attribute of the type, which indicates what struct layout the
object will use.  A type can have a larger struct than its base type,
adding its own fields after the base type's struct fields.  (The dict and
weakref fields are added -- if they are added -- *after* the base struct
fields.  If your __base__ already has them, those offsets within the
existing layout are used, which is why them being in another base class's
__slots__ isn't a problem.)

When you create a new type, CPython looks at your bases to find a suitable
__base__.  If two of your bases inherit from each other, the ancestor can
be ignored, keeping the more-derived one as a candidate __base__.  If a
base adds only __dict__ and/or __weakref__ (or neither) to its __base__,
then its __base__ is a candidate (not recursively, though).  If at the end
there is more than one base left standing, then it's an error, since you
have bases with incompatible layouts.

That is not a precise description of the algorithm, but that's the gist of
how it works.  __base__ is a slot on the type object and is tracked at the
C level in order to sort out layouts like this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140429/55ae6f1f/attachment.html>