[Python-Dev] Things to Know About Super

[Michele Simionato]

On Wed, Aug 27, 2008 at 5:15 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> ISTR pointing out on more than one occasion that a major use case for
> co-operative super() is in the implementation of metaclasses.  The __init__
> and __new__ signatures are fixed, multiple inheritance is possible, and
> co-operativeness is a must (as the base class methods *must* be called).
>  I'm hard-pressed to think of a metaclass constructor or initializer that
> I've written in the last half-decade or more where I didn't use super() to
> make it co-operative.
>
> That, IMO, is a compelling use case even if there were not a single other
> example of the need for super.
I have been giving a lot of thought to this use case, at least
from the time of the metaclass conflict recipe. I have always wondered
why the recipe had to be so complicated. At the end, I have come to
the conclusion that the problem was not with the recipe but with
multiple inheritance itself.
Let me explain the argument.

A possible use case for multiple inheritance of metaclasses is the
following: suppose I have a DebugMeta metaclass which adds some
debugging support to classes; now I want to apply it to a third party
framework which uses a FrameworkMeta metaclass internally. Let us
suppose the framework author wrote its metaclass correctly, by
supporting cooperation:

.. code-block:: python

 class FrameworkMeta(type):
    def __new__(mcl, name, bases, dic):
        print "Adding framework features to %s" % name
        return super(FrameworkMeta, mcl).__new__(mcl, name, bases, dic)


>>> class FrameworkClass(object):
...    __metaclass__ = FrameworkMeta
Adding framework features to FrameworkClass

Moreover, suppose I wrote my DebugMeta to support cooperation
correctly:

.. code-block:: python

 class DebugMeta(type):
    def __new__(mcl, name, bases, dic):
        print "Adding debugging features to %s" % name
        return super(DebugMeta, mcl).__new__(mcl, name, bases, dic)


Now I can add the debugging features to a class in this way:

.. code-block:: python

 class DebugFrameworkMeta(DebugMeta, FrameworkMeta):
     pass


>>> class DebugFrameworkClass(FrameworkClass):
...     __metaclass__ = DebugFrameworkMeta
Adding debugging features to DebugFrameworkClass
Adding framework features to DebugFrameworkClass

As you see everything works fine. Now, lets travel in the fictional
world of a fictional language called T-Python which is just like
Python, except it lacks multiple inheritance but has some support for
traits.  By this I mean that there is an ``include_mixin`` function
working more or less like this (it could be enhanced but I am keeping
it dead simple here for the sake of the argument):

.. code-block:: python

 def include_mixin(mixin, cls): # could be extended to use more mixins
     # traits as in Squeak take the precedence over the base class
     dic = vars(mixin).copy() # could be extended to walk the ancestors
     return type(mixin.__name__ + cls.__name__, (cls,),  dic)


I will argue that T-Python is not worse than Python for this use
case (composition of metaclasses).

In the fictional world there is not need for super since
all hierarchies are linear and you can just call the base class;
FrameworkMeta could have been written as

.. code-block:: python

 class FrameworkMeta2(type):
    def __new__(mcl, name, bases, dic):
        print "Adding framework features to %s" % name
        return type.__new__(mcl, name, bases, dic)


and DebugMetas as

.. code-block:: python

 class DebugMeta2(type):
    def __new__(mcl, name, bases, dic):
        print "Adding debugging features to %s" % name
        return mcl.__base__.__new__(mcl, name, bases, dic)


Notice that DebugMeta2 is performing a sort of cooperative call here
(``mcl.__base__.__new__``) but dead simple since there is just one base class.

The analogous of FrameworkClass can be defined as

>>> class FrameworkClass2(object):
...     __metaclass__ = FrameworkMeta2
Adding framework features to FrameworkClass2

and the analogous of DebugFrameworkClass as

>>> class DebugFrameworkClass2(FrameworkClass2):
...     __metaclass__ = DebugFrameworkMeta2
Adding debugging features to DebugFrameworkClass2
Adding framework features to DebugFrameworkClass2

So, as you see, it works. Checks of the kind
``isinstance(DebugFrameworkClass2, DebugMeta2)`` would fail, but this
is not a big issue (isinstance should not be used, or you could
register DebugMeta2 as a base class even if it is not by using
Python 2.6 ABC's).

Now, I am not claiming that I have thought of all possible usages of
multiple inheritance and metaclasses: however I have not found yet a
use case that I could not rewrite by using single-inheritance + traits
as the one I have just shown. Possibly there are cases which are
difficult to rewrite: but how common are they?

Notice that I am not advocating rewriting Python. The argument here is
purely hyphotetic and concerning a fictional language: I just want to
understand if full multiple inheritance is really that useful or not,
as a matter of principle.


  M.S.