[Python-ideas] [Python-Dev] python and super

[Michael Foord]

On 17/04/2011 14:09, Nick Coghlan wrote:
> On Sun, Apr 17, 2011 at 3:22 AM, Michael Foord
> <fuzzyman at voidspace.org.uk>  wrote:
>> No, not as I described. Where a method does not call up to its parent class
>> then super would *not* auto call the parent class (I'm *not* suggesting a
>> fully auto-call super as the original poster did), but a method not calling
>> super still wouldn't halt the chain of calls. To achieve this a call into a
>> method that doesn't call super (and would normally end the chain) instead
>> removes itself and its unique parents [1] from the mro for this call.
>>
>> It would make the semantics more complex, but it would solve this problem
>> and the original posters problem. For this specific example, if no classes
>> call up to object.__init__ then it won't be called. It would permit you to
>> have methods that can participate in multiple inheritance whilst still
>> blocking calls (overriding) up to their parent classes.
> Could you elaborate on the MRO data structure and super()
> implementation changes you would propose in order to actually make it
> possible to implement this idea?
>
> The current MRO is calculated as a linear list at class definition
> time (using the C3 algorithm), throwing away a lot of the original
> tree information regarding the actual structure of the classes.
>

Well, it isn't going to happen (no-one else is remotely in favour), so 
I'm not going to spend a lot more time on the topic. :-)

> For example, the MRO "C, B, A, object" could come from a class
> hierarchy that looked like:
>
> class A: ...
> class B(A): ...
> class C(B): ...
>
> Or one that looked like:
>
> class A: ...
> class B(A): ...
> class C(B, A): ...
>
> Or one that looked like:
> class A: ...
> class B: ...
> class C(B, A): ...
>
> If you add a "class D(C, B)" to the bottom of that hierarchy, then any
> one of those 3 structures could apply across the parent classes, but D
> would have the same MRO (i.e. "D, C, B, A, object"). And, of course,
> there are now even *more* scenarios that could produce the same MRO
> for D.
>
> So if the only information you have available is D's MRO (which is the
> current situation for super()), then you *don't know* whether B is C's
> parent, sibling or both - the calculation of the MRO discards too much
> detail about the class hierarchy.
>

But given that the mro contains the actual classes, *all* the 
information is available at runtime. (If a call to a method of B 
terminates the chain then super would be able to know what the base 
classes of B are by looking at B - even if that isn't expressed directly 
in the mro.)

> Creating a dynamic MRO for every single method call

It is *only* needed where you are mixing super calls with non super 
calls in the presence of multiple inheritance. So not for every method call.

> just so people can
> fail to think correctly about cooperative super() designs

I disagree with this categorisation. It is particularly for where you 
*are* thinking about cooperative super calls. The current algorithm has 
no way to express "don't call my parent class but continue the 
cooperative calls for other base classes".

This means that there are some multiple inheritance scenarios (like 
__init__ where more than one base class inherits directly from object) 
that you just can't do and use super. The alternative is not to use 
super at all, and for diamond inheritance that makes it hard to avoid 
calling up twice to some methods.

These are all *particularly* problems where you are mixing in classes 
that are from a third party library that you don't control. Even if 
these classes would be *perfectly amenable* for use as mixins via 
multiple inheritance, the current super semantics make that impossible. 
Using composition to get round this can be very messy and require custom 
code to do your dispatch, (again in diamond inheritance situations 
trying to avoid methods being called twice) when inheritance *could* 
just solve the problem (the problem from the original poster being one 
example of this).

Both of these are real problems, and a lot of the task of *explaining* 
super to Python programmers goes to explaining these caveats and when 
you can't use super. Yes I'm suggesting adding complexity to the 
implementation, but it should remove some of the complexity (and 
especially some of the painful caveats) when actually using it.

>   is a huge
> price to pay to gain something that probably won't help in practice.
>
> Auto-chaining of super calls just opens up a whole can of worms, and
> if it can even be done at all, I definitely don't see how it could be
> done in a backwards compatible way.

I'm not suggesting full auto-chaining. And especially given that the 
change only has any effect where you are mixing super calls with 
non-super calls, which you really can't do at the moment, I don't see 
any genuine backwards compatibility issues. If by backwards 
compatibility issues you mean preserving the current structure of 
__mro__, well it *could* be done without changing that - or by creating 
an alternative structure for multiple inheritance. Doesn't look like it 
will happen though. :-)

I'll probably blog about the idea for posterity, and then move on.

All the best,

Michael Foord

> Cheers,
> Nick.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html