On Sat, Mar 26, 2022 at 10:24 AM malmiteria <martin.milon@ensc.fr> wrote:

i mean yeah, of course you don't need super. And i also understand that feeling that composition is better than inheritance.

That's a bit beside the point -- we can use inheritance, and also be more explicit about calling superclass methods if need be.

But that's beside my point, super acts as a proxy to a parent, and is by default the feature to refer a parent from within a class method.

Almost -- I think super() acts a proxy to the *perents*, plural. even if the current class inherits from only one class, that superclass may inherit from more than one class. If you just want the superclass, you can call it directly: class B(A): def some_method(self, ...): A.some_method(self) And if you do that, you will get a good fraction of the behaviour you are expecting. The point of super() is to provide a way to, yes, implicitly, call a method on all the superclasses in the inheritance tree. There's more than one way to do that, but it's perfectly reasonable that Python provide only one way, with clear rules, which is what we have now. And if there's going to be one way, the MRO currently in use is a pretty darn good one. Yes, sometimes the specified MRO isn't what you need -- but It's my impression that your proposal really doesn't add anything over simply calling superclasses directly the way you want. But if you want to show how useful it could be, then go ahead and continue working on your prototype,and share it with the world. If people find it useful, then maybe it would be worth considering as a addition to Python. NOTE: I would work hard to find real examples where you can show how the current super() doesn't work well, and how explicitly calling superclass methods is substantially uglier than using your parent() approach. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On 2022-03-26 11:15, malmiteria wrote:

the alternative to super really is not the important part of my proposal, it's the alternative to MRO.

An example of a case where i genuinly believe my solution adds value is this :

``` class A: def method(self): print("A")

class B: def method(self): print("B")

class C(A, B): pass ```

Today, a code such as ```C().method()``` works without any problems except of course when the method you wanted to refer to was the method from B.

But that's what people have been saying: if you want to call the method from B, just call the method from B.

If class A and B both come from libraries you don't own, and for some reason have each a method named the same (named run, for example) the run method of C is silently ignoring the run method of B.

I believe it would make the programmers life easier to have an error at this time, or anything tbh, to make explicit this resolution, because it really is not explicit from most programmers perspective

To me it doesn't seem reasonable that someone would inherit from two classes and want to call a method from one without even knowing that there's a method name collision. If you're going to inherit from A and B, you need to know what methods they provide and you need to think about the order you inherit in. If you inherit from two classes that have a name collision because they provide unrelated methods that happen to share a name, I think you have a problem right there that needs to be resolved (like by having the inheriting class explicitly delegate to a specific superclass). But I don't think that is a very common situation. Based on your example, I still don't really see the advantage of your proposal. Currently super is the way to say "let Python choose for me what method implementation to call next, based on the class hierarchy". You seem to be saying that in some cases that won't call the right thing so you don't want it to call anything. But if super doesn't call the right thing, you can just not use super and instead explicitly call the method you want, so I don't get what this proposal gains. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

malmiteria writes:

Why would this be the order you wanna call all the methods provided by the parents?

It works often. ;-) My guess would be that typical multiple inheritance scenarios are of the form "I want my child class to have the behavior of classes earlier in the declaration order, *unless* the behavior is inherited and I explicitly override it by explicitly declaring an overriding class." Why this would work "right" in cases like class C(A, B) where A inherits a method from its grandparent and B inherits from its parent, I don't know. But that's the way I expect C to behave when A inherits a method and B implements it directly.

It seems possible that for some method, the wanted order won't be the same all throughout the class.

That's why super can take arguments. I know you don't like the arguments it takes, but it makes it possible to resolve the problem, although for methods not defined in the parents you may have to reach arbitrarily far into the MRO to get the instance you want of that method. I doubt that __as_parent__ solves the "arbitrarily deep" problem (although you may be able to persuade me it will work "better" "most of the time").

I think most people when they use super are just saying, call the parent method, since that's what it visibly does in simple inheritance, and that's what most people will face in their programming lives.

True. But that's irrelevant to this discussion, because (a) there are people who use it for more advanced purposes and (b) it can't cause you any confusion in that case.

The problem still relies on MRO in the end, which is the thing that needs reworking the most: My point is that : it is not correct to assume it is meaningful to give an order to multiple inheritence

But C3/super assumes nothing of the kind. First, in some cases the C3 algorithm fails to produce an MRO, and you get an error at class definition time. Second, of course it's *meaningful* to give an order to multiple inheritance. Sometimes that order is *not useful*. Other times it may be *confusing*. Pragmatically, C3 seems to give useful results frequently, unuseful or confusing results infrequently (except for people who don't understand C3 and randomly change code, who will be confused frequently).

It make sense to say a child is more specific than its parents, because that's literraly what inheritance is, but saying that parent 1 is more specific than parent 2 is... weird to say the least.

It doesn't say that. It says that parent 1 comes before parent 2 in the MRO. It turns out that this is useful to quite a few Python programmers.

Please re read the list of features i proposed for the alternative to MRO

Have you specified when the ExplicitResolutionRequired error is raised (at class declaration time or method resolution time)? I don't see it. (No, I'm not going to read your code for the implementation, I'm interested in the specification.) You can't have "reliability" without disabling super, I suspect, since you can't prevent stdlib or 3rd-party code from using it, and I believe that means method resolution in that code will depend on MROs, even if you can specify the root of the class subtree explicitly. I doubt disabling super would be acceptable to Python core devs, and it would very likely make the stdlib less than useful to you. I'm not sure I like "reliability" as a requirement. The point of MRO is that it is an attribute of the object. That means that if the object is a subclass of list and it has a my_sort method, regardless of which class in the tree calls super().my_sort, it will get the same order. You can assume that any method documented as "sorts the list in-place" will leave it in the same order. But with a "reliable" __as_parent__, it's possible that a method defined in the child calls __as_parent(SmallestFirst).my_sort(), while a method in the parent calls __as_parent(LargestFirst).my_sort(). This may be a problem. I don't understand "ancestors as targets". If there's a name collision between grandparents, that will already raise ExplicitResolutionRequired. So a child of those grandparents must use __as_parent__, which resolves the problem in the default case. Do you just mean that if for this particular grandchild, you don't want the default resolution, you can use __as_parent__(grandparent_a)? I don't understand "transmitting errors". If the error is raised at declaration of the parent, you don't need to transmit; you won't get to execute the declaration of the child. If the error is raised at method resolution time, what happens if you never try to resolve it? How about a case where the parent doesn't specify a resolution of a collision between grandparents, but no code tries to resolve that method? Shouldn't the child be allowed to use __as_parent__ there?

It is very similar to MRO in cases there's no conflict, but simply raises an error in case there's a conflict.

I don't much like your definition of "conflict". Mine is "when super() does the wrong thing", which (surprise!) it almost never does for me, and I've never used the two-argument form (except in my other post to you): class.method calls have always been adequate. I've never seen a case where I couldn't diagnose the need to override the MRO from the class declaration. Not saying problem cases don't exist, just that I'm perfectly happy with C3 MRO and super().

On Tue, 29 Mar 2022 at 04:24, malmiteria <martin.milon@ensc.fr> wrote:

Essentially, when two parent class provide a method with the same name, that's what i call a conflict.

You keep saying this, but I'm still confused: how is that different from the very normal behaviour of overriding a method? What makes one of them a critical feature of inheritance, and the other a conflict that has to be resolved? How do you distinguish conflicts from overrides? My best understanding so far is that you're doing something like: class Pizza(Crust, Topping): ... and then you could have a conflict like this: class StuffedCrust(Crust): def add_cheese(self): ... class ThreeCheeseTopping(Topping): def add_cheese(self): ... which would result in an error when you try to build this pizza: class AllTheCheese(StuffedCrust, ThreeCheeseTopping): ... But the problem with this example is that it's using Crust and Topping as superclasses when they're really not. This would be MUCH better served by something like: class Pizza: def __init__(self, Crust, Topping): self.crust = Crust() self.topping = Topping() And now you don't have a conflict. Please, if this is NOT what you're talking about, can you show us an actual example? Far too many toy examples that don't explain the problem, no realistic (or at least semi-realistic) examples that explain the underlying intention. ChrisA

malmiteria writes:

class An: def method(self): print('An') class A[n-1](An): pass ... class A1(A2): pass class A0(A1): def method(self): self.__as_parent__(An).method()

A0().method() prints 'An'

Of course it prints 'An', and so does class A00(A1): def method(self): super(A[n-1], self).method() A00().method() Equally of course, __as_parent__(An) is exactly what I mean by "arbitrarily deep". Note that all you need to identify A[n-1] in this case is its class definition, and that's probably the great majority of cases. To find out, all you need to do is type

...

...

class A00(A1): pass ... A00.__mro__ [A00, A1, ..., A[n-1], An]

so it's not hard to figure this out.

...

(a) there are people who use it for more advanced purposes and (b) it can't cause you any confusion in that case.

(a) What are those more advanced feature? On top of the classic inheritance features, I can only think of dependency injection in the inheritance tree.

Yes, that's precisely what I have in mind. However, the application is declarative composition (I think I just invented that name), as "Python's super() considered super" demonstrates with the trivial class LoggingOD[1]. This is very powerful and expressive (when it works). At this time, it seems like C3 provides the largest set of cases where it works.

Such a feature could be replaced, although i didn't provide a replacment for this feature, it doesn't seem like it's gonna be hard to do, __as_parent__ could rely on a dict, which would hold as key a class, and as value a class too. If the targeted class is in the keys, target the corresponding value instead. This class could then inherit from the class it replaced, and you got your dependency injection.

Have you read "Python's super() considered super!"? I definitely get the impression that you have not understood it. The point of several of the examples is that by using super() you can allow a future subclass to insert a method implementation in the MRO without knowing either the future subclass nor the class that currently implements the method. There are constraints on it, but the MRO allows you to express many such dependencies in a straightforward declarative manner, whereas your dict implementation is going to require a lot of explicit plumbing in derived classes. Also, the MRO approach allows different classes to inject different methods, but you're going to need something more complicated than the simple class -> class mapping.

On top of that, it unties a lot more this dependency injection feature from the method resolution algorithm.

You write that like you think that's a good thing, or it's not already possible to do that. ;-)

(b) Wrong! The common use case for inheritance and super is what informs most people of its behavior,

I think that's irrelevant, since the most common case is single inheritance, and there's only one (sane) way to resolve methods in that case. Trying to extrapolate from that to behavior in a multiple inheritance context simply means you haven't thought about multiple inheritance.

My alternative doesn't do those jumps, and i think overall matches more closely what an untrained mind [...] would expect.

I think that's probably the worst possible criterion for systems design. "Principle of Least Astonishment", sure, but at some point you want your systems to have some power. In theory any 3rd grader could fly a plane with a Wii handset, but maybe it's a better idea to employ a trained pilot.

...

But C3/super assumes nothing of the kind.

At first i was gonna answer that it of course does make this assumption, but I think we aren't exactly talking about the same thing. I was saying that the assumption that it can make sense to order class inheritance trees

C3/super doesn't *assume* it makes sense. It proposes it, and then demonstrates that it satisfies the useful and intuitive properties of local precedence ordering and monotonicity. Experience shows it *obviously* *does* make sense in many cases, in particular with single inheritance. But there are also many cases where it makes sense with multiple inheritance.

which it turns out can't solve all scenarios. I think this proves that this assumption was wrong.

But the assumption that something needs to solve all scenarios to be useful and usable is *your* assumption, which is clearly *wrong*.

So overall, i think we agree on the matter of fact of today's C3 behavior, and overall, i think we agree that C3 is an incomplete solution?

Yes. Who cares?

The solution i propose would be able to cover those cases C3 doesn't. Because it wasn't build around the idea of ordering.

Sure, but it gives up on many useful cases of declarative composition, basically limiting it to sets of mixins with disjoint attribute sets. And in any case, super() also covers those cases, and a few more besides.

Not knowledgable people should be accounted for to.

Sorry, not by deleting features, please. Less knowledgeable people should be taught. That's all the accounting they need.

Knowledgable people would benefit from an easier to use langage anyways.

I guess it would be easier to *use* super's first argument if it were the target class instead of the class preceding it in the MRO. But then it would be hard, and to me more confusing, to express the constraint on the second argument. Not an obvious win.

But yeah, I agree that the confusing behavior are not in the majority. They still exists.

The question is do they exist among programmers who read the Library Reference. Seriously, if you're not going to read that for super, you deserve what you get. super() is obviously magic (all other ways to call a method require an explicit object (including class) to bind it to). Anybody who has been annoyed by def __init__(self, x): self.x = x and doesn't realize super() is magic and requires study is gonna Find Out.

...

It turns out that this is useful to quite a few Python programmers.

No matter what you feed a comunity, some people will make gold out of.

Declarative composition that works is gold by design, even if it only works half the time.

What matters here is that if we change anything, the new world allows what the old one did. Feature wise, I mean.

Your proposal does not, since it is specifically designed to prevent useful behavior that you find confusing, and does so in a draconian way. There is *zero* reason to error on C or D below, because it's completely obvious which parent's method is desired. class A(object): # parent specified for emphasis def method(self): pass class B(object): def method(self): pass def C(A, B): pass def D(B, A): pass I guess you could allow both your approach and super, but then you have the problem that the interactions are definitely going to be confusing.

Do we have a list of use case / features provided by the current state of MRO + super?

The features and two typical use cases are described in the Library Reference. Declarative composition is described in "Python's super() considered super!" Desiderata for the MRO and the C3 algorithm are described in https://www.python.org/download/releases/2.3/mro/. I'll grant the docstring for super is not very helpful, but it's very easy to find those other references. It doesn't speak well for you that you didn't check them.

We would wanna keep those behavior in the alternative.

You can't. That's the whole point of your proposal, to get rid of behavior you don't like.

To me, on top of the classic inheritance features, there's only the dependency injection in the inheritance tree.

Well, the rest of us think that's very important.

As explained above, it isn't hard to produce an alternative for this feature in the realm of my alternative to MRO + super.

I think it's harder than you think, as explained above. I don't see how your proposal works if you don't know the detailed inheritance tree of the child class, and it certainly won't be robust to refactoring in the way that super() is.

We can think of a few stuff, like adding a flag to be able to switch from one feature to the other.

Since we can already do everything with super() that we can with your approach, this adds complexity for no benefit.

Well, [__mro__] *has* to be a class attribute for it to work.

What? Of course it doesn't. It could be computed at each attribute access. And it, too, is magic (try '__mro__' in dir(subclass)).

Obviously, it feels weird to get rid of it, because today mro is so unobvious that having it has a class attribute is a must.

What? "Method [in fact, attribute] resolution order" is quite obvious. In a von Neumann architecture, there will be an order (although it might be non-deterministic). The question then is can we make it deterministic, in a "nice" way. C3 has a couple of nice properties: 1. Local precedence ordering: Where class C(A, B), C precedes A precedes B regardless of the inheritance trees of A and B. 2. Monotonicity: If C1 precedes C2 in the linearization of C, then C1 precedes C2 in the linearization of any subclass of C. That's about as good as it's going to be.

but my solution just doesn't need.

Which strongly suggests that super() has capabilities that __as_parent__() does not.

...

But with a "reliable" __as_parent__, it's possible that a method defined in the child calls __as_parent(SmallestFirst).my_sort(), while a method in the parent calls __as_parent(LargestFirst).my_sort(). This may be a problem

I'm sorry, i completely fail to understand what you mean, can you provide a code example to illustrate what you mean?

It's very simple: two ancestor classes A, B, derived from list both provide an in-place 'my_sort', but they generate the reversed order from each other. A parent class C derives from A, B and uses __as_parent__(A).my_sort in a method bisect_search. Then the child class D uses __as_parent__(B).my_sort in one context and __as_parent__(C).bisect_search in another. Then the underlying list will be in different orders after my_sort vs. bisect_search. This does not happen if super() is used properly. Avoiding this is the responsibility of the programmer in your model.

...

I don't much like your definition of "conflict". Mine is "when super() does the wrong thing", which (surprise!) it almost never does for me, and I've never used the two-argument form (except in my other post to you): class.method calls have always been adequate. I've never seen a case where I couldn't diagnose the need to override the MRO from the class declaration. Not saying problem cases don't exist, just that I'm perfectly happy with C3 MRO and super().

I think what i refer to as conflict is what you refer to as collision.

Yes, I understand that. super() resolves the collision without a conflict. Sometimes we may not like the particular resolution. Much of the time reordering the parent classes will give us what we want. When it doesn't, typically the two-argument version of super will do so.

Idk how often you are confronted to multiple inheritance,

All the time. Code I work on uses a lot of mixins.

I personnaly am fairly often, for example with class based views in django, and It is not uncommon to have to place a mixin before the View class we wanna inherit from. A possible reason is that the View class has colliding methods with the mixin (maybe simply __init__) in which they don't do a call to super, as they are at the top part of the inheritance tree. Which in term, completely lose a branch of the inheritance tree __init__.

I don't understand what you lose here. This is way too abstract to make sense.

When designing View, who can blame them for not thinking of that?

This usually isn't all that difficult to work around, though. You just derive a child of the View class that *does* use super() where needed.

On top of that, if the collision is not on a dunder method, adding a call to super in the top parent class would raise an error called outside multiple inheritance, as its parent don't have those method, but in case of multiple inheritance, it's needed. How are anyone supposed to solve for that?

I have no idea what you're talking about. "Python’s super() considered super!" explains that you can interpose a "root" class whose only job is to stop super() from trying a further level of parents. In the case above, that root class would be WrappedView. If View doesn't def or inherit a method, WrappedView's implementation of that method can either assert, raise a handleable exception, or just no-op.

I also wanna point out that most of us here are very familiar with the current state of MRO + super, and our easiness to work with it could show that we are knowledgable as much as it could show that the feature is easy to use. But it doesn't distinguish between those two explanation, so i'm not confortable with this argument.

System design is hard. I don't think super() makes it harder. If you don't need to allow composability when deriving new classes, don't use super(). If you do need it, most of the time it's trivial to use: just prefix the method call with "super()." This ensures consistency of method resolution and robustness against refactoring of a parent class. The only time it's hard is if you are being deliberately inconsistent (this is not a bad thing, just requires more knowledge). In that case you always have the option to hardcode the call to the method implementation in an appropriate ancestor, or take full advantage of the child's MRO (true, that's wizardry, but sometimes it's a useful option). I don't think __as_parent__ makes system design *harder*, but it does put substantial burden on the programmer to ensure consistency, and I suspect it requires more knowledge of the inheritance tree than use of super does in the majority of actual cases. Footnotes: [1] Of course LoggingOD provides nothing in recent Python since the current implementation of dict (also a Hettinger innovation IIRC!) already provides "insertion order is iteration order". But it does show how powerful declarative composition was before that dict implementation was introduced.

On Sat, Mar 26, 2022 at 06:15:57PM -0000, malmiteria wrote:

class C(A, B): pass ```

Today, a code such as ```C().method()``` works without any problems

Actually, no, it does not. Only the A.method() runs, because A was not designed for multiple-inheritance. C inherits from both A and B, but only calls one of the methods.

except of course when the method you wanted to refer to was the method from B.

Why are you inheriting from A if you don't want to inherit from A?

If class A and B both come from libraries you don't own, and for some reason have each a method named the same (named run, for example) the run method of C is silently ignoring the run method of B.

Right. Multiple inheritence in Python is **cooperative** -- all of the classes in question have to work together. If they don't, as A and B don't, bad things happen. You can't just inherit from arbitrary classes that don't work together. "Uncooperative multiple inheritance" is an unsolvable problem, and is best refactored using composition instead. -- Steve

On Sun, 27 Mar 2022 at 23:03, malmiteria <martin.milon@ensc.fr> wrote:

...

Actually, no, it does not. Only the A.method() runs, because A was not designed for multiple-inheritance. C inherits from both A and B, but only calls one of the methods.

Yeah, my point was to say that no error is raised, but that the behavior of the program is not really what you'd expect, so yeah.

I'm not sure what you DO expect, though. You're making the assumption here that nobody expects the Spanish resolution order, but I think my experience with Python is very different from yours, so you're going to have to go into a bit more detail about what you expect here. One of the reasons to use the MRO that Python currently does is that it is perfectly possible and plausible to change the MRO by subclassing again. That is to say, even if class C's MRO is (C, B, A, object), you could create a class D which has an MRO of (D, C, Mixin, B, A, object), which would then allow D to affect the behaviour of C's super() calls. How does your proposal handle this? What is its definition of "conflict"? In proper cooperative multiple inheritance, there's no such thing as a conflict; as the programmer, you choose which methods override and which methods augment, simply by creating those methods and choosing whether to call super's method or not.

Essentially, i have a problem with code behavior being unexpected, and my proposal would add an error here, to attract the attention of the developper, for them to solve that problem.

I'm very unclear on what constitutes an error. Simply having the same method name in multiple parents is most definitely NOT an error, as that's the entire point of inheritance and super.

And yeah, A is not designed for multiple inheritence. Today it would have to be designed for it, but that's (one of) my problem here, why would a parent class need to know about its childs, it seems like a poorly attribute responsibility, especially since this child can have other parents, and its behavior can't be fully known from A.

Class A shouldn't have to care what other parents its children have. (Which, I have to say, would make Thanksgiving dinner an extremely awkward time.) For example, here's a cut-down version of (more-or-less) how one of my brothers' projects uses inheritance: class Channel(Gtk.Frame): def __init__(self): ... bunch of GTK stuff, including setting up ... ... events that will call self.refract_value(x) ... def refract_value(self, value): self.update_position(value) self.update_target(value) def update_position(value): pass def update_target(value): pass Normally, a subclass of Channel just has to provide those last two functions, overriding the stubs. But if it needs to make a small adjustment somewhere else, it can. And it's perfectly reasonable to have a Channel subclass that has full functionality, and then subclass *that* class to make a small tweak to the behaviour, by overriding some specific method. The behaviour cannot be fully known from the Channel class, but you know what? It doesn't need to. All refract_value has to know is: there are methods called update_position and update_target, and I call those. The purpose of those methods is consistent, and their function signatures are consistent, so it doesn't need to care exactly which subclass is providing them. And if a mixin class overrides refract_value to add a third place to send the value, that's absolutely fine! It would look something like this: class ChannelLogger(Channel): def refract_value(self, value): self.log_value(value) super().refract_value(self, value) def log_value(value): ... And now, any class can inherit from (ChannelLogger, SomeOtherChannel) to augment the behaviour of an existing class. So where, in this hierarchy, is the "error" that needs to be reported? Which of these super() calls would, by your description, need to raise an error? It's far less obvious than you perhaps think, so please elaborate, please show exactly what constitutes an error.

Okay, i wasn't clear enough, my bad ``` class A: def call_me_in_A_first(self): # calls super def call_me_in_B_first(self): # calls super

class B: def call_me_in_A_first(self): # calls super def call_me_in_B_first(self): # calls super

class C(A,B): def call_me_in_A_first(self): # calls super def call_me_in_B_first(self): # calls super ```

Today, super locks you in the C A B order, even if that is not the order you want

If that's not the order you want, don't define "class C(A, B)". I honestly cannot imagine a situation in which you would create this scenario, but if you did, maybe one of A and B should be broken into two classes. It seems like you're trying to augment superclass behaviour in a very complicated way, but if you did something like "class C(B1, A, B2)", you could have some parts of what's currently in B happen before A, and other parts happen after A.

A solution in some cases would be to reorder the parent in the class definition of C This scenario here highlights a case when such a workaround is not enough Obviously you can still use the class.method syntax, but that's the problem : the current super feature doesn't work here

My alternative to super would, since you can just pass it an argument telling what parent it should target, it would look like that

``` class A: def call_me_in_A_first(self): # don't have to calls super def call_me_in_B_first(self): # don't have to calls super

class B: def call_me_in_A_first(self): # don't have to calls super def call_me_in_B_first(self): # don't have to calls super

class C(A,B): def call_me_in_A_first(self): __as_parent__(A).call_me_in_A_first() __as_parent__(B).call_me_in_A_first() def call_me_in_B_first(self): __as_parent__(B).call_me_in_B_first() __as_parent__(A).call_me_in_B_first() ```

What happens if I create these classes? class D(B): ... class E(C, D): ... What is your __as_parent__(A) and __as_parent__(B) behaviour here? The MRO for E would be (E, C, A, D, B, object). Presumably D is a modified version of B and should be augmenting its behaviour. How is __as_parent__ supposed to handle this?

...

You can't just inherit from arbitrary classes that don't work together. "Uncooperative multiple inheritance" is an unsolvable problem, and is best refactored using composition instead.

It isn't solved today, and that's the point of my alternative to MRO and my alternative to super. Although i'm not sure exactly how you define "uncooperative multiple inheritence" But this seems to be solvable by raising error on conflicting names, right?

I mean, it's not so different from a conflict when merging a git branch into another one, i'll explain that more in an answer down below, i'm trying to answer every one as much as i can in a timely manner

More broadly, I would say that, in any class hierarchy, the pinnacle class (Channel in my above example) needs to define the protocol, and every other class needs to follow that protocol. With that established, the MRO is your way to control which ones have priority, and you should never need to explicitly select a parent, since augmentation can happen at any point in the hierarchy. If you have a class that doesn't follow protocol, it is broken. Maybe more real-world-ish examples would help? All the toy examples that say "this needs to happen before that" are a bit too trivial to be clear, since it's highly unobvious what things would be provided where in a more realistic hierarchy. ChrisA

On Tue, 29 Mar 2022 at 07:39, malmiteria <martin.milon@ensc.fr> wrote:

Chris Angelico writes:

...

I'm not sure what you DO expect, though.

in this case : ``` class A: def method(self): pass

class B: def method(self): pass

class C(A,B): pass ```

it is unclear what C().method should resolve to. So instead of forcing a resolution, i'd prefer an error to be raised, stating that the resolution can't be done automatically. Hence the name ExplicitResolutionRequired.

This is what i would expect from calling C().method in this example. Or at least, this is how my solution would behave.

Please, can we get a non-toy example? I look at this example and go "of course it should resolve to A.method", but that's because, with these names, there is absolutely no clue as to your intention. The only thing I have to go on is the code itself, and to my mind, there are two equally plausible options: either the first one named takes precedence, or both methods should be called. Python chose one of those options. Other languages have chosen the other. But with nothing to go on but the code, all I can say is: the current behaviour looks perfectly fine to me.

...

Which of these super() calls would, by your description, need to raise an error?

None, actually. super should not raise the ExplicitResolutionRequired error, never. Accessing a method from a child class, which don't redefine it, when multiple parent have it, should raise the error.

Sure, but whatever. Whether it's the super call itself or the attribute lookup on the super object, which ones need to raise? And once again, you're just giving us toy examples with names that tell us nothing. Please, real examples. Show us how this is actually useful.

Updating C's code like that : ``` class C(A,B): def method(self): self.__as_parent__(A).method() # calls method defined in A self.__as_parent__(B).method() # calls method defined in B ```

now running ```C().method``` works fine.

The trouble is, if I now create these: class D(B): pass class E(C, D): pass then C's methods should be delegating to D and not to B. That's a fundamental problem for any sort of explicit lookup. Also, how is this different from simply writing: class C(A, B): def method(self): A.method(self) B.method(self) ? If you're going to explicitly call for a particular parent, why not just do that?

Note that with the use of super, you can't tell in the body of the C method how to combine A and B's methods. You have to rely on super order of visiting those methods. Meaning that in this specific example you'd need either to add a call to super in A, for it to call B's method, or a call to super(A, self) in C for it to call B's method, without having to change A's definition.

Yes. That's intentional. Usually, that is a deliberate and desired feature.

...

It's far less obvious than you perhaps think, so please elaborate, please show exactly what constitutes an error. I apologise if i wasn't clear, i hope it's better now. I'll do my best to explicit my idea more in the future too.

Unfortunately, no, it's not. With names like A, B, and C, we have to figure out our own purpose behind things, and I've spent so much time with Python's existing system that everything seems fine to me. You'll need to show exactly what doesn't work with the current system, instead of assuming that we understand the problem. Real example.

...

"class C(B1, A, B2)", you could have some parts of what's currently in B happen before A, and other parts happen after A.

You can't always break a class down, especially when you import those class from a library.

Again, real example please.

The fact that the current super + MRO make it the norm to simply chop off the behavior of class B's method parent call to replace it by C's method behavior is literraly insane to me.

In what world extending a parent method means replacing part of it with some of another parent?

Well....... generally, since Python doesn't allow you to call two functions at the same time, the *only* way to extend a parent method is to replace *all* of it. To replace part of it, you replace all of it and then call the other function, That's precisely what super().method() lets you do. So.... what you call "literally insane", I call "a perfectly normal feature". Hence, again, the need to see what you're actually trying to accomplish - not "replace the MRO", but what you're really trying to do with your code. Take a step back and show us your actual goals, because we cannot see the problems the way you see them. ChrisA

Stephen J. Turnbull writes:

Shouldn't A and B derive from Parenting? Or is it C that should?

Having Parenting anywhere in the inheritance tree would work with the current implementation i have for it. I intended it to be a root parent of any class without pre existing parent. --

How does the behavior of class C differ from class D? I don't think it does.

As a part of bigger inheritance trees, most scenarios would behave the same, but since mro is prone to dependency injection, having a call so super(A, self) and super(D, self) in the same method is dangerous. And especially, in this case your goal with making this use of super is to control tightly what parent method is called in what order. MRO as it stands prevents you from any certainty on that regard, whereas my solution is much more reliable on what method it targets. An idea that i can add on this example is that it seems unlikely you'll be using super without the intent of always resolving the same method. I mean, when else do you call a method but would think reasonable to expect to be delivered another one? Do you even do that when using super? i doubt it. A close enough scenario is when you intent to do dependencies injection. Which is often portrayed as a feature of the current MRO + super objects. But such a feature is not hard to introduce within my explicit method resolution + __as_parent__. (more on that later) And my __as_parent__, with or without this dependency injection feature added on the side will keep behaving like you'd expect to. Of course, denying the possibility of dependency injection is not something i'm arguing, i think it should be preserved, but as a side feature. To tell a bit more about how to allow dependency injection with my __as_parent__, the target argument could be matched against a remap dict, that could be set as extra argument, class attribute, or whatever, to allow for replacing a target class. The replacment can then inherits from the original class, and voila, you got your dependency injection. Without the need for MRO

wait, what?

We may need some more clarity as to what you are after. You need SOME method resolution order, and Python's MRO is pretty simple and straight froward.

You don't need a method resolution *order*, you just need method resolution. That's essentially the punchline. It makes sense that a child is more specific than its parents. It doesn't make sense that any parent would be more specific than any others (at the same depth at least). That's what my alternative does, i've coded it already, you can take a look at it : https://github.com/malmiteria/super-alternative-to-super I've tested it thoroughly, so you can also take a look at the tests to figure out how it behaves. But essentially, I do *not* order parents. I let conflict raise an error when they need to (and only when they need to). Then of course, solving the issue requires to define the method in the child, and eventually say which parent's method it should refer to, or in which order visit both, depending on cases. Since super relies on MRO, and i produced an alternative to MRO, i have to produce an alternative to super. And since my alternative to MRO doesn't order the inheritance tree, my alternative to super can't rely on that order. so essentially i can do it like that : ``` class A: def A_first(self): # calls super def B_first(self): # calls super class B: def A_first(self): # calls super def B_first(self): # calls super class C(A,B): def A_first(self): # calls super def B_first(self): # calls super ``` becomes ``` class A: def A_first(self): # doesn't call super def B_first(self): # doesn't call super class B: def A_first(self): # doesn't call super def B_first(self): # doesn't call super class C(A,B): def A_first(self): self.__as_parent__(A).A_first() self.__as_parent__(B).A_first() def B_first(self): self.__as_parent__(B).A_first() self.__as_parent__(A).A_first() ``` You'll notice that no parents need specific code to be used in multiple inheritance. Todays alternative with class.method, there are cases where you couldn't make it work like you want. Especially if on of the classes calls super and not that class.method alternative, it could lead to part of the methods being called in a loop. Now, why do i want this change? let's take a comparison: today's MRO feels like saying that git merge isn't needed, since we can order changes priority in the code. It makes sense to say one change should be applied if only one branch does it. It doesn't make sense to apply one of the changes first, by asusming we can order branches in any ways. There is a conflict, and no clear rules exists, resolution depends on context. Anyone telling you that you can just apply changes in order would seem crazy to you, and it would seem obvious to you that the current solution, which relies on letting the programmer solve the conflict manually when it araise is much safer. That's my point here. Conflicting names of method existing in both parents should really be considered a conflict, and require manual solving, as i illustrated it above. The alternative i produced (again, feel free to take a look at my repository) does not require so much work, in case of a conflict, and is behaving almost identically to current MRO + super in simple cases where implicitness is definitely enough. ------

But the author of C should certainly test that, and choose what order to put them in. It's possible they would want one method from A, and one from B, but then they could override those methods if they need to.

Yes, my solution does only raise error when needed, with the explicit purpose to give that information to the developper : They need to override the method in the child and choose the order. Current day MRO + super doesn't allow for you to decide this order. And doesn't raise any error, which can lead to unexpected behaviors. Essentially, my proposal is to just stop assuming we can solve this problem in a generic way. We can't, but we can identify it, and raise an error when it happens, to request the developper explicit solution. ----

I don’t think that’s even possible.

Even a tiny modicum of testing would take care of that.

Yeah, today it's not possible. super is bound to inheritance trees, class.method isn't which means you can accidentally get 'out of bound' of the inheritance tree when using class.method. My alternative keeps it bound to inheritance trees, and raises appropriate errors when you're getting out of bound. It's only to make life easier when writing code. It makes the programmers life easier, just a tiny modicum. But that's still an improvment. ---

Steven A mentioned the two seminal articles “super considered harmful” and “super considered helpful” — the thing is, they both say the same thing— in order for super() to work you have to follow certain rules. And then it works predictably. Whether that’s helpful or not depends on your use case.

I'm familiar with part of those seminal, but they seem to miss something. Ordering is not the solution to multiple inheritance method resolution. My alternative doesn't require you to follow certain rules, not as much as current super + MRO does, which makes the langage easier to learn and use.

I mean, overriding or augmenting is the solution in case of what i would call a conflict. Essentially, a conflict araise when you're trying to access a method of a class that don't define it, and has multiple parent defining it. Todays solution would resolve the child method to the most left of its parent capable of delivering it, which does not make sense to me. If you want a real life example, I can't really share that code, but a coworker of mine once tried to reorder classes one of our class inherited from. His exact wording when doing it was "i just changed it to match the order django seems to prefer, it shouldn't break anything". I'll let you take a guess at if he was right or not. Multiple inheritance is fairly common in django, so if you want real life examples, you should find some in this community. I don't think what i'm looking for is different from inheritence, i think it's more a variant of the current way of doing things. Feel free to take a look at the readme of my github repository where i dive in some depths into my reasoning here : https://github.com/malmiteria/super-alternative-to-super

On Sat, Mar 26, 2022 at 10:40:45AM -0700, Christopher Barker wrote:

Almost -- I think super() acts a proxy to the *perents*, plural.

Correct. In general, classes do not have a parent. They have one or more parents, each of which in turn can have one or more parents. Anyone talking about the "parent" or "base class" of a generic, unspecified Python class is stuck in a single inheritance world. Years ago, James Knight wrote a provocatively titled blog post called "super considered harmful". https://fuhm.net/super-harmful/ After much discussion on one of the Python lists, I think it was Python-Dev, he backed down somewhat and admitted that actually yes, super is great, "but you can't use it." Except of course thousands of people do use it. And it works.

If you just want the superclass, you can call it directly:

class B(A): def some_method(self, ...): A.some_method(self)

Please please please don't do that! That just needlessly breaks multiple inheritance, and for no good purpose. Just use super(), even for single inheritance. Anyone who hasn't read Raymond Hettinger's "super considered super", please do. https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

The point of super() is to provide a way to, yes, implicitly, call a method on all the superclasses in the inheritance tree.

No, it is not implicit. Please don't repeat that mistake. `super().some_method(self)` is perfectly explicit. It is just that the specific class used is computed at runtime, not hard-coded. We wouldn't say that get_parent().some_method(self) is "implicit" because the parent is computed at runtime. We should stop saying that super() is "implicit". It is not. It is an explicit call to resolve the mro and call some_method on the appropriate parent. The only "implicit" part of super is that it implicitly knows the current context. But that's not a point against it. Nobody complains that functions "implicitly" know their global environment, and therefore name resolution in functions is broken due to implicitness: # thismodule.py def func(): return "something" def main(): return func() # Oh no, it is "implicit"! # should write: return thismodule.func() # Much more explicit!!! Despite what the Zen says, sometimes a modicum of implicit knowledge is worth an imperial ton of explicit verbosity. And the zero-argument form of super() is one of those times.

There's more than one way to do that, but it's perfectly reasonable that Python provide only one way, with clear rules, which is what we have now. And if there's going to be one way, the MRO currently in use is a pretty darn good one.

The C3 linearization used by Python is provably correct. I am not aware of any other non-broken linearization algorithms for multiple inheritance. Perhaps they exist; perhaps not. Before we had C3, the ways multiple inheritance was resolved in Python 1.x through 2.2 was broken. Note carefully that we had **two** ways of resolving multiple inheritance, for classic classes and new-style classes, and both were broken in different ways, until we moved to a proven algorithm in Python 2.3. https://www.python.org/download/releases/2.3/mro/ -- Steve

On Sun, Mar 27, 2022 at 01:33:05PM -0000, malmiteria wrote:

C3 linéarization might be mathematically proven to be optimal, as in, it's the best way to order a graph. Sure. But why would we order the inheritance tree?

How else are you going to get inheritance without a linear order? The interpreter can only call superclass methods one at a time, in some linear order. You might have a tree that looks like this in part: # class C(A, B): ... A B \ / \ / C but C.method() cannot call A.method() and B.method() *simultaneously*, it must call one followed by the other in *some order*. The whole point of inheritance is that (to the degree that it is possible) we should not explicitly care about where the methods are defined, only that they are defined *somewhere* in the MRO. If you do want to explicitly specify where the methods are defined, you are not using inheritance. You're just calling a function. # No need to inherit from B, if you don't inherit from B. class C(A): def method(self): # Manually call B.method. result = B.method(self) # Assumes that C duck-types as B. do_something_with(result) So long as your C instances duck-type as B, and B does not insist on actual subtyping (by checking self with isinstance(), but who does that?), then you don't need to inherit from B to call B methods. If you want to manage your "inheritance" manually by specifying the order, then just don't use automatic inheritance, and manually specify the order by calling functions in whatever order you want, when you want, where you want. -- Steve

Steven D'Aprano writes:

How else are you going to get inheritance without a linear order? The interpreter can only call superclass methods one at a time, in some linear order.

The whole point of inheritance is that (to the degree that it is

You can decide what method should be the one resolved to *after* visiting all superclass methods. I've already implemented it here, it's a "simple" recursion (no recursion is ever simple xD): https://github.com/malmiteria/super-alternative-to-super/blob/59ff90029db6e4... Essentially, as long as parent don't have it, you keep looking higher in the inheritance tree, either you find no such method, and you raise an AttributeError, if you found it only once, you return it, and if you found multiple, you raise the ExplicitResolutionRequired error. The order in which you visited the parent is irrelevant to the result, what matters is only to stop exploring a branch when it ends, or when you found the method. Each parent is looked up in its own branch, independantly of any other branches. The end result is absolutely not affected by the order in which you've explored those branches -- possible) we should not explicitly care about where the methods are defined Agreed. My solution doesn't require you to be explicit about where a method is defined. It only eventually raises an error in case of collision, which you can resolve by redefining the method in the child class, only if you intend on calling it (not calling a method that would raise an error is fine with my solution, whereas current MRO + super fails at class definition time no matter your use of that class). my __as_parent__ allows to explicitely call each one of the parents method individually wherever it matters to you --

If you do want to explicitly specify where the methods are defined

I don't, at least that's not what my solution is for / requires you to do. --

If you want to manage your "inheritance" manually by specifying the order, then just don't use automatic inheritance

That's what i have to resolve to today yeah. The feature has its limits, working around it is painful, that's why i'm proposing this change to the language

On Tue, Mar 29, 2022 at 06:23:06AM +1100, Chris Angelico wrote:

If I'm reading this correctly, you're defining a "conflict" as finding two implementations of the same-named method in independent subtrees of inheritance - that is, when you say "class Pizza(Crust, Topping):", you're treating Crust and Topping as completely independent, and if they both define an add_cheese method, that's a conflict.

You've never had pizza with cheesy crust *and* cheese as a topping?

That's not inheritance. That's composition.

I wouldn't call it composition unless it was implemented using composition instead of inheritance. I would call it a *subset* of inheritance. Artima is back up, so people interested in this topic **REALLY** should read what Michele Simionato has to say about it. I'm not going to apologise for dumping so many links here, I think that Michele is really onto something: OOP with inheritance is an important technique to use, but it is also widely over-sold as the solution to maintainability in large software projects and frameworks, especially when you move from single to multiple inheritance. Unfortunately, MI is just as likely to *hurt* maintainability as to help it, hence the move to composition and generics, or more restrictive forms of inheritance such as traits https://pypi.org/project/strait/ Things to know about Python super: Part 1 https://www.artima.com/weblogs/viewpost.jsp?thread=236275 Part 2 https://www.artima.com/weblogs/viewpost.jsp?thread=236278 Part 3 https://www.artima.com/weblogs/viewpost.jsp?thread=237121 Mixins considered harmful: Part 1 https://www.artima.com/weblogs/viewpost.jsp?thread=246341 Part 2 https://www.artima.com/weblogs/viewpost.jsp?thread=246483 Part 3 https://www.artima.com/weblogs/viewpost.jsp?thread=254367 Part 4 https://www.artima.com/weblogs/viewpost.jsp?thread=254507 Setting Multiple Inheritance Straight: https://www.artima.com/weblogs/viewpost.jsp?thread=246488 The wonders of cooperative inheritance, or using super in Python 3 https://www.artima.com/weblogs/viewpost.jsp?thread=281127 Multiple inheritance in Python works the way it does because it is modelling cooperative MI and the MRO and super are the Right Way to handle cooperative MI. That doesn't mean that cooperative MI doesn't have problems. Other languages forbid MI altogether because of those problems, or only allow it in a severely restricted version, or use a more primitive form of super. MI as defined by Python is perhaps the most powerful, but also the most potentially complicated, complex, convoluted and confusing. -- Steve

On Sat, Mar 26, 2022 at 10:50 AM malmiteria <martin.milon@ensc.fr> wrote:

why does it have to be automatic?

because that's the entire point of super() -- if you don't want automatic, do what you want by making direct method calls.

if they think one is more specialized than the other, they make one inherit from the other, they don't set them both as parent of a third.

I think the best use case for multiple inheritance is mixins -- which do have a natural order -- as you say, in other cases, you can do single inheritance.

Tho, to make it clear, i've no doubt it's doable without super today. My point is that MRO as it is, has some flaws.

Also, super() actually calls the method on all the superclasses (but not the same one twice) -- so that right to left thing doesn't matter. It does matter with regular calling of methods, though, but you need SOME rule there. -CHB

Also, Eventual hierarchy where the rule won't fit are simply not allowed today, no matter your use of super. take this one for example : ``` class A(X, Y): pass class B(Y, X): pass class C(A, B): pass ```

This code fails, you can overcome it through composition, but nevertheless, this doesn't justify this problem. Essentially it's a case of "this is broken, but something else close enough isn't". It works, but it's still broken

Oh and just to make it more explicit, MRO and super are two different concern of mine here. They could be addressed separately, but since those feature are related, i felt like sharing it in one post _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NH5JOZ... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On 26 Mar 2022, at 20:19, Christopher Barker <pythonchb@gmail.com> wrote:

Also, super() actually calls the method on all the superclasses (but not the same one twice) -- so that right to left thing doesn't matter.

Super() calls the method on only one class. Not all. It finds that one class by using the mro to find the one class. Barry

Because that is an inconsistent hierarchy, which makes the code broken, and leads to bugs.

Bugs come from the MRO and previous solutions to it. Which i'd argue come from the obsession on that idea that ordering is the way to go. it is not. A solution that can resolve method in what you call inconsistent hierarchy would make those hierarchy not inconsistent anymore. I think that's a non argument, or maybe it shows that there's no proper full solution to multiple inheritance to this day. But think about it, would you call that inheritance tree inconsistent if somehow there was no resolution issue? Of course not. It is only inconsistent within the current MRO system. I'm aware of some of the evolution of those MRO algorithm. I know that previous iteration were broken in some weird cases. You'll notice that my solution behave exactly like the current solution in case simple inheritance, and that i've tested it thoroughly in what i assume is all the possible cases, given the set of rules i decided for my Explicit Method Resolution. Feel free to take a look at it here : https://github.com/malmiteria/super-alternative-to-super and let me know if there's any of those weird scenarios you can think of that i missed

class decorator aren't propagated through inheritance, so i just made it with a good ol' parent class. again, feel free to take a look at my implementation of it here: https://github.com/malmiteria/super-alternative-to-super It's thorougly tested, and hopefully a good showcase of what it does. But yeah, i think i agree with everything you say, the child needs to implement its version of the method for it to work properly, and the inheritance is a tight coulpling. My proposal would just make it more explicit from the developper perspective, as in, if you're trying to access a method you didn't override in the child class, and on top of that, both parent can resolve it, you get an error. I assume in other cases, the resolution could still be implicit. I think the set of rules i propose for my version of method resolution covers all cases, by simply adding this error case to what's the current external behavior of MRO Because you're not supposed to let that hanging like that. And current MRO would not raise any error, and be completely silent on that. which is an issue.

ROUND 2 @everyone this is... a lenghty post to say the least. That should give you the 'full' picture. It's quite a lot of things to talk about, so i'm asking you to please explicitely state what proposal you are adressing when responding to this email. If possible with a "CONTEXT : ..." line somewhere, so i can easily find it. If it is unrelated to any proposal, feel free to be explicit about it in this CONTEXT: ... line I apologise for eventual typos, i didn't wanna let you wait too long, and this took me way too long to write that already. I'm copy pasting it down below, but it is also accessible here : https://github.com/malmiteria/super-alternative-to-super/tree/master/convers... A quick TLDR: there's 5 proposal, i've tried to make them independant of one another, but they would have to come in order (as far as i can tell) 1) a dedicated module for dependency injection and more called alterhitance 2) a way to declare inheritance after defining the class (postponed inheritance) 3) a solution to the diamond problem 4) changes to the proxy feature of super (that i already mentionned earlier in this thread) 5) changes to the method resolution algorithm (that i already mentionned earlier in this thread) ========= The current state of MRO + super is being used in multiple ways. No explicit feature is isolated for any of those use cases. Let's list those feature. I'll dive in more depth about illustrating them, and my thoughts about them later. feature 1 (of super alone): proxying the parent. What most people think super does (and expect it to do): it allows to call method from *the* (most people don't think of multiple inheritance) parent. It can be used by working around MRO to proxy any parent directly. I'd argue this isn't reliable, which makes it an incomplete feature, but that's still a use made of super If i had to specify only one problem i have with super today : it's use in simple inheritance hints at a behavior of super, which turns out to be widely different from its actual behavior. a feature more consistent on that regard would make the langage much easier to learn, for everyone. feature 2 (of super + MRO): sideway specialization / mixins It however can jump sideways in the inheritance tree. allowing for same level class to still specialize one another. For this scenario, today, we rely on the order in which class are inherited. This is usefull when for some reason it is not possible to make a class inherit from another at definition time. feature 3 (of super + MRO): dependency injection / inheritance tree alteration If you wanna inject a class wherever in some pre existing class inheritance tree, you can through the use of some clever new classes. If you wanna deny its parent to one class, and essentially prune of branch off the inheritance tree, you can too. feature 4 (of MRO alone): method resolution when trying to resolve a method / attribute of a class, if not present in its definition, looks up the parent(s) to come up with an answer. If no parent can resolve it, you get the attribute error, if at least one does, you get it. I'd argue that it makes it possible to silently ignore a parent method when multiple parent define such a method. Which makes it especially painfull to debug those scenarios, which makes it an incomplete feature. On top of that, it renders some class definition invalid, when really, the method resolution shouldn't break more than the method resolution. This should be a gettribute error, not a class definition error. "feature" 5 : diamond problem. Not really a feature, more a lack of feature. Today's diamond problem is solved by making the top layer appear only once in scpecialisation, which is a direct consequence of ordering the inheritance tree. Whatever changes we decide on the 4 previous features, the impact on the diamond problem will either be null, or make the top layer be specialsed by all it's chilren. Either strategies aren't covering all needs. This requires an extra dedicated feature. Also, as mentionned, i'd argue that some of those feature aren't fully delivering what they promise. Essentially, they compromise for the extra feature, which in the context of a lack of properly isolated implementation for those feature seems fair. # Why am i here? What motivated me first here was the few problems with the feature 1 (proxying the parent) and feature 4 (method resolution) which I consider to be suboptimal. Of course, the different use cases made out of the current super + MRO should be preserved. Essentially, I do *not* intend to lose any strength / any use case of the langage Listing all the use cases made out of current super + MRO matters then. And I think i already cover the most important ones with the 4 features i described. If you feel I'm missing something, please let me know. ## My goal : essentially, producing features for parent proxying and method resolution that wouldn't exhibit the flaws of their current version is my goal. I also (of course) wanna preserve all current use case made out of the current version. Since MRO is the base building block on which super relies, and MRO + super are the bases for those extra feature, it follows that I need to provide alternative for those feature. Those changes would be massive, and require a lot of effort / time to make their ways into python, so it also matters to take this into consideration, at some point. As much as possible, I don't wanna change the external behavior / API of the code. If my alternative looks the same, it should behave the same. If it behaves the same, it should look the same. Some amount of changes are needed of course, but my goal is to make it an easy transition, as much as possible. Breaking changes? feature 1 (proxying the parents) Any alternative that don't rely on MRO is bound to make the other features break, so I guess there's no way out this being a breaking change, if not for spreading alternative not reliant on MRO for those other extra features. But it would be possible to have a non breaking change that introduce the alternatives feature 2 and 3, followed after transition of code to those feature by another non breaking change of this feature 1, which would help the migration. However, unmigrated features 2 and 3 are bound to break. feature 2 (sideway specialization) Introducing an alternative without removing the old ways of doing things won't be a breaking change. This could be published in a first batch, with time to let every new commers learn to use it, and old timers switch to it. It would obviously need to be robust enough to suit everyones need, but that's only a matter of designing the feature. Removing the current version of this feature would come from removing / drasitcally change MRO + super current behavior, and can't really be done without such a change, so I'd say that feature 2 new alternative can be done without breaking change. Feature 1 breaking change would also break the current version of feature 2 feature 3 (dependency injection / inheritance tree alteration) : Same as for feature 2. feature 4 (method resolution) Any alternative to it is bound to breaking changes, since MRO is so tightly coupled with feature 1 2 3. Those features alternative should come first. If possible, let's limit it to one breaking change total :p Once the alternatives to feature 2 and 3 are here, an alternative to feature 1 can be produced, without removing the current version of feature 1 2 and 3. unlike for feature 1 2 and 3, I don't think it's possible to produce an alternative to feature 4 without replacing it. I might be wrong tho, at least for part of the feature. Essentially : produce alternatives to feature 1 2 and 3 (most urgently 2 and 3) without removing the current version of those feature, then the alternative to feature 4 can be done, which would be the *only* breaking change # code examples: ## diamond tree, repeat top CONTEXT : This exemple makes use of today's features. No replacement feature is used there The goal is to showcase today's behavior, and add my remarks. ``` class HighGobelin: def scream(self): print("raAaaaAar") class CorruptedGobelin(HighGobelin): def scream(self): print("my corrupted soul makes me wanna scream") super().scream() class ProudGobelin(HighGobelin): def scream(self): print("I ... can't ... contain my scream!") super().scream() class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): # 50% chance to call ProudGobelin scream, 50% chance to call CorruptedGobelin scream ``` when writing down the class HalfBreed, we expect the behavior of HalfBreed().scream() to be one of its parent behavior, selected at random with 50% chance each. with the use of super, it could look like that, intuitively ``` class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): super(HalfBreed, self).scream() else: super(ProudGobelin, self).scream() ``` However, super(HalfBreed, self) does not deliver ProudGoeblin behavior, but a mix of ProudGobelin, CorrupteGobelin, and HighGobelinBehavior. We would expect the output to be : "I ... can't ... contain my scream!" "raAaaaAar" But it is : "I ... can't ... contain my scream!" "my corrupted soul makes me wanna scream" "raAaaaAar" Getting the correct behavior requires to let go of super, in such a way: ``` class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): ProudGobelin.scream(self) else: CorrupteGobelin.scream(self) ``` Which is an option multiple of you pointed out to me. This options however is flawed in muliple ways, mainly because it looses 3 of the 4 features i describe earlier. - we lose the proxy feature, which to be fair, is aqueen to syntactic sugar. But this is still a feature loss. - we lose the sideway specialisation, coming from super's behavior, which, in this case, is the goal, so we're not complaining. - we lose the possibility of class dependency injection, since today, it relies on a consistent use of super. As some of you mentionned (with pain and agony, for some of you ;) ), loss of feature is a big problem. I think at least for this scenario, assuming that we indeed managed to produce an alternative to all those feature, we are gaining the feature you are defending, not losing it. ## Can't assume on parent's more specialised. CONTEXT : This exemple makes use of today's features. No replacement feature is used there The goal is to showcase today's behavior, and add my remarks. ``` class Glider: def push(self) print("I can't move so good on the ground") def jump(self): print("I'm free! I can fly now!") class Wheel: def push(self) print("let's roll !") def jump(self): print("oh damn, i'm falling fast!") class WheelGlider(Wheel, Glider): def push(self): # calls Wheel push method first # calls Glider push method second def jump(self): # calls Glider push method first # calls Wheel push method second ``` In this example, we expect WheelGlider push method to output : "let's roll !" "I can't move so good on the ground" and we expect WheelGlider jump method to output : "I'm free! I can fly now!" "oh damn, i'm falling fast!" this could be achieved in such a way: ``` class WheelGlider(Wheel, Glider): def push(self): super().push() super(Wheel, self).push() def jump(self): super(Wheel, self).jump() super().jump() ``` This raises a lot of questions - how do those jump and push method behave in case Wheel and Glider are refactored to inherit from a parent? - If this refactoring introduces the diamond case, we're back at the problem showcased in the diamond tree example - If no diamond case (no parent is shared by both, no matter how deep in the inheritance tree), then it would behave consistently with what's expected now. - How do those jump and push method behave in case WheelGlider is used as a parent of another class, which might or might not inherit from Wheel or Glider? - The API here is suboptimal, to say the least, since we have to pass arguments to super that will then be processed for us to reach our target. super(Wheel, self) proxies Glider, which is far from being obvious. - How does one uses all the features of super + MRO here? I don't see a case where you'd want sideway specialisation in this scenario, but it doesn't mean it wouldn't happen. But what about dependency injection / inheritance tree alteration? Let's investigate : Let's say you wanna make use of dependency injection to mock WheelGlider parents in a unit test setting. The way to reach this goal today is simple, make use of super "weird" behavior in diamond scenario cases to inject your class in the middle. ``` class MockedWheel(Wheel): def push(self): print("mocked wheel push") def jump(self): print("mocked wheel jump") class MockedGlider(Glider): def push(self): print("mocked glider push") def jump(self): print("mocked glider jump") class MockedWheelGlider(WheelGlider, MockedWheel, MockedGlider): d def push(self): print("mocked WG push") super().push() def jump(self): print("mocked WG jump") super().jump() ``` To get some bird eye view of this inheritance tree, we know MRO will order it like that: (1) MockedWheelGlider < WheelGlider < MockedWheel < MockedGlider (2) WheelGlider < Wheel < Glider (3) MockedWheel < Wheel (4) MockedGlider < Glider after a quick sandbox test, the complete mro is: MockedWheelGlider < WheelGlider < MockedWheel < Wheel < MockedGlider < Glider (< object) This happens to work fine: ``` MockedWheelGlider.jump() # prints "mocked WG jump", then "mocked glider jump", then "mocked wheel jump" MockedWheelGlider.push() # prints "mocked WG push", then "mocked wheel push", then "mocked glider push" ``` If one of the super were replaced by the class.method approach, the MockedWheelGlider would fail to fully mock Wheel and Glider. But as far as i'm concerned, this feature is quite robust in this scenario (when super is consistently used). For deeper inheritance tree alteration, such as removing a branch (can be done by messing with __bases__), they could break some super calls (which is to be expected anyways) Other than that, I would argue that for the sake of symetry, it would probably be preferable for super to allow / disallow the exact same API for the use of super, no matter the parent it targets. Since in this scenario, we explicitely *don't* think of one parent as coming after the other (in term of specialisation, that is), it shows a disconnect between the feature and its use. Which i take as a hint that there is a feature to produce to match this need more accurately. ## Way too big combinatory possibilities CONTEXT : This exemple makes use of today's features. No replacement feature is used there The goal is to showcase today's behavior, and add my remarks. let's say you're making a web framework. You might wanna provide a few View classes: ``` class View: def render(self): # does some generic view stuff print("view") class ListView(View): def render(self): # does some list view specific stuff print("list view") super().render() class FormView(View): def render(self): # does some form view specific stuff print("list view") super().render() ``` That's just an example, but you might get more of those View classes depending on the scenarios you wanna cover You might also end up providing more features, that relate to the View, such as a LoginRequiredMixin, and a PermissionMixin ``` class LoginRequiredMixin: def render(self): # does login specific stuff super().render() # notice the call to super, despite the class not having parents class PermissionMixin: def render(self): # does permission specific stuff super().render() # notice the call to super, despite the class not having parents ``` Again, those are just examples, you might end up needing more of those mixins. Now, each View class would benefit from a variant with the permission mixin's behavior, a variant with the login required mixin, and a variant with both. Add more mixins, and the combinatory explodes. And you'd want to produce all those classes variant for each View classes. The "proper" way to produce those variant would be through inheritance. After all, those variant are simply "more specialised" variants of the base class. In order to do it, you would have to produce a LoginRequiredView which inherit from View, a PermissionView inheriting from View, and a LoginRequiredPermissionView which would maybe inherit from PermissionView? But the problem would then be, how do you not repeat yourself a million time, writting down the login specialisation code in each variant that requires it? You can't really, if you wanna stick with inheritance at this stage. However, the use of mixin, such as describe in the code blocks allows a DRY approach, if used correctly by whoever integrates those classes later on. For an end user, such an integration could look like: ``` class MyView(LoginRequiredMixin, PermissionMixin, View): # would call render in some appropriate method, but not necesserly redefine it. ``` which would provide the expected behavior, with no need for the framework editor to produce all the possible combination, and allows for a DRY approach. Essentially tho, this is a use case of multiple inheritance to provide the behavior of what really could have been simple inheritance. The inheritance tree in this scenario is: Login Permission View \ | / MyView When it fact it was meant to be View | Permission | Login | MyView The practical reason is valid however, it is to me a symptom of a missing feature. Turns out it is not possible today (or at least, not properly integrated in the langage) for a class to be defined without knowing its parent, and later on to be attributed parents. It is also not possible to define deep layers of it's inheritance tree when defining it. I'll be making a proposal for that later on. # Let's talk about the features ## Proxying the parent CONTEXT : I'm talking here *exclusively* about the parent proxying feature. I will talk about the other features on their dedicated sections. I'm describing the current state of this feature, to the best of my understanding. I'll also add my remarks on what we would wanna keep / get rid of for an optimal feature. But i am not at this point proposing an alternative. This feature allows to proxy a class, which render it's method (and attributes to some extent) accessible from the parent to the child, "as" the parent. ``` class Dad: def say_hi(self): print("Good morning") class Child(Dad): def say_hi(self): print("....") super().say_hi("Good morning") ``` This is a way to access the parent method as if through an instance of it. It is made to be used inside class method definitions, more than anything else, but can be used to define child attribute based on parent attributes. ``` class Dad: age = 100 class Son(Dad): # super().age fails with a RuntimeError (no argument) # super(Son, Son) fails here with a NameError (Son is not defined yet) pass Son.age = super(Son, Son).age - 30 ``` The 'syntactic sugar' aspect of super's proxying feature is that calls to parent methods through super don't require to pass the instance as first argument, which makes for less redundant code. This should be kept in the alternative feature. There is a 'disconnect' between the argument passed to super (when needed) and the class it will target to be a proxy of. super(Son) is not a proxy of class Son, it is a proxy on the next in MRO (it feels redundant saying MRO order, but weird just saying in MRO ...) So in cases where the automatic solve of the argumentless super doesn't suits our needs, such as in the exemples above, there is some mental gymnastic, and knowledge to be known to reach our goal. This should be simplified in the alternative feature. The possiblity of calling super with no argument is quite handy, and should probably be kept too. In case of multiple inheritence, the argumentless expression of super does lack explicitness to me. I would wanna make the use of the alternative more explicit, in this scenario. TLDR: - keep : - syntactic sugar - argumentless option in case of simple inheritance - don't keep: - argument and proxy target being not the same - argumentless syntax in multiple inheritance ## Sideway specialisation / Mixins CONTEXT : I'm talking here *exclusively* about the sideway specialisation / mixin feature. I will talk about the other features on their dedicated sections. I'm describing the current state of this feature, to the best of my understanding. I'll also add my remarks on what we would wanna keep / get rid of for an optimal feature. But i am not at this point proposing an alternative. Today, it is possible to have a class designed to inherite from another, but without having it explicitely inherit from it, when defined. This is useful in case the specialistion class should be applied to multiple base class, especially when more specialisation classes exists, and all combination of the specialisations should be provided. This is showcased in the "way too many combinatory" exemple i gave earlier. If you have those base classes : ``` class View1: def render(self): # ... ... class View10: def render(self): # ... ``` And those specilaser mixins: ``` class Spec1: def render(self): # ... super().render() # ... ... class Spec10: def render(self): # ... super().render() # ... ``` The amount of combination is insanely huge, something like, if i'm not mistaken: for n in range(amount of spec): (amount of views) * [(amount of specs)!/(n!)] which in this case gives 62353000 Anyone attempting to manually do that would be insane. Obviouly, it's rare to have 10 specs, but the number is still high with 3 specs : 90 More accessible, but definitely not a fun task How is it done then? Through multiple inheritance. ``` class MyView(Spec1, Spec4, View3): pass ``` is a class that inherits from what would have been the combination Spec1 inherits from Spec4 inherits from View3, would it have been possible to define such inheritance between the specs and view. Since MRO in this case is MyView < Spec1 < Spec4 < View3, any call to super from Spec1 will proxy Spec4, and any call to super from Spec4 will proxy View3. Today's super allows to provide all those combinations, without having to explicitely define any combination This feature is a must keep I would argue tho, that today's solution produces a multiple inheritance tree, when all it need in fact is a simple inheritance tree. It essentially "forget" about the inheritance it meant to provide, but falls back into working thanks to the ability of super to side jump. An alternative would probably benefit from rendering the inheritance feature explicit. Today, those scenarios are simply denying the inheritance they really meant to benefit from. Let's make them actual inheritance. I would also argue that multiple the inheritance syntax is definitely not appropriate, as it is very not clearly keeping track of the order of classes the child inherits from. I've seen people try to reorder the classes in such a scenario, because it was weird to them to have mixins come after the view class. Which is the most important one, so they felt it made sense to have it placed first. Of course our automated tests made it clear it wasn't working, but it also means that had we not have tests, testing the mixin features, we would have silently lost this feature. And testing those feature was debated in our team, since those are essentially the web framework responsibility, not ours. I can see this bug reaching production for this reason. As much as this is an UX consideration, this is definitely a must get rid of. An alternative should be designed to make it clear there is an order. TLDR: - keep: - Not having to explicitely define all combinations of (n) mixins + feature class - don't keep: - Loss of simple inheritance when it's the dedicated feature for this need - Current UX for this scenario lack of explicit order ## Dependency injection / inheritance tree alteration CONTEXT : I'm talking here *exclusively* about the dependency injection / inheritance tree alteration feature. I will talk about the other features on their dedicated sections. I'm describing the current state of this feature, to the best of my understanding. I'll also add my remarks on what we would wanna keep / get rid of for an optimal feature. But i am not at this point proposing an alternative. Today's MRO + super allows for dependency injection. Let's take the "Can't assume on parent's more specialised." example: ``` class Glider: def push(self) print("I can't move so good on the ground") def jump(self): print("I'm free! I can fly now!") class Wheel: def push(self) print("let's roll !") def jump(self): print("oh damn, i'm falling fast!") class WheelGlider(Wheel, Glider): def push(self): super().push() super(Wheel, self).push() def jump(self): super(Wheel, self).jump() super().jump() ``` MRO for WheelGlider here is : WheelGlider < Wheel < Glider This order will never change, however, it is possible to squeeze between any two consecutively ordered classes by making use of a good understanding of MRO: MockedWheelGlider < WheelGlider < MockedWheel < Wheel < MockedGlider < Glider implies: WheelGlider < Wheel < Glider This new order can be obtained with: ``` class MockedWheel(Wheel): def push(self): print("mocked wheel push") def jump(self): print("mocked wheel jump") class MockedGlider(Glider): def push(self): print("mocked glider push") def jump(self): print("mocked glider jump") class MockedWheelGlider(WheelGlider, MockedWheel, MockedGlider): d def push(self): print("mocked WG push") super().push() def jump(self): print("mocked WG jump") super().jump() ``` This features essentially allows to reparent any class, anywhere in any inheritance tree, this is definitely a must keep. I would argue that, as much as this feature is very reliable today, it relies on super and MRO which don't always cover developpers needs. Today's alternative class.method does most definitely not allow for proper dependency injection, as super calls implictely target the next in MRO, and class.method are hardcoded to target the same method no matter MRO, so the dependency injection dependance on super and MRO is a downfall of this feature. This feature also requires to create a new class, inheriting from the class to inject dependency on, meaning it is not an inplace change.This should probably be kept like that. Although, i guess it doesn't hurt to give the option to the developper to have the change inplace or not. The current state of this feature UX is quite poor tho, as the amount of knowledge you have to pour into the code to make it work like you'd expect is quite high. Making this feature not accessible to most developpers. The alternative would most definitely benefit from a simpler API. On the topic of the link between dependency injection and super + MRO, it is important to note that any slight change on MRO + super could have an impact on this feature, essentially locking super and MRO in their current state. TLDR: - keep: - the ability to reparent any class anywhere in any inheritance tree. - the not inplace change. - don't keep: - the reliance on super + MRO - the amount of knowledge needed to make use of this feature ## Method resolution CONTEXT : I'm talking here *exclusively* about the method resolution feature. I will talk about the other features on their dedicated sections. I'm describing the current state of this feature, to the best of my understanding. I'll also add my remarks on what we would wanna keep / get rid of for an optimal feature. But i am not at this point proposing an alternative. Method resolution is the feature that allow child class to access parent method as if it was their own, assuming they do not override it with a method with the same name. ``` class Dad: def joke(self): print("I'm afraid for the calendar. Its days are numbered.") class Son(Dad): pass ``` In this exemple ```Son().joke()``` will resolve to joke method defined in Dad However: ``` class Dad: def joke(self): print("I'm afraid for the calendar. Its days are numbered.") class Son(Dad): def joke(self): print("*UNO reverse card sound* Hi dad, I'm son.") ``` In this exemple ```Son().joke()``` will resolve to the joke method defined in Son Had Dad not have a joke method, an AttributeError would have been raised In case of multiple inheritance, we still want the child to be able to inherit its parent methods. ``` class Dad: def joke(self): print("I'm afraid for the calendar. Its days are numbered.") class Mom: def joke(self): print("What do you call a small mom? Minimum.") class Son(Dad, Mom): pass ``` In this exemple the use of a method resolution *order* comes into play. Since multiple parent have a joke method to provide, and Son method resolution can only resolve to one of them, it is resolved to the first in order capable of delivering the method. Here, ```Son().joke()``` resolves to the joke method of Dad. Had Mom not have a joke method, nothing would change. Had Dad not have a joke method, ```Son().joke()``` would have resolved to the joke method of Mom. Had both Mom and Dad not have a joke method, an AttributeError would have been raised Method resolution is also applied to class attributes. Essentially, a method bound class can be considered a callable attribute of this class, there's no specification of non callable attributes that would make them not need a method resolution algorithm. I would argue that it is less than optimal that one parent method can be silently ignored by the method resolution on the pretext that another parent had a method with the same name "before". In some case, as the mixin case, such an order matches the need, but in the generic case, it is too speicifc of an assumption. The ability for the child to resolve its parent method should be kept in, obviously (that's what inheritance is in the end) However in case of multiple inheritance, and maybe more specifically, multiple resolution, a "manual" merge in the child class definition should be requested. This goes for method and attributes, as of today, it is assumed one method is prefered, but other strategies might be the more viable ones. This assumption is something i disagree with. We should allow for multiple strategy, instead of enforcing one, and request a strategy when the default one can't provide a result. The ability for the child to resolve automatically one of its parent method when only one parent can provide it is a viable default method, and should be kept as the default one, in this scenario. Today's MRO doesn't allow for all possible class inheritance trees, as some can't be ordered, or would have inconsistent order. such as : ``` class A: pass class B(A): pass # B < A class C(A,B): pass # A > B ``` TLDR: - keep: - resolution on class body method first - resolution implicit on parent method (if not present in own class body) when only one parent has it - don't keep - implicit resolution when multiple parent can resolve it - assumption of an order of parents. One might be more specialised than the other, but does not have to. ## Diamond problem The diamond problem is essentially the question, should methods from a class which appears multiple time in an inheritance tree be called multiple time, or only once. Essentially, multiple answers are possible, depending on your specific needs. You might wanna want the bottom class inherit from the full behavior of all of it's parent classes, in which case, you'd call the grandparent method each time. You might also want the grandparent class method to be called only once, after all other specialisation provided by all the parents. Today, the only option provided by super + MRO is or the grandparent to be called only once. I'd argue this is not enough. We should be able to chose the strategy ourself, depending on our needs. The diamond problem illustrate the problem when a class appears multiple time in an inheritance tree, but this shape doesn't have to be diamond. For example: ``` class Top: pass class Side(Top): pass class Bottom(Side, Top): pass ``` TLDR: allow for multiple inheritance strategies, instead of enforcing one. ## the weird case of __slots__ Someone mentionned __slots__ I didn't consider it at first, and now i'm weirded out by it. (I'm running those exemples in python 3.6, according to more probing on my part, __slots__ behaves differently on python 2.7) ``` class A: __slots__ = ['a'] class B: __slots__ = ['b'] class C(B,A): pass ``` This code raises an error: TypeError: multiple bases have instance lay-out conflict Turns out, C doesn't like that A and B both define __slots__ So, my intuition was that redefining __slots__ in C would fix the issue ``` class A: __slots__ = ['a'] class B: __slots__ = ['b'] class C(B,A): __slots__ = ['c'] ``` This raises the exact same error. Multiple inheritance works for __slots__, at the strict condition that only one parent defines __slots__: ``` class A: pass class B: __slots__ = ['b'] class C(B,A): pass ``` This doesn't raise an error. Essentially, it seems the default method resolution, when applied to this __slots__ attribute wasn't satisfactory to whoever built it. They then decided to change the method resolution on this method to not allow implicit method resolution for multiple inheritance. But, somehow, redefining __slots__ in the child isn't allowed either. Which should be an option to allows proper method resolution here. In case of simple inheritance, more strange behavior occur. ``` class A: __slots__ = ['A'] class B(A): __slots__ = ['B'] ``` This works, and now, B.__slots__ is equal to ['B']. Over riding works in this case. I'm assuming this feature is still evolving, and testing in higher version of python would lead to different behaviors. I feel the current features of method resolution wasn't satisfactory to __slots__, so they implemented their own strategy. ### My proposal CONTEXT : I'll be exposing possible alternatives, but not go into much depths about the implementations. I'll be proposing them in the order i think they should be introduced, so each new one can benefit from previous ones. 1) Alterhitance CONTEXT: I'm assuming this proposal to be the first to be implemented. No other proposal has to come first. I will still try to pay attention to the final product. However, this proposal is independant of any other coming proposal. As such, it should be evaluated for the values it bring on its own first, and for the value it brings in the complete update (with all my other proposal) second. This is the dedicated feature for dependency injection / inheritance tree alteration. This could be a dedicated module, hosting the few utils functions allowing for easy access / use of the feature. This one is fairly straightforward, the key idea is that i don't wanna rely on super or MRO, not so much at least. It is possible to simply set the value of __bases__ of any class, so that would be my way to go. In the same example as before : ``` class Glider: def push(self) print("I can't move so good on the ground") def jump(self): print("I'm free! I can fly now!") class Wheel: def push(self) print("let's roll !") def jump(self): print("oh damn, i'm falling fast!") class WheelGlider(Wheel, Glider): def push(self): super().push() super(Wheel, self).push() def jump(self): super(Wheel, self).jump() super().jump() ``` The way to inject the mocked class in would be: ``` from alterhitance import Alter class MockedWheel(Wheel): def push(self): print("mocked wheel push") def jump(self): print("mocked wheel jump") WithMockedWheel = Alter(WheelGlider).replace(parent=Wheel, new_parent=MockedWheel) class MockedGlider(Glider): def push(self): print("mocked glider push") def jump(self): print("mocked glider jump") WithMockedWheelAndGlider = Alter(WithMockedWheel).replace(parent=Glider, new_parent=MockedGlider) class MockedWheelGlider(WithMockedWheelAndGlider): d def push(self): print("mocked WG push") super().push() def jump(self): print("mocked WG jump") super().jump() ``` This highlight the ability of this feature to produce each new class with a Mocked parent with relative ease. Note that contrary to what we have to do now, MockedWheelGlider only needs to inherit from one class. The way i've illustrated it here showcases that each class can be injected at once, without having to one shot everything. If you were to create the mock of WheelGlider first, and wanted to mock one of it's parent down the inheritance tree, it could be doable like that: ``` from alterhitance import Alter class MockedWheel(Wheel): def push(self): print("mocked wheel push") def jump(self): print("mocked wheel jump") class MockedGlider(Glider): def push(self): print("mocked glider push") def jump(self): print("mocked glider jump") class MockedWheelGlider(WheelGlider): d def push(self): print("mocked WG push") super().push() def jump(self): print("mocked WG jump") super().jump() WithMockedWheel = Alter(MockedWheelGlider).replace(child=WheelGlider, parent=Wheel, new_parent=MockedWheel) WithMockedWheelAndGlider = Alter(WithMockedWheel).replace(child=WheelGlider, parent=Glider, new_parent=MockedGlider) ``` Note that in this example, we not only define each mock in accordance *only* to what class they are mocking, but we still have a very simple interface for the bottom class MockedWheelGlider. It too only needs to inherit from WheelGlider. Finally, having to call the replace method twice feels a bit redundant, maybe a syntax such as: ``` FullyMockedWheelGlider = Alter(MockedWheelGlider).replace(child=WheelGlider, remap={Wheel: MockedWheel, Glider: MockedGlider}) ``` would be a nicer API. What's to know : Alter class from the alterhitance module take a class at initialisation. all later operation it will run will happen on this class inheritance tree. Assuming we're in the Alter(Example) instance: the replace method takes a few arguments : - child, which is the class in the inheritance tree of Example (all occurence of this class, not only the first one, might need more argument to select one in particular if it is present multiple times today) - parent, which is supposed to be present in the __bases__ of child - new_parent, which is gonna take the place of parent in __bases__ of child (same index) - remap, which is essentially a dict of parent : new_parent. I'm not sure if the argument child is needed, as finding out the child class in the inheritance tree is essentially the same problem as finding the parent class in the inheritance tree. However, it could be a way to lock the remap only to parent in the __bases__ of the correct child class. This argument should probably be optional tho. Alter could provide more methods, such as prune, and add_branch We could also want to be able to select only a branch from the inheritance tree, which would help inspection of the class. That could turn out usefull for dynamically generated classes, for example, classes altered by Alter itself. Pros of this solution: - It makes it easier to mock classes individually as seen in the exemple. In general, it make dependency injection step by step possible. - The API as presented here allows to name each responsibility (Alter, replace, prune, add_branch) , which today's solution do not provide. This makes this feature much more accessible than today's very roundabout ways. - No knowledge of MRO or super is required. - it relies on __bases__, which is a little bit more straightforward (this is essentially the value that defines a class parents), and would make this feature more resilient to changes on MRO and super (no matter how MRO resolves anything, alterhitance updates the __bases__, so MRO will find the replacment at the same time it found the original). - does most definitely *not* need to change anything with current MRO + super to work. Open questions: - I don't know exactly if we need to keep track of the remapping in inheritance trees that have been reworked. super's argument being classes, that we might be remapping or not, should super be changed to account for the new target? I think it works fine now, but in the long term goal of switching to a feature that would take the target of proxying as argument, that would turn out to be needed. - I'm not fixed on any method names, or the overall structure, with the Alter class, alterhitance module, and so on. - I'm not fixed either on methods signatures either - I'm not exactly sure how to cover remap of classes that appear multiple times in the inheritance tree. I'd want a way to target a specific appearance alone, this would allow to "undiamond" diamond cases. I also don't know how much this is needed or not. 2) Postponed / Delayed inheritance OR deep inheritance definition. I'm not fixed on the name yet. CONTEXT: I'm assuming this proposal comes in second. I'm assuming the altheritance proposal was accepted, and implemented first. I will still try to pay attention to the final product. However, this proposal is independant of any other coming proposal. As such, it should be evaluated for the values it bring on its own first, and for the value it brings in the complete update (with all my other proposal) second. The example "Too many combinatory": ``` class View: def render(self): # do smthg class ListView(View): def render(self): # do smthg class DetailView(View): def render(self): # do smthg ... class Permission: def render(self): # do smthg super().render() # do smthg class LoginRequired: def render(self): # do smthg super().render() # do smthg ``` Showcases a usecase where we really want inheritance, but in practice, can't be reasonably expected. The two reasons being: - there's way too many combinations - it's not really possible to reuse a child for another parent. I propose that such scenarios would be able to define the inheritance links between the View classes and the mixins after their definition, when themselves inhreited from: ``` class MyView( LoginRequired( Permission( View ) ) ): pass ``` would be a valid syntax. Essentially, the idea is that, at definition, the syntax ```Son(Dad)``` means Son inherits from Dad. So, in the example above, it would mean : MyView inherits from LoginRequired, which inherits from Permission, which inherits from View. On its own, this syntax doesn't account well for cases where a mixin would be inheriting from multiple parents. So it might be beneficial to add a Placeholder class like that : ``` class Permission(placeholder as view, placeholder as template): ... class MyView( Permission( view = View, template = Template ) ): ``` Thinking about the long term full proposal, having a way to tell which class replaces which without relying on the order of their declaration is relevant, as the proxy feature will likely have to account for remapping. This feature could probably (partially at least) be provided by the alterhitance module. ``` Inherited = Alter(Permission).add_branch(View) Inherited = Alter(LoginRequired).add_branch(Inherited) class MyView(Inherited): ... ``` But this is not as nice an API as this proposal. Pros: - it essentially extends the capacity of class definition. - it makes it very obvious which class specializes which, and produces the inheritances tree accordingly - it is consistent with the inheritance syntax we all know. - it isolates the feature from super + MRO more than today's solution, as the ability of super to target sideway, on the cases where we couldn't produce normal inheritance, is now covered - it replaces a multiple inheritance scenario with simple inheritance. 3) The Diamond problem CONTEXT: I'm assuming this proposal comes in third. I'm assuming the altheritance proposal was accepted, and implemented first. I'm assuming the postponed inheritance proposal was accepted, and implemented second. I will still try to pay attention to the final product. However, this proposal is independant of any other coming proposal. As such, it should be evaluated for the values it bring on its own first, and for the value it brings in the complete update (with all my other proposal) second. Implementation specific details might require some more features tho, I'm still unclear on that. As illustrated by the example "diamond tree, repeat top", it is possible to come up with scenarios in which the methods from the top part of the diamond are expected to be specialised by each parent extending it. It is also very easy to come up with exemples where we would want to top parent specialised only once. For exemple, if the top parent has a save method, that commits something to a database. We wouldn't want one call to the bottom child save method to perform multiple commits This highlights a need for multiple strategies. I propose we add either a decorator, a class attribute, or maybe just an unbound method to specify this strategy. ``` from specialisation_strat import spec, strats class Model: @spec(strats.after_last) def save(self): # call me once class LeftModel(Model): def save(self): super().save() class RightModel(Model): def save(self): super().save() class BottomModel(LeftModel, RightModel): def save(self): super().save() ``` In this exemple, a call to BottomModel().save() visits Model save method only once, when the last call to super refering to it is performed. ``` from specialisation_strat import spec, strats class HighGobelin: @spec(strats.after_each) def scream(self): print("raAaaaAar") class CorruptedGobelin(HighGobelin): def scream(self): print("my corrupted soul makes me wanna scream") super().scream() class ProudGobelin(HighGobelin): def scream(self): print("I ... can't ... contain my scream!") super().scream() class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): super(HalfBreed, self).scream() else: super(ProudGobelin, self).scream() ``` In this exemple, a call to HalfBreed().scream() visits HighGobelin scream method every time a super().scream() call refering to it is performed. We could also allow the child to decide how it wanna specialise it's inheritance tree, so instead of decorating the top parent, we would decorate the bottom method Both option seem to make sense to me, i'd argue we could allow top parent definition *and* override from a child class. For a child class to get more control, we could add another strategy after_classes that would expect a list of class. this list of class would be the list of class whose call to super().method would not be ignored. Whatever we decide, this would apply for cases more complex than the diamond cases, essentially, whenever a class appears multiple times in an inheritance tree. I'm assuming more specific need could be covered, such as allowing strats per branch of the inheritance tree, but this might not be meaningful, as talking about branch doesn't make so much sense when they merge back together. I think this could be implemented by making use of the altheritance module. A strat that visits only after last call is the default option, but the strat to call it every time might need some magic from that. Possibly by using a clone of the top class, (through something copy.deepcopy?) so that super and MRO don't identify it as the same class, and would therefore allow for multiple visits. When the future proxy + method resolution features are implemented, altheritance would most definitely be the way to go i think. Pros: - essentially solve the diamond problem - implementation specifics might rely on MRO and super, but this extract a behavior that we can easily evolve with those feature later on, to preserve the default behavior of today, and allow for more. 4) Proxy CONTEXT: I'm assuming this proposal comes in fourth. I'm assuming the altheritance proposal was accepted, and implemented first. I'm assuming the postponed inheritance proposal was accepted, and implemented second. I'm assuming the diamond problem proposal was accepted, and implemented third. I will still try to pay attention to the final product. However, this proposal is independant of any other coming proposal. As such, it should be evaluated for the values it bring on its own first, and for the value it brings in the complete update (with all my other proposal) second. Since we covered all features that relied on super relying on MRO, we can detach the proxy feature from the MRO feature. This would be a breaking change, as after this change, all code not migrated to the 3 previous features would break. Other than not relying on MRO, there's one thing i wanna change, the proxy feature argument should be the class it targets, instead of today's indirection where you have to pass as argument the previous in MRO order. with the Halfbreed class in the previous example, it would change from ``` class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): super(HalfBreed, self).scream() else: super(ProudGobelin, self).scream() ``` to: ``` class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): self.__as_parent__(ProudGobelin).scream() else: self.__as_parent__(CorrupteGobelin).scream() ``` In case of simple inheritance, the parent wouldn't need to be passed to __as_parent__: ``` class Dad: def joke(self): print("I love your mom") class Son(Dad): def joke(self): print("dad said") self.__as_parent__().joke() ``` Note that i'm not passing self as an argument to __as_parent__, i'm calling __as_parent__ on self instance instead. This makes it awkward to call __as_parent__ for class attributes, like we 'could' (the class wasn't available in its definition) with super: ``` class A: value = 10 class B(A): value = super().value # wouldn't actually work ``` would be replaced by ``` class A: value = 10 class B(A): value = __as_parent__().value # wouldn't actually work ``` However, direct A.value calls are enough, the proxy feature is not needed here at all ``` class A: value = 10 class B(A): value = A.value ``` We would need to adapt the diamond strats, as now passing the target class as argument, instead of implicitely selecting the next in MRO order switches the default behavior in diamond case from call after last, to call after each. The default strat should be kept on call after last, to allow for smoother transition The implementation of the diamond strat module would have to adapt to the change of targeting strategy of the proxy feature, which might not be automatic The altheritance module should not require a lot of extra changes. Maybe, thinking of a class to add for pruned branches would be nice, as it would allow the childs to proxy it, instead of crashing for the lack of a target. A remap dict should probably be stored in classes that are remapped, so the proxy feature can reroute the calls from the target to the remapped parent. The postpone inheritance feature would probably benefit from the introduction, if not done already, of the placeholder parent, so they could serve as target of the proxy (which has to be explicit calls in case of multiple inheritance). Pros: - implicitness of the target is still possible - explicitness of the target is required when multiple parent are present (easier to read / understand the code later) - same simplicity of API. 5) Method resolution CONTEXT: I'm assuming this proposal comes in last. I'm assuming the altheritance proposal was accepted, and implemented first. I'm assuming the postponed inheritance proposal was accepted, and implemented second. I'm assuming the diamond problem proposal was accepted, and implemented third. I'm assuming the proxy proposal was accepted, and implemented fourth, with all the update to the previous 3 features to survive this breaking change. I'm talking here only about the changes to the method resolution, and eventually how it integrates with previous faetures, but not about the previous features themselves. As such, it should be evaluated for the values it bring on its own first, and for the value it brings in the complete update (with all my other proposal) second. Today's MRO forces resolution in any scenarios, which comes at the cost of not allowing some inheritance trees. My proposal is that we stop ordering the parents. The specs are: a method defined in a class body should always be the one resolved to. when only one parent can resolve it, resolves to this method. when no parent can resolve it, raises an AttributeError when multiple parent can resolve it, raises an ExplicitResolutionRequired to give some examples : straightforward ``` class A: def method(self): # A.method resolves to this one pass ``` one parent: ``` class P1: pass class P2: pass class P: def method(self): # A.method resolves to this one pass class A(P1, P2, P): pass ``` multiple parents: ``` class P1: def method(self): # this is a candidate for A.method pass class P2: def method(self): # this is a candidate for A.method pass class A(P1, P2): pass # A.method have multiple candidate, it raises (at runtime) an ExplicitnessRequiredError ``` how to 'survive' an expliciteness required error: ``` class P1: def method(self): # this is not a candidate for A.method anymore pass class P2: def method(self): # this is not a candidate for A.method anymore pass class A(P1, P2): def method(self): # this is what A.method resolves to pass # eventual calls to self.__as_parent__(P1).method() and self.__as_parent__(P2).method() here ``` Pros: - It requires explicitness when it can't resolve implicitely - It works with any inheritance tree (even circular ones if we had some code to prevent loops...) - You won't end up accidentally losing one method / attribute from one parent only because another one had it first

Chris Angelico writes:

And is it still based on the fundamental assumption that super() calls THE parent class? what are you even talking about?

I don't have the time to read that long a post. Then just read the list of 4-5 features i describe as features of current MRO + super, at the top of my post, and tell me if you agree with this analysis or not.

If the proposals are actually independent, maybe propose them one at a time? a lot of features are currently rendered by super + MRO, in a very intricate way. I'm not opposed to dedicating their own posts to each proposal, but i already tried to dedicate this post only to method resolution and super proxy features. You were amongst the many participants that raised complaints about those other features such as dependency injection for example, which is understandable. This leads us to such a lenghty post. Not much i can do about this, super and MRO have way too many responsibilities today. Unless you're ready to start over the discussion on my proposal on method resolution, under the explicit assumption that other features provided by super + MRO today are rendered in their own independant way, which would be the topic of another proposal.

I think you were the one that requested real life code exemples, there's a dedicated sections, i guess you could read it too

Paul Moor writes :

> PS Please don't respond to this with another wall of text. I won't
> read it.
I'll try my best, just tell me if this is too long.

> your fundamental
> misconception that there is always a single well defined "parent
> class"
I don't hold such a conception

>  In practice there are cases (which have been noted here) that
> don't follow that pattern. And if you don't take that into account,
> any proposal you make will be flawed.
That's why I am accounting for them.
To quickly list the cases i account for :
  - the ability of super to 'side jump' used for mixins
  - dependency injection
  - diamond inheritance
I dedicate one proposal to each of those cases, and one to the proxying feature of super, and one to method resolution.

> > a dedicated module for dependency injection and more called alterhitance
> That's not a feature of the current MRO+super.
Today's MRO + super allows for dependency injection (by inserting a class in between other in MRO order), my proposal is to dedicate a module to this feature, which as you mentionned could be published to PyPI now, and i'll be working on it

> > a way to declare inheritance after defining the class (postponed inheritance)
> If you want this as a Python feature, you'll need to describe use
> cases and semantics
Use cases : Mixins.
When we define a child class that could be applied to multiple parent class, we have to define the child independently first, not inheriting from any parent, otherwise we would need to repeat the child definition for each parent.
Syntax: 
```
class MyView(
    PermissionMixin(
        View
    )
):
```
Which just makes use of the child(parent) syntax, and extends it to allows for deeper inheritence tree definition at class definition time.

> > a solution to the diamond problem
> You haven't demonstrated that there's a problem to be solved.
The problem called "diamond problem" is a classic of multiple inheritance, an exemple here that was already posted on this thread : https://stackoverflow.com/questions/55829798/c-diamond-problem-how-to-call-base-method-only-once.
When a class appear multiple time in an inheritance tree, should call to this class methods occur multiple times?
In case the class appearing multiple time does commits to a database, we don't want multiple calls.
In case such as the exemple i described in my lengthy post, we do want all those calls.
I propose a decorator / attribute that would allow users to define the strategy they want.

> > changes to the proxy feature of super (that i already mentionned earlier in this thread)
> What's wrong with how it is now?
when we wanna use super to proxy a class that is not the next in MRO order, we have to pass as argument of super the class that comes before the one we target in MRO order. This API is a bit weird.
And again, i think that despite all the values of its current form, the fact that it can target sideway for one is not easy to understand (i mean, that's what y'all are accusing me of, not understanding it) and makes for a hard learning curves. Which is a problem. A smoother learning curve would be preferable

> > changes to the method resolution algorithm (that i already mentionned earlier in this thread)
> Same as for (4), what's wrong with it now?
Essentially two things :
1) when two parent class provide the same attribute, the child resolves it to the first parent attribute, and silently ignores the second parent attribute.
2) Some inheritance tree produce inconsistent MRO, those class definition raise an error at definition time.


> Overall, you stand very little chance of getting anywhere with this as
> you seem to be unwilling to make the attempt to understand people's
> objections
I've incorporated most peoples remarks and objection to my proposals, as i hope this shorter answer helped you realise.


PS: I'm trying my best to understand anyone remarks / the current state of the features.
I don't think most of you are fair tho. I do understand fairly comprehensively those features.
Idk how some of you are still under the impression that i think super targets the same class no matter what for exemple.
I eventually disagree that this makes for a great proxying feature, which i assume some of you understand as me completely rejecting it's value.
So, allow me to make it clear, I do get its value. Its value appears in the other use cases i account for, and wanna provide dedicated features for. (before getting rid of super and MRO's ability to provide those use case). 

PSS: I hope it wasn't too long.

Chris Angelico writes:

It's a very real problem in C++, because C++ has a completely different concept of multiple inheritance.

Then what's the solution to the diamond problem in python? in this example : ``` class HighGobelin: def scream(self): print("raAaaaAar") class CorruptedGobelin(HighGobelin): def scream(self): print("my corrupted soul makes me wanna scream") super().scream() class ProudGobelin(HighGobelin): def scream(self): print("I ... can't ... contain my scream!") super().scream() class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): super(HalfBreed, self).scream() else: super(ProudGobelin, self).scream() ``` You want calls to ProudGobelin scream method to visit ProudGobelin, then HighGobelin Not ProudGobelin, then CorruptedGobelin, then HighGobelin. If your solution is to revert back to the class.method syntax, you're again denying the proxy feature, the dependency injection use case, and the sideway specialisation (which to be fair is something we explicitely don't want here). Mocking only HighGobelin through the dependency injection of super and MRO is not possible unless you can provide a solution for this case with super. class.method is then an incomplete solution.

On Mon, Apr 04, 2022 at 08:28:31AM +1000, Chris Angelico wrote:

What do you intend for dependency injection to do with this scream method? What would make sense? You've already designed something that works differently.

I don't know why malmiteria keeps talking about dependency injection. None of his examples show dependency injection. https://www.jamesshore.com/v2/blog/2006/dependency-injection-demystified In Python, dependency injection is usually called "passing an object to the constructor". # No dependency injection. class Car: def __init__(self): self.engine = Engine() # With dependency injection. class Car: def __init__(self, engine): self.engine = engine mycar = Car(Engine()) test_car = Car(MockEngine()) With duck typing, the car's dependency, the engine, doesn't have to be an instance of Engine. It can be anything that quacks like an engine: a proxy, a mock engine, a thousand hamsters in wheels, whatever you want to pass in as the dependency, so long as it has the right interface. In the design pattern world, millions of words have been written about this. (The DP folks love to over-engineer their code.) In Python, we just call it passing in an object :-) In any case, it has nothing to do with super or the MRO. -- Steve

On 4/04/22 9:12 am, malmiteria wrote:

So, allow me to make it clear, I do get its value. Its value appears in the other use cases i account for, and wanna provide dedicated features for. (before getting rid of super and MRO's ability to provide those use case).

So, if I understand correctly: 1. You hate the existing MRO and super() mechanism with a passion and want to rip it out. 2. People have objected that this would remove useful funcionality. 3. You address 2 by proposing a bunch of new mechanisms to replace the lost functionality. But you haven't adequately justified step 1 yet. There's no point in discussing any of your other proposals until we get past that. -- Greg

On Mon, 4 Apr 2022 at 03:33, malmiteria <martin.milon@ensc.fr> wrote:

Chris Angelico writes:

...

And is it still based on the fundamental assumption that super() calls THE parent class? what are you even talking about?

I'm talking about this:

feature 1 (of super alone): proxying the parent. What most people think super does (and expect it to do): it allows to call method from *the* (most people don't think of multiple inheritance) parent. It can be used by working around MRO to proxy any parent directly.

You start out with the assumption that MOST PEOPLE think of super as a way to call THE, singular, parent. If this is indeed a problem, then it's not a problem with super, it's a problem with expectations. And you're talking about *working around* the MRO, as if it's a problem. I got a little bit further into your post and found you fighting hard against the existing feature, and that's when I gave up on reading it. Obviously you're going to have problems if you completely misunderstand a feature and then try to work around those expectations. The same happens when people start by assuming that "for-else" loops will run the else clause if the iterable is empty. You've already been given a solution to your problem: if you don't want super, don't use super. The purpose of super is not what you're doing with it. Learn how super is meant to be used, then decide whether it's right for your class hierarchy. Also - you stand a FAR better chance of your proposal being read if you spend the time and effort to edit it down to something manageable. A gigantic wall of typo-filled text is not something we're going to want to sift through. At the very least, test all your code blocks and fix the typos in those. ChrisA

On Mon, 4 Apr 2022 at 07:35, malmiteria <martin.milon@ensc.fr> wrote:
>
> Chris Angelo writes :
> > You start out with the assumption that MOST PEOPLE think of super as a
> > way to call THE, singular, parent. If this is indeed a problem, then
> > it's not a problem with super, it's a problem with expectations.
> This is not so much a problem as this is the context we're working under.
> If we're defining / updating features of the langage, people understanding of the langage do matter.
> A feature that would stick closer to the expectation of most people would provide a much smoother learning curve, for most people.
> And just to make it clear, I am not under the impression super can only target one class.
> What i'm saying here is that it is the most common impression the average developper would have.
>
>
> > I got a little bit further into your post and found you fighting hard
> > against the existing feature, and that's when I gave up on reading it.
> ugh.
> Let's make it simple then. Do you agree this list to be a comprehensive list of use case / feature of today's MRO + super:
> 1) method resolution
> 2) parent proxying
> 3) dependency injection
> 4) sideway specialisation (Mixins use case)
> Not really a use case / feature, but related:
> 5) the diamond problem
>
> If you have any doubts about what i mean by any of those, i describe them at the top of my lengthy post.

The purpose of the MRO and super() is method resolution. That is all.
Everything else is simply ways of using them.

> > At the very least, test all your code blocks and
> > fix the typos in those.
> I do test my code blocks, but if you find errors, please let me know.
> About typos, english is not my native langage, so i'm most likely blind to some of them, as much as i wanna apologise, it's not really in my control, again, let me know if some of them hurt your eyes.
>

I can ignore typos like "langage" in here, but the few code blocks I
looked at in your wall of text were so full of errors that I would
have had to spend quite a while correcting them before being able to
run them. If you did test them, you must have then done further
editing afterwards, NOT tested them again, and then posted.

Don't test your code in French, translate to English, and post without
testing. Test the *exact code you are about to post*.

ChrisA

On Mon, Apr 04, 2022 at 07:17:55PM +1200, Greg Ewing wrote:

On 4/04/22 9:35 am, malmiteria wrote:

...

And just to make it clear, I am not under the impression super can only target one class. What i'm saying here is that it is the most common impression the average developper would have.

How do you know this? Have you undertaken a survey of randomly selected average developers?

I think that malmiteria is probably correct that most developers misunderstand or misuse super, or find it surprising. I know I do. Now matter how many times I remind myself that calling super().method is **not** the same as "call the method of my superclass", I still get surprised by how it actually works in the complicated cases. Simple cases (single inheritence) are fine. But I'm going to be brave, or possibly foolhardy, and say that when it comes to multiple inheritence, if you think you understand what super does in all the fine details, you're either superhuman or wrong :-) In SI, calling super() is effectively the same as calling the superclass, and its all fine. But in MI, calling super() can resolve to something unexpected. See malmiteria's Gobelin example. Now tell me that before you ran the code, or read my analysis of it, you would have predicted that super(ProudGobelin, self) would skip ProudGobelin's method and call CorruptedGobelin's method. I know I didn't. I think that where malmiteria gets it wrong is that he thinks that super is broken. Its not. But we might want to avoid MI, and use something like traits instead. Or composition. Changing super would be tantamount to banning MI, and that would be a massively backwards incompatible breaking change. It isn't going to happen (and nor should it happen, MI is fine for those who need it). -- Steve

Steven D'Aprano writes:

I think that where malmiteria gets it wrong is that he thinks that super is broken I do not, i think its design could be improved overall, but i don't think it's broken. At best I'd argue it can't be used in all of the use case you'd expect for it.

Changing super would be tantamount to banning MI I think my overall proposal accounts for ML pretty well, and i definitely do not intend on removing ML from python. I actually believe my proposal would allow for more ML, as some inheritance trees today are not allowed, and my proposal would allow them.

and that would be a massively backwards incompatible breaking change Not so much, actually, standard simple inheritance scenario would be essentially unchanged. The change to mixin would require this kind of change: From:

class MyView(LoginRequiredMixin, PermissionMixin, View):
    ...

To: ``` class MyView( LoginRequiredMixin( PermissionMixin( View ) ) ): ... ``` And that's it, no big deal The eventual diamond case would be covered by the strategy module, allowing for multiple diamond case strategy, the default one could match today's behavior, so those case would be virtually unchanged. The eventual dependency injection... btw, when i talk about dependency injection, i mean, in the inheritance tree, which allows to mock individual classes in the inheritance tree, for example. So, those scenarios, they could be covered by the alterhitance module i describe in my long post (which would allow to alter inheritance trees by modifying the __bases__ classes attribute) Arguably, those would require the most change. I'm not aware of any actual occurence of this use case in any code tho, so it's hard for me to evaluate how big a deal it might be.

Greg Ewing writes:

It does this because the way it's intended to be used is to pass it the class in which the method you're calling it from is defined.

It doesn't really matter what's the intended way for it to be used tho, what matters when it comes to designing / updating the design of a feature is how people are using it today, and what people expect it to do naturally. Taking this into account is the easiest way to make an intuitive feature. The way it is intended to be used as virtually no value (when it comes to designing / updating the design of a feature). And updating the super feature(s) is what i'm proposing here.

Now that we have argumentless super, there's no longer any need to do that [pass argument to super].

Some exemple i give including, but not limited to the gobelin exemple are a fair use case in which you'd want to use the argumented version of super. The fact that today's super doesn't work in those scenario just means super doesn't work in all scenarios, I believe the scenarios i built to be fairly consistent with the average developper expectation of super's behavior, based on acquired knowledge on use of super in the general cases.

If you're using super at all with MI, your methods should all be designed so that it *doesn't matter* exactly what order they get called in This doesn't match the mixin case, which is the most common use today for ML. Essentially, in this case, you have a collection of parent class, all of which would benefit from having the same variant child class. The exemple i always give : you have multiple view class, you have the permission mixin, so you (as a lib author) wanna provide all possible combinations of simple view class, and permission mixin augmented view class. today's only (reasonable) way is ML.

class MyView(Permission, View): ...

This explicitely rely by design on the fact that permission comes before view. I'd argue this is a case of "I can't declare inheritance at definition, and I can't declare it later either, so we have to resort to ML" And this is covered by my adoption proposal, which allows for setting inheritance relationship in the MyView class. ``` class MyView( Permission( View ) ): ... ``` But I agree with the feeling that you should not expect an order between the parents. And essentially, if i get you right here, you're saying that ML should be designed (when we use it, at least) so that we could reorder the parent in the class definition, and it wouldn't matter, wouldn't break anything. Essentially, (i'm talking about what i understand you consider a properly designed ML scenario): ``` class A(P1,P2): ... ``` Is 100% equivalent of ``` class A(P2,P1): ... ``` correct me if i'm wrong. My proposal makes this an explicit feature of ML. As you have to think of a lot of things to make it work this way today, and this would be the default with my proposal. And as mentionned, there are cases that simply can't work this way. Also, i do wanna mention that you do in fact care, to some extent to which order things are called in. Not always, but in some cases you do, and not being able to target one of the parent first, instead of the default one, when the default one is not appropriate is a problem. Essentially, if for any reasons today, MRO is not consistently the good order for you, all you can do is forget about it. When really, there's no reason (not in term of UX i mean) to be so opinionated.

On Wed, Apr 06, 2022 at 08:28:59PM +1200, Greg Ewing wrote:

I actually think super() is misnamed and should really be called next_class() or something like that. There might be less confusion about its intended use then.

Heh, we should rename it frábær(), that will ensure that nobody will use it without reading the documentation! :-) -- Steve

On Sun, Apr 03, 2022 at 03:09:18PM -0000, malmiteria wrote:

feature 1 (of super alone): proxying the parent. What most people think super does (and expect it to do): it allows to call method from *the* (most people don't think of multiple inheritance) parent.

For single inheritance, that is exactly what it does, and there is no problem. Just use super() and it Just Works. If you disagree, then please show me an example of *single* inheritance where super does not do the right thing, where each class simply calls its direct parent.

It can be used by working around MRO to proxy any parent directly.

I don't know what you mean by this. Why are you "working around" the MRO? If your class is skipping some of its superclasses, then it is either broken, or you shouldn't be using inheritance for the relationships between the classes.

feature 2 (of super + MRO): sideway specialization / mixins

This is just a form of multiple inheritance where the mixins are not designed to be instantiated themselves.

feature 3 (of super + MRO): dependency injection / inheritance tree alteration

I don't know what you mean by dependency injection here. Dependency injection is one of those Design Patterns beloved by Java developers who over-engineer it into something complicated, but in Python is so trivial that it barely deserves a name. To quote James Shore: “Dependency Injection” is a 25-dollar term for a 5-cent concept. https://www.jamesshore.com/v2/blog/2006/dependency-injection-demystified Inheritance tree modification... I think that means you want to copy a class hierarchy, but with changes? Again, this sounds like inheritance is the wrong tool here. Nnot everything is an inheritance relationship.

feature 4 (of MRO alone): method resolution

This is the one and only thing that super() if for. Well to be precise, its not just methods, but any attributes.

"feature" 5 : diamond problem.

The diamond problem is a feature of multiple inheritance with diamonds. super() can't solve it in the fully general case because there is no general solution.

What motivated me first here was the few problems with the feature 1 (proxying the parent)

What problems? Here you are talking about *the* parent, so we have single inheritance: ``` class Parent: def method(self): print("the parent does stuff") class Child(Parent): def method(self): ... # what problems with super? ``` Please show me what problems you have with super in that simple case of a class with only a single parent class. [...]

class HighGobelin: def scream(self): print("raAaaaAar")

class CorruptedGobelin(HighGobelin): def scream(self): print("my corrupted soul makes me wanna scream") super().scream()

class ProudGobelin(HighGobelin): def scream(self): print("I ... can't ... contain my scream!") super().scream()

class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): # 50% chance to call ProudGobelin scream, 50% chance to call CorruptedGobelin scream ```

when writing down the class HalfBreed, we expect the behavior of HalfBreed().scream() to be one of its parent behavior, selected at random with 50% chance each.

Okay.

with the use of super, it could look like that, intuitively

Not it can't. This is wrong. You have misunderstood what super does. What you are doing in this broken class below:

class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): super(HalfBreed, self).scream() else: super(ProudGobelin, self).scream()

is to try (and fail!) to use inheritance for behaviour which needs to be modelled by delegation. By definition, inheritance requires HalfBreed to call the methods of all three superclasses. (That is at least the definition used in Python, other languages may do differently.) If HalfBreed is not using the methods of all its superclasses, it is not really an inheritance relationship. And you provide the right solution: delegation.

class HalfBreed(ProudGobelin, CorrupteGobelin): def scream(self): if random.choices([True, False]): ProudGobelin.scream(self) else: CorrupteGobelin.scream(self)

Which is an option multiple of you pointed out to me. This options however is flawed in muliple ways, mainly because it looses 3 of the 4 features i describe earlier.

- we lose the proxy feature, which to be fair, is aqueen to syntactic sugar. But this is still a feature loss.

How do you lose a feature you don't want? You don't want to call the parent classes' methods using inheritance. If you did, you would use super, and the HalfBreed would inherit from all three superclasses. But you don't want that, do you do something else. Problem solved.

- we lose the sideway specialisation, coming from super's behavior, which, in this case, is the goal, so we're not complaining.

What sideway specialisation? You have no mixin classes here.

- we lose the possibility of class dependency injection, since today, it relies on a consistent use of super.

No it doesn't. Dependency injection has nothing to do with super. -- Steve

On Tue, 5 Apr 2022 at 03:58, malmiteria <martin.milon@ensc.fr> wrote:

Steven D'Aprano writes:

...

...

feature 1 (of super alone): proxying the parent. What most people think super does (and expect it to do): it allows to call method from *the* (most people don't think of multiple inheritance) parent. For single inheritance, that is exactly what it does, and there is no problem. Just use super() and it Just Works.

I 100% agree. I don't see a problem with super and MRO for today's single inheritance.

That's why I explicitely care to make my porposal equivalent to today's super and MRO in single inheritance scenarios.

...

...

It can be used by working around MRO to proxy any parent directly. I don't know what you mean by this. Why are you "working around" the MRO?

super(A, self) does not proxy to A, but to the first *after* A in MRO order. When you're actually in need to passing arguments to super, you very likely know what class you're gonna be targeting, and having to run MRO logic in reverse to call super to the class that comes first before your target in MRO order is what i refer to as "working around MRO".

This right here, this is the fundamental problem. When you call super(A, self), you are expecting to call A's method. This is simply wrong. This is not super's job. If you want to call A's method, call A.method(self). Don't call super. If you want to call the next method after A, use super(A, self). And most of the time, what you want is "the next method after this class", which is super(__class__, self), and can be abbreviated super(). You are working around super because you assume that every call must go through it. That's simply not the case. You are working around *your own misunderstanding of super*, and then blaming super for your awkwardness. *STOP USING SUPER* where it's the wrong tool for the job. Your problems will vanish.

...

Again, this sounds like inheritance is the wrong tool here. Nnot everything is an inheritance relationship. Nnot the only one making typos hehe :p

I never ask people to avoid making any typos... but I do ask people to test *code blocks* before posting them. (And, yes, I have been guilty of posting untested and buggy code blocks. But you'd be fully justified in calling me out on that one.)

...

...

feature 4 (of MRO alone): method resolution This is the one and only thing that super() if for. Well to be precise, its not just methods, but any attributes. super does not provide method resolution. MRO gives the order in which attribute / method would be looked up for.

``` class A: def method(self): pass class B(A): pass

B().method() # works despite B body not declaring "method", and resolves to the method defined in A body. ``` This is method resolution. super is not a part of it Although, super do rely on it, since it will target the next in MRO order to select the class to proxy.

They're closely linked. The MRO exists, and super is used as a means of skipping part of it.

...

The diamond problem is a feature of multiple inheritance with diamonds. super() can't solve it in the fully general case because there is no general solution. That's why my proposal includes a inheritence strategy module, which would allows users to pick and choose which kind of strategy they need. It would default to the current strategy which consists of calling the parent that apppears multiple times in an inheritance tree only on its last occurence. Other strategy could be to call parent that appear multiple times on each occurence, or even to specify which occurence should be called, and which should be ignored. That's a "no general solution" problem solved.

Python *already has* all these strategies. Some of them involve the use of super. Some do not.

...

is to try (and fail!) to use inheritance for behaviour which needs to be modelled by delegation. [...] And you provide the right solution: delegation. Today's right solution. Again, i'm showcasing a limit of today's super.

Is super the wrong tool for the job? Don't use super. Python's integer type has some fairly severe limitations, notably that it can't handle fractional values. Let's change the int type to be more intuitive - dividing an int by an int should return an int, with the exact value of that fraction. So 17/3 should return an int with the value of five and two thirds. That's intuitive, right? Today's int type has this weird limit. Or maybe I should be using Fraction or float instead of int. Maybe that's a better solution.

...

By definition, inheritance requires HalfBreed to call the methods of all three superclasses. (That is at least the definition used in Python, other languages may do differently.) If HalfBreed is not using the methods of all its superclasses, it is not really an inheritance relationship. By this definition, the behavior i wanna implement in HalfBreed is one of inheritance, as it is using the methods of all its superclasses. As you were quick to point out, super does not cover this scenario, which is the problem i'm highlighting with this example.

Is it a problem that super doesn't handle this (somewhat weird and extremely hypothetical) situation? You can use explicit method resolution instead. ChrisA

On Tue, 5 Apr 2022 at 07:21, malmiteria <martin.milon@ensc.fr> wrote:

Just for everyone reading this, I'm writing this one only for Chris Angelico. As much as some others here have been heating up during the discussion, which is understandable, i believe most of you were willing / able to provide a fair discussion, and still are, which shows maturity, and is something i respect. This is something i believe you, Chris Angelico, are not capable of.

If you meant it only for me, you could have sent it privately.

Chris Angelico writes:

...

malmiteria writes:

...

super(A, self) does not proxy to A, but to the first *after* A in MRO order. When you call super(A, self), you are expecting to call A's method.

At this point this is cognitive dissonance. Are you not willing / able to understand what i am saying to you?

I completely understand what you are saying. I disagree with the underlying assumptions.

I do not expect super(A, self) to call A's method. I very much know it doesn't. As i am *very explicitely* telling you in those quotes What i am saying is that *if* super(A, self) *were* to call A's method, that would be a simpler API. If you understand that this is my proposal, and not my understanding of today's super, then for the love of god, address my proposal.

I understand that this is your proposal and I think it is a bad proposal. That is why I have kept saying what I have been saying.

Read that again, but slowly.

Read it again.

Once more.

Do you still believe i think super(A, self) calls A's method? Whatever your answer here, read that again.

Again.

I do not believe super(A, self) calls A's method.

Read that again.

Are you starting to understand?

Alright, enough, I get the idea. You do not understand what my point is, and you think that I don't understand yours. There's not a lot of point discussing further.

You do not seem to understand the current links between super and MRO

Excuse me?

Is the API i propose better than today's API?

No, it is not.

I genuinly want a discussion here.

Really? Then start by understanding what everyone has been saying: that you do not have to use super for everything. You're trying to warp super to fit your expectations, instead of understanding what it does and doesn't do. I am trying my hardest to have a discussion with you, instead of what I probably should have done several posts ago and simply deleted your emails and moved on with my life. But since you are clearly not returning the favour, I am now done. Good luck with your proposal, maybe you can team up with jmf and make a new version of Python that fixes all the problems in it. ChrisA

On Mon, 4 Apr 2022 at 22:19, malmiteria <martin.milon@ensc.fr> wrote:

What i am saying is that *if* super(A, self) *were* to call A's method, that would be a simpler API.

I understand that you're specifically responding to Chris, but to be honest, this is the first time you've explicitly stated this (unless it was previously lost in a sea of words somewhere). So I'd like to respond. If you're suggesting *changing* super's behaviour like this, you have to explain how you expect to handle the (quite possibly literally) millions of lines of code that would be broken by making such a change. So far, all you've offered is that "it would be a simpler API" and had no-one agree with you that this is the case. That's not even remotely close to being sufficient to make such a drastic change.

If you understand that this is my proposal, and not my understanding of today's super, then for the love of god, address my proposal.

If that *is* your proposal, then just drop it. You seem incapable of presenting a case that will justify such a massive breakage, so you're just wasting everyone's time making them read your huge emails. Paul

On Mon, 4 Apr 2022 at 18:59, malmiteria <martin.milon@ensc.fr> wrote:

super(A, self) does not proxy to A, but to the first *after* A in MRO order.

Correct, that's how it's defined to work.

When you're actually in need to passing arguments to super, you very likely know what class you're gonna be targeting, and having to run MRO logic in reverse to call super to the class that comes first before your target in MRO order is what i refer to as "working around MRO".

Not at all, you pass *your own class*, and super() works things out for you. Or more simply, just omit the type and super() will work as you want it to. class A(...): def foo(): super().foo() # or super(A).foo() Note that I left the set of bases unspecified, as ..., to demonstrate that you don't need to care what they are. That's the point - the type argument to super() can be omitted in 99% of cases, and whenever it can't, it should be the last class you know about. There's no "running the MRO in reverse". In reality, I can't think of a single realistic case where you'd specify a type argument to super() anyway. Before you mention it, your xxxGoblin/HalfBreed example should be using delegation, as Steven D'Aprano pointed out - using super() in a case where you're trying to force a specific resolution path is *not what super is for*. Maybe you do have use cases where super isn't the right tool for the job. That's entirely possible. But that doesn't mean we should modify super to handle those cases as well as its current function - *especially* not if there are other solutions available in Python today, which don't use super. If you're trying to hit a nail into a piece of wood, and your screwdriver isn't doing a good job at it, that means that you should learn about hammers, not that you should propose that screwdrivers get modified to get better at hitting nails... Paul

On Mon, Apr 04, 2022 at 09:45:32PM -0000, malmiteria wrote:

All we have today is the screwhammer, and in cases the screw part don't work, you're telling me to get rid of it all. hammer part included.

That's not how I see it. I see that we have a screwdriver (inheritence, including multiple inheritence) which is designed for the cases where you, the programmer, don't want to manually specify which superclass method to call, you want the interpreter to do it using the defined linearisation. And then we have a hammer, composition/delegation, for when you *do* want to control what method is called. As I see it, it is you who wants to combine the two into a screwhammer, so that you can use the same mechanism (super and inheritence) both for the inheritence case (follow the linearisation) and the composition/ delegation case (manually specify the method you want to call). That is exactly what you did in your Gobelin example, with the HalfBreed sometimes using plain old inheritence using super, and sometimes trying to jump part of the MRO and call a specific method, but you still used super for that. ``` class HalfBreed(ProudGobelin, CorruptedGobelin): def scream(self): if random.choice([True, False]): # This is plain old inheritence, in Python 3 we can # write it as just super().scream() super(HalfBreed, self).scream() else: # And this is a failed attempt to ignore inheritence # and delegate to ProudGobelin.scream using super. super(ProudGobelin, self).scream() ``` So in this example at least, you try to use the same screwdriver (super) for both inheritence (driving screws) and delegation (hammering nails). -- Steve

On 7/04/22 5:22 am, malmiteria wrote:

Also, i believe the idea of using anything but super to access a parent methods is far from obvious to most people.

That might be true for people who learned Python recently enough. When I started using Python, super didn't even exist, so I got used to thinking of Class.method as the *normal* way to call inherited methods. When super first appeared I saw it as something you only use when you particularly need it, i.e. when doing cooperative MI.

This alone justifies the idea that any newcomer to this kind of problem would try to use super, at least at first. And possibly wouldn't expect super targeting to behave like it does.

If I were teaching a newcomer about super, I wouldn't even tell them that it *has* a class argument. So they wouldn't have any expectation about targeting, because they wouldn't know about it.

if proudgobelin and corruptegobelin are published by a game_engine library, the game_engine user would most likely not be aware (nor should he care) that they both inherit from a same parent.

If someone is going to munge those classes together using MI, they'd better learn everything they possibly can about them. It's a delicate operation that requires knowing a *lot* about the classes you're blending together. In this case, the fact that both class names have the form <adjective>Gobelin would make me suspect quite strongly that they *do* have some common ancestry. -- Greg

Chris Angelico writes:

I'm curious when you would ever be subclassing something from another library without knowing its hierarchy.

When the class is a public API, no? I'm not sure why this isn't obvious, am I missing something? One really plausible example is given in Raymond's piece: a later version of the same library refactors a "monolithic" class as a child of one or more "private" classes that are not intended to be exposed in the public API, but your multiply-derived class *written before the refactoring* Just Works. As far as I can see, super(), and maybe even the deterministic MRO, is needed to make that work. Steve

If the only feature you need from super is the proxy one, why don't you code your own parent-proxy-type? class parentproxy: def __init__(self, cls, obj): self.__thisclass__ = cls self.__self_class__ = type(obj) self.__self__ = obj def __getattr__(self, name): attr = getattr(self.__thisclass__, name) if hasattr(attr, '__get__'): attr = attr.__get__(self.__self_class__, self.__self__) return attr class A: x = 0 y = 1 def method(self): return 'A.method' class B(A): y = 2 def method(self): return 'B.method' obj = B() # prints 0, 0, 0 print(obj.x, parentproxy(B, obj).x, parentproxy(A, obj).x) # prints 2, 2, 1 print(obj.y, parentproxy(B, obj).y, parentproxy(A, obj).y) # prints B.method, B.method, A.method print(obj.method(), parentproxy(B, obj).method(), parentproxy(A, obj).method()) It gives the proxy you need and it works well with multiple inheritance too. Le jeu. 7 avr. 2022 à 13:14, malmiteria <martin.milon@ensc.fr> a écrit :

Greg Ewing writes:

...

If I were teaching a newcomer about super, I wouldn't even tell them that it *has* a class argument. So they wouldn't have any expectation about targeting, because they wouldn't know about it.

I would probably teach newcomers the argumentless form only too. That's all they'll need. But argument or not, they need to know "what" does super proxy, and the simple answer, for a newcomer, is "the parent". If they were to ask me what happens when there's multiple parents, i would definitely have to tell them not to go there, today at least, since today's multiple inheritance is complex. But if there was a way to tell super what class it should be a proxy of, that would be very easy to explain : when there's two parent, just give the parent you want to target as an argument to super.

...

It's a delicate operation that requires knowing a *lot* about the classes you're blending together. In this case, the fact that both class names have the form <adjective>Gobelin would make me suspect quite strongly that they *do* have some common ancestry.

It requires knowing a *lot* about the classes you're inheriting from, *today*, and that's a problem I'm trying to adress. And you being able to tell those 2 classes have some sort of common ancestry wouldn't be of much help if you don't already know about MRO and the effect it might have on inherited classes in case of multiple inheritance. So you would already need to be quite knowledgable about MRO and multiple inheritance to got the result you expect here.

If super were to only target the parents, either the only parent for the argumentless form, or the specified parent in case of MI, this scpeific scenario wouldn't require you to have any extra knowledge on MRO / super / MI. It would already behave exactly like you would expect.

Of course there are use cases for super following MRO. Those need to be covered too, and not get rid of. I think the ability of super to handle remaps after MRO injections should be conserved, so the scenarios in which we would pass directly the class we wanna target should allow for remapping of the parents. Not that there's a common use for it, but still.

The Mixin use case, where we explicitely use the super ability to side jump so that our mixin 'specialise' the last class in MI order would really benefit from a feature allowing a class to select a parent after being defined. This is the meaning of my adpotion proposal, and i believe this use case to be the most common use case of MI in python. We could actually consider it a case of simple inheritance that didn't say its name. The adoption proposal would convert those MI cases back to SI cases.

The diamond problem on its own requires specific attention, as the gobelin exemple i give showcases a case where we would most definitely want the top class to be called every time it appears in the inheritance tree, but there are other cases where we most definitely want the top class to be called only once, case such as ORMs doing commits to databases. My proposal is to implement a way to allow the programmer to decide what strategy fits their needs. the 'use_parent_mro' keyword of super would work nice in today's context, perhaps not so nice assuming all other changes i advocate for passed first. A class decorator / attribute deciding the strategy super should use is an option too. Follow MRO / follow inheritance tree for exemple would be 2 simple options. honestly, idk what design would be the best for this case, all i know is that the solution should allow for multiple strategies.

---- Chris Angelico writes:

...

I'm curious when you would ever be subclassing something from another library without knowing its hierarchy. This is common in Django.

---- Stephen J. Turnbull writes:

...

One really plausible example is given in Raymond's piece: a later version of the same library refactors a "monolithic" class as a child of one or more "private" classes that are not intended to be exposed in the public API, but your multiply-derived class *written before the refactoring* Just Works. As far as I can see, super(), and maybe even the deterministic MRO, is needed to make that work. I'm curious about this exemple, do you have a link to share so i could have a look at the code / change in code? This could be a good exercice.

---

Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LRKJNN... Code of Conduct: http://python.org/psf/codeofconduct/

-- Antoine Rozo

If these examples were possible (I wouldn't say they are smart designs) they would lead to recursion errors. Limitations on MRO are good, they force to keep a quite simple structure. Le jeu. 7 avr. 2022 à 17:41, malmiteria <martin.milon@ensc.fr> a écrit :

Antoine Rozo writes:

...

If the only feature you need from super is the proxy one, why don't you code your own parent-proxy-type?

I did : https://github.com/malmiteria/super-alternative-to-super/blob/master/parent....

This is irrelevant to the discussion we're having i think. Essentially, I'm arguing against today's state of some edge case of MRO + super, and against the UX associated with it. Those are issues with today's python, and the update that i propose would reduce the UX problems with super and MRO, would allow for use case of super more in line with the expectation of the majority, and would open the door to a few cases locked behind MRO errors today. Technically, with my proposal, you could even do circular inheritance, which is definitely unheard of today: ``` class Day: def tell_time(self): print("it's daytime") sleep(10000) super().tell_time()

class Night(Day): def tell_time(self): print("it's night time") sleep(10000) super().tell_time()

Day.__bases__ = (Night, )

Day().tell_time() # infinitely loops over "it's daytime" and "it's night time" ``` That would be an incredibely easy way to articulate process that repeat in a cycle, with no end, cron style. No need to get multiple class too: ``` class CronTask: def task(self): # do something time.sleep(10000) super().task()

CronTask.__bases__ = (CronTask, )

CronTask().task() # runs the task forever with a time sleep in between ```

I'm convinced there's some smart designs that are banned from python because of MRO and super's limitations. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BKFSLL... Code of Conduct: http://python.org/psf/codeofconduct/

-- Antoine Rozo

On Fri, 8 Apr 2022 at 22:56, Joao S. O. Bueno <jsbueno@python.org.br> wrote:

Hi. I've replied to the first e-mail on this thread, more than 10 days ago. I am back, though I've read most of what was written.

I don't think things have improved, but you sure are consuming everyone's time

You are still repeating this: "more in line with the expectation of the majority, "

Though, as already asked, there is zero (nothing) to support that. I've seen exactly _one_ e-mail among those in the thread, that seemed to need something different from the current status quo - though not exactly what you offer. I replied in private as that user's needs could be fulfilled with a custom metaclass, offering personal help with that (and did not get a reply).

So, I'd suggest to you, if not for others, at least for myself, that you'd get some backup on what this "majority" you claim could be. Could you set, I don't know, some online form? With questions like:

"on the following scenario, what do you [think|prefer] 'super' [does|could do]?"

Then we can check. No need for "majority" - get at least some 10 respondents, with 2 or 3 of those thinking the same as you, and then maybe it would make sense insisting on this path, as there could be something in there.

While I admire the intent here, unfortunately, a survey like that is almost completely useless. It's easy to trap people into thinking that super does something different from what it does, but that still doesn't show that super needs to be changed. It might be better to word it like this: In the given scenario, which of these lines of code would you expect to have this behaviour? * super().method() * ParentClass.method() * method() * ::method() * ^method() * super[1].method() and ask people to rank them in order of which ones make the most sense. I still don't think the survey would be hugely useful, but this sort of wording is a better way of judging people's expectations than asking them to describe the behaviour of a short form like "super().method()". ChrisA

On Sat, Apr 9, 2022, 7:31 AM malmiteria <martin.milon@ensc.fr> wrote:

Joao S. O. Bueno writes:

...

You are still repeating this: "more in line with the expectation of the majority, " Though, as already asked, there is zero (nothing) to support that.

Here's some more evidence of a sort: I've taught hundreds, maybe thousands, of scientists and software developers Python. Some only knew different programming languages, some wanted to understand Python more deeply. Malmiteria is the FIRST person I've met "confused" by super(). Most of those are not confused because they've simply never had any reason to give it deeper thought. It never did and never will matter to them. A much smaller number were not confused because they are accustomed to using deep and branched inheritance trees, and therefore read Michele's paper on C3 MRO. Another open question is the threshold at which we would all agree there's

a problem to be fixed i guess?

I guess the main threshold to try to cross is ONE person other than malmiteria in the universe of Python users.

On Sun, Apr 10, 2022 at 01:53:37PM -0000, malmiteria wrote:

David Mertz, Ph.D. writes:

...

I guess the main threshold to try to cross is ONE person other than malmiteria in the universe of Python users.

I don't believe that David's denials that people are confused by super() are even a little bit reasonable. Of course people are confused by super, especially when it comes to multiple inheritance. Its a confusing technique. https://fuhm.net/super-harmful/ The problem with super is not super, but MI, which is inherently hard to use correctly even if you don't misunderstand super().

A coworker of mine once switched the order in which class were inherited from stating it wasn't gonna change anything.

Yes well that was just silly. Of course the order matters. Even in single inheritance, the order matters: Spam inherits from Eggs inherits from Cheese is not the same as Spam inherits from Cheese inherits from Eggs in the general case. We've all done "What the hell was I thinking?!?" errors when programming. I'm sure I've done sillier. -- Steve

On Sun, Apr 10, 2022, 10:35 AM Steven D'Aprano

I don't believe that David's denials that people are confused by super() are even a little bit reasonable.

I take the words "unconcerned" and "confused" to be distinct.

On Sun, Apr 10, 2022, 11:51 AM malmiteria

"class A(B)" feels very different from "class B(A)" but "class A(B,C)" doesn't feel so obviously different from "class A(C,B)"

Are you likewise "confused" by the fact that `a = b - c` is generally different from `a = c - b`?!

Can you think of ANY context in Python in which the order of items in parentheses isn't important?! (a, b, c) != (c, b, a) func(a, b, c) != func(c, b, a) etc. You are arguing that defining inheritance order is "intuitively" the one and only context in which the order of items in parentheses makes no difference. On Sun, Apr 10, 2022, 1:23 PM malmiteria <martin.milon@ensc.fr> wrote:

David Mertz, Ph.D. writes:

...

Are you likewise "confused" by the fact that `a = b - c` is generally different from `a = c - b`?!

Why do you always quote confused?

I'm not, but that's because i've been taught / i've experienced that since primary school. I have been taught math, but not python, nor any programing language.

Most people experience with python will lead them to understand easily that parenthesis are definitely not symetrical, since calling a function is a common thing to do in python, and most programming language. As much as they would be able to understand that there is many different context in which what's define inside the parenthesis doesn't always mean the same thing. And overall, the "parenthesis operation" doesn't exists as one simple thing. Sure there's __call__, but parenthesis are also used in method definition, method call and class definition, on top of class calls. As much as there is some importance of the order of the arguments in method definition, it's definitely a fair assumption, based on generic experiences with the language, that parenthesis in the syntax "class A(B)" simply means something different, and is a different operation.

However, the class definition allows you to refer the class by the name placed before the parenthesis, so it's obvious even only with a generic python experience that the "inside/outside" parenthesis order matter. Much more than the order within the parenthesis. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/43AEEI... Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, 11 Apr 2022 at 03:57, malmiteria <martin.milon@ensc.fr> wrote:

David Mertz, Ph.D. writes:

...

Can you think of ANY context in Python in which the order of items in parentheses isn't important?! kwargs

Since Python 3.6, order of kwargs is preserved, and some functions do care about it (although only in minor ways, like the display order of attributes). ChrisA

Steven D'Aprano writes:

In any case, how do you know what "most people" think? Have you done a survey, or are you just guessing? I haven't done a survey, but I'm getting one ready, I also wanna let you all have a read at it before running it, so we can all agree the data we would get from it are meaningful, if that's okay with you.

I am not guessing either however, knowledge is formed in a somewhat deterministic way, and UX design to some extent plays a role in it. You are very likely to form knowledge coherent with what you experience, and the most experience in case of use of super is simple inheritance, which means you're likely to learn a lot about super and SI before you face a MI case, in which, you'd have only the SI experience to build off of an expectation.

Actually, we are defining arguments to be passed: the listed superclasses are passed on to the class metaclass as a tuple, which preserves order, and the MRO is generated from that ordered tuple. Same problem as before, this is definitely not obvious to most python users, they never have to dive in those metaclasses logics, and therefore, it can't play a role in their understanding of what's happening at class definition. So again, the most common intuition on what happens at class definition would would be very different from the most common intuition about method definition / calls. Essentially, I'm saying that most people intuition is something, and you're answering that this thing almost no one knows about shows otherwise. It doesn't, since they don't know about it. You get my point.

Steven D'Aprano writes:

So in the general case, order matters. We have to linearize the superclasses, and call them in that linear order, and the best way to do that is with the MRO and super. Why would we *have* to do that? When multiple parent provide candidate to a method resolution, raise an error. The child class now has to redefine the method, and in the body of that method, can decide which parent method call, or in what order call them. That's essentially the basic idea of my proposal. What makes this impossible for you? I think i've adressed most if not all problems i had raised against it.

This gives much more power to python users, clarifies possible missed name collisions and changes absolutely nothing on SI. And why do you say MRO and super is the best way to do that? If you mean it's the best way to do linearisation fine, but if you're saying it's the best way to do method / attribute resolution, i disagree. Linearisation is litterally an operation that consist into converting a multiple inheritance tree into a simple inheritance tree. That's bound to lose some info, and can't cover all cases.

Ronald Oussoren writes:

To be blunt: That’s not going to happen because this is big backward compatibility break. Either that, or this adds a second way to define classes. Both are good reasons to keep the status quo.

the breaking is limited MI with name collision, so likely rare enough, and i've already mentionned that i'm perfectly willing to implement my proposal and run it against whatever selection of python repos, to check for real life exemple of breakings, so we can measure how bad it would actually be. The most common case of breaking would be the django style mixins, which i propose a dedicated feature for, adoption. Essentially a way to declare more than your parent, but their parents too, recursively. would look like that: ``` class A(B(C)): ... ``` the adoption syntax is non breaking, as it's simply an addition to the language, and has values on its own. I'll make a dedicated post, but i'm a bit out of time for that now, especially since i'm getting the survey ready. I'll add some question to get infos that could help debate this feature too, so i guess i'll wait for the survey result. Once adoption is ... adopted, and start getting used, the more it goes on, the less breaking change will occur. And, on top of that, the actual change needed to switch from today's solution to adoption is extremly simple. replace ``` class A(B,C): ... ``` with ``` class A(B(C)): ... ``` That's it. So the amount of actual breakings left shouldn't be enough to justify denying this feature IMO. Again, we'll have a clearer view on that once we get experimental data. What's to look for is usage of super in MI cases, since each super call in MI today should be replaced by one super call per parent, in my proposal. If this turns out to be an issue, we can try to imagine solution for it. Maybe a solution such as : super calls implicitely run a super call to each parent when there's multiple parent, and super isn't given arguments. This i think would behave strictly as today's super + MRO, except for cases where one class appears multiple time in an inheritance tree. The last breaking change would be that scenario, class appearing multiple time in an inheritance tree. And we're getting on even rarer occasions here, but again, we can think of solutions here. As much as i understand we don't want breaking change, i don't think it is *that* strong an argument, in this case. And, on its own, breaking changes aren't a strict veto, there's been some, and there's gonna be more. Overall, they are an upgrade cost we really wanna make as less expensive as possible, but an worthwhile upgrade overcomes the upgrade costs. And, if it improves something, the longer we wait, the more the current behavior can deal its damage, it should be considered in the balance too. Breaking changes are an momentary cost, while not upgrading might have a smaller, but constant cost.

Chris Angelico writes:

A proposal that breaks existing code, and which introduces a new way to do things, will usually require multiple versions of deprecation time in order to be accepted. Nothing is preventing us from doing that here. I already intend to have the adoption syntax come first, since it doesn't break anything pre existing, it can be added, and a few version later, the breaking change can happen, you've got your deprecation time.

That usually means either a compatibility period, or a means of continuing to use the older syntax. A compatibility period is probably the only way for the entire proposal, but for most parts of it, i'm pretty sure it could be possible to have the older syntax still apply.

In the case of the "except Exception, e:" syntax, the justification is that it is confusingly similar to "except (Exception1, Exception2):" in a way that causes hard-to-debug problems. I've already been over how MRO causes hard to debug problems, since it makes it possible for one class to ignore one of its parent attribute, while using the other parents attribute instead, without any warning / errors.

Actually i got another exemple recently (at work, i wasn't really looking for it), on django doc : https://docs.djangoproject.com/fr/4.0/topics/auth/default/#django.contrib.au... some classes dedicated to test some users all inherit from the same UserPassesTestMixin class. This class has some NotImplemented errors, in the method it provides to it's childrens. If we were to make multiple TestMixins, like described in the link i shared, calls to super aren't possible, and would lead to the mixins needing to be inherited all together. So there's no point writing down multiple of them, might aswell write down just one, and why would you, when you have multiple tests that have nothing to do with one another. The doc concludes it's impossible to use multiple TestMixins, and doesn't mention at all the class.method syntax that all of you seem to find so obvious. Given the popularity of django, it speaks volume to what i've been telling you for so long now. class.method syntax is *not* a solution, since it's very unlikely someone in need will find it. So there's very much a need to be covered here.

Don't simply dismiss the concern; answer it by showing the need that justifies it. Hope it answers that remark. I'll still be working on that survey... it's taking me forever Will you have a read at the question before i run it, so you can agree before hand the questions are fair to you?

--- Eric V. Smith writes:

I'm trying to save you some time here, but if you're committed to continuing this, then you can't say you weren't warned when it's ultimately rejected I'm aware of this, thanks for the warning nonetheless.

My suggestion is to rework your proposal to not break any existing code I've been doing that already, and i'll keep doing it, i'm very aware of this constraint.

What are the biggest breaking change i haven't covered yet, in your opinion? If you can think of a few that seem way too bad please let me know.

On 4/13/2022 7:22 AM, malmiteria wrote:

Eric V. Smith writes:

...

My suggestion is to rework your proposal to not break any existing code I've been doing that already, and i'll keep doing it, i'm very aware of this constraint.

What are the biggest breaking change i haven't covered yet, in your opinion? If you can think of a few that seem way too bad please let me know.

I'm not tracking your exact proposal very closely. The problem would be any code that uses existing syntax and raises the exception you're proposing when an attribute is defined in multiple base classes. I can't find the exact name you're proposing. I don't think we need to find existing code that meets this criteria. It doesn't matter if it requires diamond inheritance, non-diamond inheritance, or whatever. If the exception can possibly be raised in code that's valid in any currently supported python version, then you have a backward compatibility problem. Eric

On 4/12/2022 12:57 PM, malmiteria wrote:

So the amount of actual breakings left shouldn't be enough to justify denying this feature IMO. Again, we'll have a clearer view on that once we get experimental data. What's to look for is usage of super in MI cases, since each super call in MI today should be replaced by one super call per parent, in my proposal. If this turns out to be an issue, we can try to imagine solution for it. Maybe a solution such as : super calls implicitely run a super call to each parent when there's multiple parent, and super isn't given arguments. This i think would behave strictly as today's super + MRO, except for cases where one class appears multiple time in an inheritance tree. The last breaking change would be that scenario, class appearing multiple time in an inheritance tree. And we're getting on even rarer occasions here, but again, we can think of solutions here.

I have code in a non-public repository that your proposed change will break. I'm sure I'm not the only one. It is not realistic to implement a change like this that will break code and then say "there's a workaround that you'll need to implement". Especially for library code that needs to run across multiple python versions.

As much as i understand we don't want breaking change, i don't think it is *that* strong an argument, in this case. And, on its own, breaking changes aren't a strict veto, there's been some, and there's gonna be more.

As a long-time core developer, I can assure you that this is one of those cases where a breaking change will not be allowed. I'm trying to save you some time here, but if you're committed to continuing this, then you can't say you weren't warned when it's ultimately rejected. My suggestion is to rework your proposal to not break any existing code. It's a constraint we've all had to live with at one time or another, as much as we don't like it. Eric

On Tue, Apr 12, 2022 at 11:17:27AM -0000, malmiteria wrote:

Steven D'Aprano writes:

...

So in the general case, order matters. We have to linearize the superclasses, and call them in that linear order, and the best way to do that is with the MRO and super.

Why would we *have* to do that?

Because if you don't, you aren't doing multiple inheritence. If you don't want MI, if you want delegation or composition, then of course you can do something else. But if you want MI in its full generality, that's really the only thing you can do.

When multiple parent provide candidate to a method resolution, raise an error.

Then you aren't doing full MI any more, just a limited, restricted version known as traits.

The child class now has to redefine the method, and in the body of that method, can decide which parent method call, or in what order call them. That's essentially the basic idea of my proposal. What makes this impossible for you?

Its not impossible, I have been telling you about traits since my earliest posts in this thread. Maybe you need to stop thinking you have invented some brilliant idea which has solved a problem nobody else has noticed, and read some of the Artima links I have days ago.

I think i've adressed most if not all problems i had raised against it.

Except the most important one: why are you using multiple inheritence when you clearly don't want multiple inheritence, instead you want delegation?

Linearisation is litterally an operation that consist into converting a multiple inheritance tree into a simple inheritance tree.

That is absolutely not what linearization does.

That's bound to lose some info,

No, the point of linearization is to avoid losing info.

and can't cover all cases.

Correct, there are inconsistent inheritence hierarchy which are impossible to linearise because they are inconsistent. It is better to get an error when you try to define the inconsistent hierarchy, rather than pretend it is consistent and then wonder why inheritence is not working and methods are called in inconsistent order. -- Steve

On 13/04/22 8:29 am, Steven D'Aprano wrote:

...

When multiple parent provide candidate to a method resolution, raise an error.

Then you aren't doing full MI any more,

That sounds like a "true Scotsman" argument. Who defines what "full MI" means? I can think of at least two languages that do something very similar to what malmalitia is proposing: C++ and (if I remember rightly) Eiffel. I don't think I've heard anyone claim that C++ doesn't do "full MI". Here's a C++ example: #include <stdio.h> class A { public: void f() { printf("f in A\n"); } }; class B { public: void f() { printf("f in B\n"); } }; class AB: virtual A, virtual B { }; int main() { AB ab; ab.f(); } and compiling it gives this: mi.cpp:22:6: error: member 'f' found in multiple base classes of different types ab.f(); ^ mi.cpp:5:10: note: member found by ambiguous name lookup void f() { ^ mi.cpp:12:10: note: member found by ambiguous name lookup void f() { ^ 1 error generated. -- Greg just a limited, restricted

version known as traits.

...

The child class now has to redefine the method, and in the body of that method, can decide which parent method call, or in what order call them. That's essentially the basic idea of my proposal. What makes this impossible for you?

Its not impossible, I have been telling you about traits since my earliest posts in this thread.

Maybe you need to stop thinking you have invented some brilliant idea which has solved a problem nobody else has noticed, and read some of the Artima links I have days ago.

...

I think i've adressed most if not all problems i had raised against it.

Except the most important one: why are you using multiple inheritence when you clearly don't want multiple inheritence, instead you want delegation?

...

Linearisation is litterally an operation that consist into converting a multiple inheritance tree into a simple inheritance tree.

That is absolutely not what linearization does.

...

That's bound to lose some info,

No, the point of linearization is to avoid losing info.

...

and can't cover all cases.

Correct, there are inconsistent inheritence hierarchy which are impossible to linearise because they are inconsistent. It is better to get an error when you try to define the inconsistent hierarchy, rather than pretend it is consistent and then wonder why inheritence is not working and methods are called in inconsistent order.

Greg Ewing writes:

That sounds like a "true Scotsman" argument. Who defines what "full MI" means? +1

for the record, i consider it multiple inheritance as soon as one class can inherit from multiple parents, no matter anything else.

I can think of at least two languages that do something very similar to what malmalitia is proposing malmiteria xD not malmalitia

--- David Mertz, Ph.D. writes:

I also co-authored a widely read article on then-new C3 linearization in Python with Michele Simianato before he wrote his really great explanation of it in the Python docs, and his also great 15+ year old pieces on traits you mean this one? : https://www.python.org/download/releases/2.3/mro/

--- Steven d'Aprano writes:

Its not impossible, I have been telling you about traits since my earliest posts in this thread. And traits aren't what i want. My proposal is very different from what i understand of trait so far. Although it's still new to me, so if you feel i'm missing something, feel free to explain it to me.

...

Linearisation is litterally an operation that consist into converting a multiple inheritance tree into a simple inheritance tree.

That is absolutely not what linearization does.

That's very much what it does, implicitely tho. Take the mro from any class. create a copy of the first class, and assign it only one parent. This parent is the copy of the next in mro order, which too takes only one parent. until we reach the last one in mro order, which is object. This 100% simple inheritance tree behave as far as i can tell 100% like the original class. No matter if the original class was multiple inheritance or not. Resolution order is the exact same, super visiting order is the exact same... class.method calls are independant, so they wouldn't change either. this can't be done for every class very nicely, as editing __bases__ attribute sometime is not permitted. so you'll excuse me for not providing a nice bit of code that does the convertion in the inheritance tree for you.

Greg Ewing writes:

...

malmiteria xD not malmalitia Sorry! Should have gone to Specsavers... No worries :)

On Fri, 15 Apr 2022 at 20:40, Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, Apr 14, 2022 at 10:46:46AM +1200, Greg Ewing wrote:

...

On 13/04/22 8:29 am, Steven D'Aprano wrote:

...

...

When multiple parent provide candidate to a method resolution, raise an error.

Then you aren't doing full MI any more,

That sounds like a "true Scotsman" argument. Who defines what "full MI" means?

If you look at languages that implement MI, and pick the implementations which allow it with the fewest restrictions, then that is "full MI".

I'm with Greg on this one, for the simple reason that a future language could have fewer restrictions than Python does, and therefore would become the only thing that offers "full MI", displacing other languages. It's a meaningless concept, unless there is some form of absolute completeness that can be attained (and if you go for that, NO language offers "full MI"). My view: If a class inherits two parents, in any meaning of the word "inherits" that follows the normal expectation that a subclass IS an instance of its parent class(es), then it's MI. So if you have "class Foo(Spam, Ham):" and it is reasonable to treat a Foo instance as if it were a Spam instance *and* reasonable to treat a Foo instance as if it were a Ham instance, then it's MI. Erroring out when there is a conflict is a restriction, but I would avoid the term "full MI" because it's more emotive than useful. ChrisA

On Fri, Apr 15, 2022 at 2:44 PM malmiteria <martin.milon@ensc.fr> wrote:

I got an idea that *should* allow for some (keyword : some) of the changes i want without any breaks, i kinda wanna take the time to think about it, and once i'm a bit more sure of it, i'll talk about it in details.

Since you are thinking of ways that won't break current code, you might as well think of ways that won't need any syntax modification/adding extra features to the language. The current capabilities we get, including being able to customize the metaclass __getattribute__ method, might allow for that - and you'd have the advantage that your ideas could be immediately be made available in a pypi package.

On Fri, Apr 15, 2022 at 11:12:26PM +1000, Chris Angelico wrote:

...

If you look at languages that implement MI, and pick the implementations which allow it with the fewest restrictions, then that is "full MI".

I'm with Greg on this one, for the simple reason that a future language could have fewer restrictions than Python does,

Not without making inheritance incoherent and inconsistent. E.g. in Python 2.2 and below, it was possible to design class hierarchies that resulted in methods being called twice or more times. Other signs of an incoherent MI model might include: * loops or cycles in your precedence graph; * superclasses being skipped; * inconsistent ordering (class A coming before class B for one method, but B before A for another method); * violating local precedence order, e.g. class Spam has A come before B, but in Spam's subclass the order flips to B before A; * or the order changes if you change the name of a class and nothing else about it. These are all Bad Things™ and Python avoids them all. You can only generalise MI up to a certain point, after which it becomes inconsistent and therefore buggy. That's why Python 2.2's MRO was dropped for the C3 linearization in 2.3: it was buggy. I'm assuming automatic conflict resolution. If its not automatic, it's not what I'm calling inheritance. If you have to manually specify which superclass to call, that's delegation. That's okay: inheritance is not the be all and end all of OOP. You can use delegation to solve problems too, or manual conflict resolution by renaming methods, as Eiffel does. And as I said before, it might be that the Eiffel or C++ model is *better* than Python's model. But given the assumptions that: - the inheritance model automatically resolves conflicts; - the MRO is entirely dependendent on the shape of the inheritance graph, and not on incidental properties like the name of classes; - the inheritance model is consistent, monotonic and preserves local precedence order (C3 linearization); then I believe that the Dylan/Python/Ruby/Perl/Raku MI model is as general as you can get, and models like Eiffel or C++ are more restrictive. I believe that you cannot drop any of those restrictions without the very idea of inheritance becoming incoherent. It is not a "No True Scotsman" fallacy. Other models of MI are legitimate even if they are not fully general. I have suggested that Python's fully generalised MI model may ultimately be worse than more restrictive models that provide less freedom to write unmaintainable code. I have repeatedly linked to the writing of Michele Simionato who explores these issues in excruciating detail.

and therefore would become the only thing that offers "full MI", displacing other languages. It's a meaningless concept, unless there is some form of absolute completeness that can be attained

Well duh Chris, sometimes I wonder if you read my posts before jumping in to disagree with me, that is *exactly* what I am arguing. If you exclude models of MI which are logically incoherent and inconsistent, (such as Python's prior to version 2.3), then Python's model of MI is objectively as complete as you can get. Whether I am right or wrong, this is an objective matter of fact, not a matter of taste or opinion, and it is certainly not a "No True Scotsman" value judgement against using more restrictive forms of MI.

My view: If a class inherits two parents, in any meaning of the word "inherits" that follows the normal expectation that a subclass IS an instance of its parent class(es), then it's MI.

Inheritance and "is-a" relationships are independent. In some languages (but not Python), mixins provide inheritance but not an "is-a" relationship. In Python, virtual subclassing provides "is-a" without inheritance. -- Steve

On Sat, 16 Apr 2022 at 11:07, Chris Angelico <rosuav@gmail.com> wrote:

On Sat, 16 Apr 2022 at 11:00, Steven D'Aprano <steve@pearwood.info> wrote:

...

...

and therefore would become the only thing that offers "full MI", displacing other languages. It's a meaningless concept, unless there is some form of absolute completeness that can be attained

Well duh Chris, sometimes I wonder if you read my posts before jumping in to disagree with me, that is *exactly* what I am arguing.

You placed a LOT of caveats on it. I don't count that as "absolute completeness". It is the most complete that YOU, right now, think could ever be possible.

For a slightly tangential comparison: If you assume that "numbers" are only those on the real number line, then you assume that returning an error when asking for the square root of a negative number is the ONLY way to do things, and a mathematical library that can handle all real numbers is absolutely complete. But based on your knowledge of Python's numeric tower, I think you'd agree that this view, despite having been firmly held for centuries, isn't actually complete. ChrisA

On Sat, 16 Apr 2022 at 14:25, Steven D'Aprano <steve@pearwood.info> wrote:

On Sat, Apr 16, 2022 at 11:07:00AM +1000, Chris Angelico wrote:

...

On Sat, 16 Apr 2022 at 11:00, Steven D'Aprano <steve@pearwood.info> wrote:

...

...

and therefore would become the only thing that offers "full MI", displacing other languages. It's a meaningless concept, unless there is some form of absolute completeness that can be attained

Well duh Chris, sometimes I wonder if you read my posts before jumping in to disagree with me, that is *exactly* what I am arguing.

You placed a LOT of caveats on it. I don't count that as "absolute completeness". It is the most complete that YOU, right now, think could ever be possible.

Which conditions would you drop? There's not that many, really. Five. Six if you include the "no cycles" requirement for the DAG, which I think is so obviously necessary that it is barely worth mentioning.

This is *exactly* the "no true Scotsman" fallacy: you have already excluded from consideration anything that drops a condition you didn't already drop. On the assumption that your five conditions are essential, there's no way that you can drop any of the five conditions and still have it count, therefore the five conditions are essential. Your logic is circular. It is highly arrogant to assume that nobody will ever find a way to implement MI while dropping one of your conditions. They're not fundamental to the definition, they're fundamental to *the way Python does things*.

The most subjective is the requirement for automatic conflict resolution. It is a legitimate design choice to give up automatic conflict resolution (that's what C++ and Eiffel do) but that would be a breaking change for Python.

Yes. A breaking change FOR PYTHON.

So come on Chris, back up your disagreement with something objective, not just wishy-washy "anything might happen in the future!" nonsense.

Yet you're willing to argue that other languages don't do "full MI" because they do things that would be a breaking change for Python?

No, not everything is possible. We're never going to discover a new odd number between 3 and 5, or that 7 isn't really prime, or that cats are actually a type of plant, or that Australia doesn't exist.

You would be very surprised what people HAVE discovered. For instance, it's very common to define the distance between two numbers by subtracting one from the other, but that isn't the only internally-consistent definition of distance that could be used. The only things that we can completely rule out are those which are true by definition, or can be proven mathematically or logically. A new odd number between 3 and 5 is provably impossible. 7 is truly prime, by the definition of primes, and any extension to that definition (eg complex primes or Gaussian integers) must maintain that. Discovering that Australia doesn't exist would be world news, but might indicate that we'd finally angered some nuclear country enough to get ourselves completely blown off the map. (Yeah, I know countries like Russia and the US don't have enough nukes to do that, but we have no idea what sort of arsenal Ghandi is hiding, just waiting for his stats to become negative...)

So be concrete: which of my preconditions do you want to challenge?

- The inheritance model automatically resolves conflicts.

As I said, it is a legitimate design choice to give that up, but it would be a breaking change for Python so we can rule it out.

In any case, languages without automatic conflict resolution do less than languages with them. (That might be a good thing.)

This is the essence of the "no true Scotsman" fallacy: you assume that it's not true MI without automatic conflict resolution.

- The MRO is entirely dependendent on the shape of the inheritance graph, and not on incidental properties like the name of classes.

Let's hear your justification for why breaking that condition is good.

"I changed my class name from Spam to Eggs, and suddenly the inheritance relationships between my classes changed."

"That's not a bug, that's a feature!!!"

Is it not true MI if the relationships change? Is that what you're saying? What is "full MI"?

- the inheritance model is consistent, monotonic and preserves local precedence order (C3 linearization).

Which of those three will you give up, and why is that a good thing?

"In my class Spam, superclass A takes precedence over B, but when I subclass Spam, the precedence swaps and B comes before A."

"That's not a bug, that's a feature!!!"

You keep asserting that, because something OBVIOUSLY would be a bad thing for Python, it must not be "true multiple inheritance". That's why you're being called out for the NTS fallacy. Your entire definition of "full MI" is "Python 2.3+".

...

...

If you exclude models of MI which are logically incoherent and inconsistent, (such as Python's prior to version 2.3), then Python's model of MI is objectively as complete as you can get.

If you assume that what we know in 2022 is the most we will ever know, then yes, you would be correct. Do you really think that nobody will ever learn anything new about ways of doing MI?

Yes. Its a DAG of superclass/subclass relationships. There is only one way to draw that graph that is coherent.

Can you mathematically prove that? What if there is some other way to linearize that is *also* consistent with itself, but different from C3? Is there some way to prove that this is impossible? Also, what if there is no linearization as such - what if, when you call the superclass, it actually calls ALL the parents, not just one?

...

I don't know whether you're mistaken or utterly arrogant.

Do you think they are the only two choices? Are you such a contrarian that you refuse to even consider that I might be right?

You see the problem of the excluded middle now. Can you recognize when you're making it yourself? ChrisA

On Sat, 16 Apr 2022 at 20:28, Steven D'Aprano <steve@pearwood.info> wrote:

So please stop falsely accusing me of "No True Scotsman" bullshit. If you keep doing it, I will know you're not arguing in good faith.

Fine. I'll accuse you, instead, of using "full MI" to mean "the best MI system that Steven D'Aprano can conceive". Happy now? Because that is precisely what you are doing. Any argument to the contrary is met with complaints that we need to show that something is *better* for *Python*. For something to be "full MI", in your view, it has to match what Python does, or be objectively better than it, in your personal opinion. I'm not going to bother responding to the rest of the details, as they're all the same argument repeated. ChrisA

On Sun, Apr 17, 2022 at 07:39:29PM +1200, Greg Ewing wrote:

On 16/04/22 10:26 pm, Steven D'Aprano wrote:

...

C++ and Eiffel are even stricter (more restrictive) than Python. They don't just exclude class hierarchies which are inconsistent, they exclude class hierarchies with perfectly good linearizations because they have a method conflict.

No, they don't *exclude* such hierarchies, they just require you to resolve the conflicts explicitly.

Okay, fair comment. Can we agree on this then? - C++ allows hierarchies with method conflicts so long as you do not implicitly inherit from those methods. (You must explicitly call the superclasses you want.) - Eiffel allows hierarchies with method conflicts so long as you remove the conflict by renaming the methods.

...

no matter how many times I say that other choices for MI are legitimate and maybe even better than Python's choice

So by saying that something is "not full MI", you didn't mean to imply that it is somehow inferior and less desirable?

Well done! I'm glad we're making progress. https://www.youtube.com/watch?v=Cl2pvI1nQVU How many times did I suggest that the C++ or Eiffel approach might be better than Python's approach? How many times did I mention traits, or link to Michele Simionato's blog? I linked to James Knight's (misnamed) post "super considered harmful" at least twice. It is certainly worth reading to understand some of the problems with Python's MI. If I say that Python's model of MI is more general, I mean *more general*. Its not a dog-whistle for Python-supremacists. It just means that Python handles more cases than Eiffel or C++ it doesn't imply that those languages are "inferior". For what its worth, I think that fully general MI does fit into what Paul Graham calls the "Blub Paradox". It's *more powerful* -- but it might be *too powerful* to use effectively, like unrestrained GOTO capable of jumping into the middle of functions, or Ruby's ability to monkeypatch everything including builtins. If all I want to do is drive to the local corner store and buy milk, a rocket-car that does Mach 3 in a straight line and burns 2000 gallons of fuel a minute is more powerful, but not as useful as a 30 year old Toyota. Sometimes less is more. If you're having problems with MI maybe what you need is *less of it* not more of it, and perhaps that means getting the compiler to warn you when you've attached a JATO rocket to your Toyota sedan, which is what C++ and Eiffel and Squeak traits do in different ways.

Because that's what you sounded like you were saying, and why everyone is pushing back so hard on it.

Yeah Greg, you got me, you saw through my cunning plan to denigrate C++ and Eiffel by saying that they are better than Python.

...

The requirement for automatic conflict resolution is kinda necessary for it to be inheritance

You seem to have a very different idea of what "inheritance" means from everyone else here.

Well, I don't know about "everybody else", but okay. I think we agree that when we do this: instance.method() the interpreter automatically searches for some class in a tree of superclasses where the method is defined, and we call that inheritance. I think that the distinguishing feature here is that the interpreter does it *automatically*. If we had to manually inspect the tree of superclasses and explicitly walk it, it wouldn't be inheritance: # Pretend that classes have a parent() method and we only # have single inheritance. instance.__class__.parent().parent().parent().method(instance) which is just another way of explicitly calling a method on a class: GreatGrandParentClass.method(instance) So we're not inheriting anything there, we're just calling a function: some_namespace.function(instance) And if you're just calling a function, then it really doesn't matter whether GreatGrandParentClass is in the instance's MRO or not. If the MRO isn't used, then why even have an MRO? What else could distinguish inheritance from just "call some function", if it isn't that the interpreter works out where the function is defined for you, using some automatic MRO? (That's not a rhetorical question. If you have some other distinguishing characteristic in mind, apart from automatically searching the MRO, then please do speak up, I would love to hear it.) Take that automatic MRO search away, and you're not doing inheritance, you're just calling a function in a namespace. **Which is fine.** That's the basis of composition and delegation. That's not worse than MI. Its better. (But note that this doesn't distinguish between inheritance and generics, which also has a form of automatic delegation. Oh well.) -- Steve

I don't really care what terms like "Full MI" or "True MI" are intended to mean. Python and MANY other programming languages that allow classes to have multiple parents, use a method resolution order (MRO) to decide which same-named method from a complex inheritance tree to use in a given call. If languages use an MRO, C3 is probably the best choice. But not the only one. But sure, C++ and Eiffel are free to refuse "ambiguity" of names occurring multiple times in an inheritance tree. Other languages are free to use single inheritance, or traits, or mixins. I wrote an article a lot of years ago, around the time C3 was new to Python, intending to show what a banal thing OOP is in which I added inheritance and an MRO to R (all within R itself). It was something like 20-30 lines of code in total. My MRO was something dumb like depth first or breadth first, but it WAS an MRO. R doesn't have inheritance, it's not OOP, which was my main reason to choose it for the exercise. One thing I do find a bête noire is the silly claim, that Chris repeats, that inheritance expresses "Is-A" relationships. It was torture in the 1990s hearing about vehicles and trucks and sedans as if that had something to do with programming. I never "got" OOP for at least 5 years later than I should have because of those horrible Linnaean metaphors. The reality is more clear in the actual primary definition of the word itself: "the practice of receiving private property, titles, debts, entitlements, privileges, rights, and obligations". In programming, a class can receive methods and attributes from some other classes. That's all. It's just a convenience of code organization, nothing ontological.

On Sat, Apr 16, 2022, 12:26 PM Chris Angelico > wrote:

A button IS a widget. A horizontal box IS a box. A box IS a container. A container IS a widget.

Class hierarchies in graphical systems are just as much based on those is-a relationships as any of those "horrible metaphors" you may have come across, and they are absolutely to do with programming. Just because you had a bad teacher, that doesn't mean the concepts are wrong.

Well, more like a few dozen bad books. I never studied any of this with a teacher. But when I teach it (to quite a lot of programming professionals), I never once mention that IS-A stuff, other than maybe curses sotto voce. And mostly they come away saying they finally understood it after my presentation.

R has changed a bit since 2005. My article was from then: https://gnosis.cx/publish/programming/R3.html I'm not trying to quibble about R, but simple to point out that what often mystified as "deep" about OOP is actually banal and fairly trivial. That might be how Alan Kay originally saw OOP. He famously regretted using

the term "object" because it distracted from what he saw as the genuinely fundamental parts of OOP, namely

* Message passing * Encapsulation * Late (dynamic) binding

I like Alan. I worked with him a little bit. I agree with his attitude here.

On Sat, 16 Apr 2022 at 14:33, Steven D'Aprano <steve@pearwood.info> wrote:

On Sat, Apr 16, 2022 at 11:07:00AM +1000, Chris Angelico wrote:

...

...

...

My view: If a class inherits two parents, in any meaning of the word "inherits" that follows the normal expectation that a subclass IS an instance of its parent class(es), then it's MI.

Inheritance and "is-a" relationships are independent.

In some languages (but not Python), mixins provide inheritance but not an "is-a" relationship. In Python, virtual subclassing provides "is-a" without inheritance.

Virtual subclassing is still subclassing, just implemented differently.

You are correct that virtual subclassing is still subclassing, but it doesn't provide inheritance.

...

...

...

from abc import ABC class Parrot(ABC): ... def speak(self): ... print("Polly wants a cracker!") ... @Parrot.register ... class Norwegian_Blue: ... pass ... bird = Norwegian_Blue() isinstance(bird, Parrot) True bird.speak() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'Norwegian_Blue' object has no attribute 'speak'

Yes. I would have to call this an abuse of a feature - you're registering that a Norwegian Blue is a Parrot, without it being able to do anything that a parrot should be able to. (Which I presume was your intention, given your choice of examples.) You can claim that it's a parrot all you like, and Python will happily reflect your declaration when you ask if it's a parrot, but it's a pathological case. It's like designing a class with no __str__ method, or one where __add__ takes three parameters, or in any other way violates normal expectations; you can use perfectly normal inheritance to derive from object, but then you make changes so it no longer "behaves-like-a", ie it violates Liskov.

...

What is "inheritance" if it isn't that is-a relationship? How do you distinguish inheritance from delegation?

Haven't you spent all of this thread quite happily distinguishing between using inheritance and composition/delegation until now? Okay.

Yes. I am asking you to now define the difference. My understanding was that inheritance was defined by the "is-a" relationship, and delegation was defined by other things. Now that you're claiming that inheritance is NOT defined by an is-a, I want you to define it.

Subclassing is, as you say, an "is-a" relationship. Whether you use virtual subclassing or actual subclassing, if you can say:

issubclass(Norwegian_Blue, Parrot) isinstance(Norwegian_Blue(), Parrot)

and get True for both of them, then Norwegian_Blue "is-a" Parrot.

(Note that there is a technical difference between subclass and subtype, which I don't think is relevent to Python, but let's not go there.)

Inheritance is a mechanism where a "child object" automatically acquires the properties and behaviors of some "parent object". In Python, that is handled by the interpreter when the child subclasses from the parent (but not in virtual subclassing).

The critical thing is that in the absense of overloading or overriding, calling Norwegian_Blue().speak() will inherit the method defined in Parrot. We get that inheritance from real, but not virtual, subclassing.

Is it still inheritance if there needs to be a boatload of code to make this happen? Because SOM/CORBA and the IDL want to have a word with you. In a good high-level language, there should normally be a convenient way to do this without too much effort. But is that really essential to it being inheritance? As another example: In C++, a function accepting a pointer to a base class can be given a parameter that points to a subclass. In the case of multiple inheritance, this can actually mean that the pointer has to change. So, two questions: (1) Is the fact that a function, not defined outside in the class at all, accepts a pointer, part of the behaviour of the class? And (2) If the compiler has to know "to turn a pointer-to-X into a pointer-to-Y, add an offset of 16", is it still retaining functionality? You make a *lot* of assumptions based on Python, which simply don't hold up in other languages, and then you assert that this is some kind of absolute completeness.

Composition provides a "has-a" relationship. For the sake of brevity, in simple terms (which may not be completely accurate, please don't nit- pick just for the sake of nit-picking), we use composition when we write:

class Car: def __init__(self): self.engine = Engine()

In this example, Cars are not Engines, but they have an Engine. Hence composition.

Delegation provides a mechanism of code-sharing separate from inheritance, and often used instead of inheritance, or to compliment it. One object delegates to another object if the first explicitly calls the second:

# self is a Car instance. self.engine.start()

Delegation and composition often go together, but they don't necessarily have to. One can delegate to an object "outside" of the class, although that technique is not often used in Python.

But broadly speaking, if your instance uses:

super().method()

that's an inheritance call. (It is only needed when you overload method.) If you write something like:

SomeClass.method(self)

that's delegation.

Hmm. I think that's WAY too restrictive a definition, and based on that, a huge number of C++ classes simply don't use inheritance. Or is this kind of restriction specific to Python in some way? Please, can you define inheritance in a way that *doesn't* assume Python 2.3+?

Objects can delegate to anything they like, but if they delegate to a superclass, we might call it inheritance even though it lacks the property of being automatically handled by the interpreter. So one might loosely say that inheritance is a special case of delegation.

Ehh, okay. If that's the case, then sure, but we still need a way to define inheritance that doesn't depend on Python-specific concepts.

Note that in languages without any form of super() or "next_class" or whatever you call it, that sort of manual delegation may be the only way to overload an inherited method.

...

Is inheritance only a thing if it happens on the line of code that says "class X"? (Not the case in Pike.)

I don't think the syntax is important. You could say:

@inherits_from(Parrot) class Norwegian_Blue: pass

if you prefer. Defining the decorator is left as an exercise.

I can't comment on Pike, since you haven't described how it behaves.

It's a directive inside the class block: class Norwegian_Blue { inherit Parrot; // ... } And if you do it more than once, you get multiple inheritance. The rules aren't the same as in Python, but there's a way to call *every* parent method, which is extremely convenient for constructors. It most definitely is an "is-a" relationship (for instance, an object of type Norwegian_Blue can be passed to a function that expects a Parrot), but it's also able to model some other patterns too, so pigeonholing things is hard.

As I said, in some languages mixins define inheritance without subclassing; in Python virtual subclasses define subclassing without inheritance.

...

Is inheritance only a thing if it happens as the class is first created? (Is the case with mixins.)

I don't understand what you mean here. Do you mean that mixins inject their methods into the class at creation time, and then no longer are referenced? That's not what happens in Python. Mixins use the same method resolution mechanism as other superclasses, which means it happens on demand, not at creation time.

...

...

...

class Mixin: ... def method(self): ... print("Creation time") ... class Spam(Mixin): ... pass ... Mixin.method = lambda self: print("Call time") Spam().method() Call time

I'm trying to figure out what you mean by inheritance. You said that mixins aren't inheritance. Or maybe I misread you? It was rather confusing. ChrisA

On Sat, Apr 16, 2022 at 05:56:13PM +1200, Greg Ewing wrote:

On 16/04/22 12:56 pm, Steven D'Aprano wrote:

...

If you exclude models of MI which are logically incoherent and inconsistent, (such as Python's prior to version 2.3), then Python's model of MI is objectively as complete as you can get.

You seem to be assuming that "full MI" has to include a fully automatic means of method resolution.

How very observant of you, I only mentioned that about a hundred bazillion times so far!!! (Yes, I am frustrated. But cheerfully so.)

There's nothing incoherent or inconsistent about the way C++ and Eiffel do MI.

Good thing I never said that they were incoherent or inconsistent. I'm happy to have vigorous debate, but it would be kinda nice if people would argue against my actual position. As I understand it, C++ and Eiffel still do automatic method resolution, it is only when there is a conflict that they demand you manually resolve the conflict.

The main difference is that they require you to explicitly resolve conflicts between inherited methods --

Right, another thing which I have mentioned. In that regard, they are sort of like traits, as developed in Squeak. (Scala traits are something different. Another example of different languages using the same term to mean different things.)

which is arguably more Pythonic than Python, since they refuse the temptation to guess.

There is no *guessing* in the C3 linearization algorithm. -- Steve