I wholeheartedly agree with Chris on this one. This isn't broken, nor is it something we want to change. If you're looking for a case where object is not the last of its hierarchy, Python 2 has classic classes, but they are remaining artefacts there. The MRO needs to be consistent across all classes along the MRO, with object last.

Date: Fri, 11 Dec 2015 04:29:05 +1100 From: rosuav@gmail.com CC: python-ideas@python.org Subject: Re: [Python-ideas] MRO local precedence ordering revisited

On Fri, Dec 11, 2015 at 3:36 AM, Stephan Sahm wrote:

...

As reference I take https://www.python.org/download/releases/2.3/mro/ There the section "Bad Method Resolution Orders" starts with an example which behaviour in the current python 2.7 is rather unintuitive to me.

Just to make something quite clear: "current" Python, as discussed on this list, is 3.5 or 3.6, not 2.7. There won't be a 2.8, and future releases of 2.7.x won't be changing anything like this. What you're describing is also the case in current Python 3.x, but any change to how the MRO is derived would happen in 3.6 at the very soonest. (At least, I don't _think_ the current behaviour could be considered a bug. I could be wrong.)

...

I rename the example slightly to illustrate its usecase.

...

class Mixin(object): pass

class A(Mixin): pass class B(Mixin, A): pass

this unfortunately throws "TypeError: Error when calling the metaclass bases Cannot create a consistent method resolution order (MRO) for bases A, Mixin"

The reference name above comments this case similar to the following (adapted to the easier example above):

[quoting the docs]

...

As a general rule, hierarchies such as the previous one should be avoided, since it is unclear if Mixin should override A or viceversa

While it might be the case that in Python 2.2 things where different, I cannot agree that the expected order of Method resolution is ambiguous (at least as far I see at this stage).

Therefore I would like to propose to make this MRO again a valid one.

The usecase should be obvious: If you want a Mixin to be the first thing overwriting functions, but still want to inherit as normal. (I myself want for instance a Mixin to overwrite the __init__ method, which is simply not impossible when choosing class B(A, Mixin))

What you're proposing, in effect, is that it should be possible for super() to go *down* the class hierarchy. Under your proposal, B.__mro__ would be (Mixin,A,object), which means that super().method() inside A will go directly to object, and that the same call inside Mixin will go to A - reversing the order of the calls compared to the way they would be if the object were of type A. Now it is known that the super doesn't mean "call my base class", but something more like "call the next class"; but it's currently guaranteed that the MRO for any derived class is a *merge* of the MROs of its parents, without ever reordering them. Under your rules, what would be the MRO here?

class A(object): # explicitly stating the default def __new__(cls): print("Constructing an A") return super().__new__(cls) def method(self): print("Calling method on A") super().method()

class B(object, A): pass

If you answered (B,object,A), then you've just made it possible to have some other class than object at the end of the chain. Where do super() calls go if there *is no next class*? What should happen? Can that be made equally intuitive to your proposal about mixins?

ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Dec 11, 2015 at 7:03 AM, Stephan Sahm wrote:

I am in fact not familiar with what precisely are all the problems the C3 algorithm tries to solve, however my own intuitiv MRO would not be anything proposed so far, but:

(B, mixin, A, mixin, object)

or in the simpler, more trivial version

(B, object, A, object)

As you haven't mentioned this as a possibility at all, I guess having a class twice in this list produces some weird behaviour I do not know about yet - if someone can point out, that would be great.

In terms of people's expectations of super(), I think it would be intolerably surprising for it to be able to call a method from *the same* class. If your mixin has "def method(self): super().method()", it'll call itself (once) if called from B.

P.S.: I haven't said that I want a python 2.8, nevertheless thanks for pointing out that there won't be a python 2.8 (this I in fact have not known before)! I am familar with python 3.5 as well, however I just wanted to make clear where I am working and in which language the example code is runnable.

https://www.python.org/dev/peps/pep-0404/ http://blog.startifact.com/guido_no.jpg There are some special exemptions in 2.7 (for instance, some changes went through recently that improve security, even at the expense of some backward compatibility), but the bar is extremely high; behaviour has to either be recognizably buggy, or have significant security implications (cf hash randomization and certificate checking), to be changed in 2.7. I don't think the MRO fits either category, so any change would be 3.x only. As mentioned, though, your code works on 3.5/3.6 without changes and with the same semantics. ChrisA

@Chris thanks for pointing out this self-reference, I am still not sure whether this is really suprising to me, but at least I am still not done with thinking about it, so probably it is @Ned As I understood it, the MRO is not only for searching attributes - there it is indeed impressively redundant to put the same class twice into the MRO, thanks for pointing that out - but also for the hierarchy of the super() command @all thank you all for your comments and help. My current conclusion is that I will read about the C3 algorithm in crucially more detail and what it in fact is trying to solve ... and eventually may come back best, Stephan On 10 December 2015 at 21:43, Ned Batchelder wrote:

On 12/10/15 3:03 PM, Stephan Sahm wrote:

(B, mixin, A, mixin, object)

or in the simpler, more trivial version

(B, object, A, object)

As you haven't mentioned this as a possibility at all, I guess having a class twice in this list produces some weird behaviour I do not know about yet - if someone can point out, that would be great.

The MRO is a list of classes to search for attributes. There's no point in having a class listed twice. The second occurrence would never be used, because any attribute it could provide would be found on the first occurrence of the class.

--Ned.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Dec 10, 2015, at 12:03, Stephan Sahm wrote:

Dear all,

this is a highly sensitive topic, isn't it?

Well, it's an _interesting_ topic. If there really is something new to be done here that could improve the way inheritance works, that's a big deal, so people want to think about it, which means asking you challenging questions.

=) Thank you all for your extensive responses. Of course I was a bit provocative with the sentence in bold, but I haven't said this would be a bug, and if I have, I am sorry for that.

I am in fact not familiar with what precisely are all the problems the C3 algorithm tries to solve, however my own intuitiv MRO would not be anything proposed so far, but:

(B, mixin, A, mixin, object)

or in the simpler, more trivial version

(B, object, A, object)

First, think about what kind of algorithm could get mixin to appear twice in the first list without also getting object to appear twice. I don't think there's any obvious way to do it. (Well, you could special-case object, but that doesn't help the more general diamond problem.) If you have a non-obvious answer to that, even though that still might not be a complete solution to what you really wanted, it would be a major contribution on its own.

As you haven't mentioned this as a possibility at all, I guess having a class twice in this list produces some weird behaviour I do not know about yet - if someone can point out, that would be great.

For an obvious problem: how could any cooperative inheritance tree ever super anything if object, which obviously doesn't cooperate, might end up on the MRO between one class and the next? Plus, if a class can get called twice on the super chain, how does it know which of the two times it's being called? Imagine a trivial mixin that just counts how many instances of its descendant get created. How do you avoid counting twice when a C gets constructed. (If you're thinking of using __mangled names to help, notice that _mixin__mangled and _mixin__mangled are the same name.) More generally: the code in a class generally assumes that it's not going to somehow super itself. Breaking that assumption, especially for code that adds attributes, makes the code much harder to write, and reason about. You probably can come up with a manual solution to these problems that doesn't work fully generally, but does work for your use case. (By adding specific rules on how classes in your tree interact, you can simplify the problem as far as you want.) Which you'd implement in a metaclass. But once you're writing a metaclass that interferes with the MRO mechanism, why do you care what the default MRO mechanism is? (That last might not be a rhetorical question--maybe the answer is "I wouldn't care, but it's too hard to override the setup of __mro__ in a metaclass __new__ method" or something, in which case there may be something to improve here.)

On Dec 10, 2015, at 15:19, Greg Ewing wrote:

Stephan Sahm wrote:

...

my own intuitiv MRO would not be anything proposed so far, but: (B, mixin, A, mixin, object) As you haven't mentioned this as a possibility at all, I guess having a class twice in this list produces some weird behaviour I do not know about yet - if someone can point out, that would be great.

I'm not sure what that would do to super calls. It's possible you would end up in a loop from Mixin -> A -> Mixin -> A -> ...

But in any case, this MRO doesn't solve the fundamental problem that the MROs of B and A are inherently contradictory. *You* know what MRO you want in this particular case, but there's no way for Python to know that.

After thinking about this a bit: the MRO isn't really contradictory; what's contradictory is what he's expecting super() to do with his MRO. When called from within mixin.spam(), he wants it to give him A, but when calling from within mixin.spam(), he wants it to give him object. Since those two "whens" are identical, there's his problem. But if you never call super(), there's nothing wrong with it; any attribute provided by Mixin the first time means you'll never get to Mixin the second time. (I suppose you could hack that with quantum attributes that have an x% chance of being there each time you call __getattr__, but just don't do that...) Of course he wants to call super. Or, rather, he wants to call something that's kind of like super, but that handles his MRO in a non-contradictory way. A similar MI next-method protocol that went by index rather than by name would work: within mro[1].spam() it gives you mro[2]; within mro[3].spam() it gives you mro[4]. Python doesn't come with a function that does that, but you can write one, with a bit of kludgery. And you can easily write a metaclass that sets up the MRO you want. Which means you can build exactly what the OP is asking for. It still won't do what he wants. For one thing, like super, it depends on the entire class hierarchy cooperating, which includes the root not supering but everyone else doing so--but mixin is the root, and also elsewhere on the tree, so it has to both super and not super. And there are a couple other problems. But tightening up the requirements a bit, I think you can make something coherent out of this. It's definitely not the best way to solve his problem, and it's probably not a good way to solve _any_ problem--but it does work. See http://stupidpythonideas.blogspot.com/2015/12/can-you-customize-method-resol... for a fully worked through example.