Hi Phil, Hi List, unfortunately you do not give enough code to reproduce what you are doing, but just guessing roughly: you say that you have a hierarchy like B -> A -> object, with B and A implemented in C, and then want to use B with a mixin. Programmers with a non-python background then often write class MyClass(B, Mixin): "whatever" this leads to an MRO of MyClass -> B -> Mixin -> A -> object. This is horror if B and A are written in C, because suddenly B needs to do something with Python code if it wants to have to do something with its superclass Mixin, like creating a new object. I am just guessing that this is what your code tries to do. And this is what the comment considers silly. With class MyClass(Mixin, B): "whatever" there is no problem at all. We get an MRO of MyClass -> Mixin -> B -> A -> object. There is no need for B to do anything special, being written in C it already knows by itself how to construct A as well, no need to fiddle with Python at all. In general, it is usually not necessary to deal with super() in C code at all. The statement that there is only single inheritance on the C level becomes obvious once you look at the MRO: that one is always linear, it is always effectively a single inheritance. This is also why you have to call super() only once even if you have multiple superclasses: super() just follows the effective single inheritance of the MRO. Hope that helps. Cheers Martin

Hi Greg, Hi List, Greg wrote:

class MyClass(B, Mixin):

...

"whatever"

this leads to an MRO of MyClass -> B -> Mixin -> A -> object.

If you do the tp_new stuff correctly at the C level, you can still create such a class. The only limitation is that if Mixin has a __new__ method written in Python, it won't get called.

Yes, it is indeed possible to write a class like this. But that would be, as mentioned in the comment, silly. Now there is nothing bad with being silly once in a while, especially while programming that cannot be avoided at times, I am just warning to use such a construct in a wider context. Because the limitation is actually much more general: if the Mixin has any method written in Python, it won't get called, unless you do a lot of heavy lifting. Now why do you write something in C at all? Two reasons: once for speed, which we can effectively exclude here because the super() construct is slow, or because you want to interact with a library written in another language. Unless your library knows about cooperative inheritance, which is very unlikely, the code from class B is not aware that it is not supposed to call the method inherited from A, but the one from the Mixin instead. So by writing class MyClass(B, Mixin): "whatever" I have major problems to understand what you actually want to achieve, what you could not achieve by writing class MyClass(Mixin, B): "whatever" Since one cannot achieve anything with the former that could not be achieved by the latter, just with much less hassle, I would call the former code silly. That said, I might be wrong and there is indeed something one can achieve with the former but not with the latter, I would be very pleased to hear about this, so please come forward! (For sure, all is based on B and A being written in C, not Python) Cheers Martin

On 01/02/2021 23:50, Greg Ewing wrote:

On 2/02/21 12:13 am, Phil Thompson via Python-Dev wrote:

...

TypeError: object.__new__(B) is not safe, use B.__new__()

It's not safe because object.__new__ doesn't know about any C-level initialisation that A or B need.

But A.__new__ is calling object.__new__ and so can take care of its own needs after the latter returns.

At the C level, there is always a *single* inheritance hierarchy.

Why?

The right thing is for B's tp_new to directly call A's tp_new, which calls object's tp_new.

I want my C-implemented class's __new__ to support cooperative multi-inheritance so my A class cannot assume that object.__new__ is the next in the MRO. I did try to call the next-in-MRO's tp_new directly (rather that calling it's __new__ attribute) but that gave me recursion errors.

Don't worry about Python-level multiple inheritance; the interpreter won't let you create an inheritance structure that would mess this up.

Phil

On 02/02/2021 14:18, Greg Ewing wrote:

On 3/02/21 12:07 am, Phil Thompson wrote:

...

On 01/02/2021 23:50, Greg Ewing wrote:

...

At the C level, there is always a *single* inheritance hierarchy.

Why?

Because a C struct can only extend one other C struct.

Yes - I misunderstood what you meant by "at the C level".

...

I want my C-implemented class's __new__ to support cooperative multi-inheritance

I don't think this is possible. Here is what the C API docs have to say about the matter:

-----------------------------------------------------------------------

Note

If you are creating a co-operative tp_new (one that calls a base type’s tp_new or __new__()), you must not try to determine what method to call using method resolution order at runtime. Always statically determine what type you are going to call, and call its tp_new directly, or via type->tp_base->tp_new. If you do not do this, Python subclasses of your type that also inherit from other Python-defined classes may not work correctly. (Specifically, you may not be able to create instances of such subclasses without getting a TypeError.)

-----------------------------------------------------------------------

(Source: https://docs.python.org/3.5/extending/newtypes.html)

This doesn't mean that your type can't be used in multiple inheritance, just that __new__ methods in particular can't be cooperative.

Thanks - that's fairly definitive, although I don't really understand why __new__ has this particular requirement. Phil

On 02/02/2021 23:08, Greg Ewing wrote:

On 3/02/21 4:52 am, Phil Thompson wrote:

...

Thanks - that's fairly definitive, although I don't really understand why __new__ has this particular requirement.

The job of tp_new is to initialise the C struct. To do this, it first has to initialise the fields of the struct it inherits from, then initialise any fields of its own that it adds, in that order.

Understood.

Initialising the inherited fields must be done by calling the tp_new for the struct that it inherits from. You don't want to call the tp_new of some other class that might have got inserted into the MRO, because you have no idea what kind of C struct it expects to get.

I had assumed that some other magic in typeobject.c (eg. conflicting meta-classes) would have raised an exception before getting to this stage if there was a conflict.

Cooperative calling is a nice idea, but it requires rather special conditions to make it work. All the methods must have exactly the same signature, and it mustn't matter what order they're called in. Those conditions don't apply to __new__, especially at the C level where everything is much more strict type-wise.

Thanks for the explanation. Phil