Thread.__init__ should call super()
Since one of the legit use-cases of using the Thread class is subclassing, I think it's __init__ should call super() to support cooperative inheritance. Or perhaps there is a good reason for not doing so? Best Regards, Ilya Kulakov
On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:
Are you talking about threading.Thread or some other Thread? If you are talking about threading.Thread, its only superclass is object, so why bother calling super().__init__? To be successful, it would need to strip out all the parameters and just call: super().__init__() with no args, as object.__init__() takes no parameters. And that does nothing, so what's the point? I'm afraid I don't see why you think that threading.Thread needs to call super. Can you explain? -- Steve
On Friday, October 27, 2017 at 8:05:17 PM UTC-4, Steven D'Aprano wrote:
The way cooperative multiple inheritance works is that if someone defines class SomeClass(Thread): def __init__(self, **kwargs): super().__init() they expect this will initialize the base class Thread as desired. Now, if they add another base class: class SomeBase: def __init__(self, base_x): self.base_x = base_x then they need to pass up the arguments: class SomeClass(SomeBase, Thread): def __init__(self, **kwargs): super().__init(**kwargs) Unfortunately, if the order of base classes is reversed, this no longer works because Thread doesn't call super: class SomeClass(Thread, SomeBase): def __init__(self, **kwargs): super().__init(**kwargs) # SomeBase is not initialized! As things get more complicated it's not always possible to ensure that Thread is the last class in the inheritance, e.g., if there are two classes like Thread that don't call super.
On Sat, Oct 28, 2017 at 12:14:31AM -0700, Neil Girdhar wrote:
I didn't realise that Ilya was talking about *multiple* inheritance, since he didn't use the word, only "cooperative". But since you are talking about multiple inheritence:
That won't work, since you misspelled __init__ and neglected to pass any arguments :-) You need: super().__init__(**kwargs) otherwise Thread will not be initialised.
That's not going to work either, because you're passing arguments to SomeBase that it doesn't understand: all the args that Thread expects. And of course its going to doubly not work, since SomeBase fails to call super, so Thread.__init__ still doesn't get called. If you fix all those problems, you still have another problem: in Thread, if you call super().__init__(**kwargs) then object.__init__ will fail, as I mentioned in my earlier post; but if you call: super().__init__() then object is satisfied, but SomeBase.__init__ gets called with no arguments. You can't satisfy both at the same time with a single call to super.
You can't just add classes willy-nilly into the superclass list. That's why it is called *cooperative* multiple inheritence: the superclasses all have to be designed to work together. You *can't* have two classes that don't call super -- and you must have one class just ahead of object, to prevent object from receiving args it can't do anything with. And that class might as well be Thread. At least, I can't think of any reason why it shouldn't be Thread. -- Steve
On Sat, Oct 28, 2017 at 7:15 AM Steven D'Aprano <steve@pearwood.info> wrote:
That's totally fine. That's how cooperative multiple inheritance works in Python. SomeBase is supposed to pass along everything to its superclass init through kwargs just like I illustrated in my corrected code.
No, this works fine. The idea is that each class consumes the keyword arguments it wants and passes along the rest. By the time you get to object, there are none left and object.__init__ doesn't complain. If there are extra arguments, then object.__init__ raises.
You don't need "one class just ahead of object". Every class calls super passing along its arguments via kwargs. And it's very hard when combining various mixins to ensure that a given order is maintained.
And that class might as well be Thread. At least, I can't think of any reason why it shouldn't be Thread.
I'm sorry, but I don't agree with this. Unfortunately, there are some various oversights in Python when it comes to cooperative multiple inheritance. This is only one of them that I've also run into.
Neil, thank you for doing much better job explaining the problem. Generally, I'm cool with Python's standard library classes not calling super(), as many of them are not designed for subclassing. But those which are should do that. E.g. take a look at more recent asyncio's Protocol and Transport classes: they all properly call super(). One potential problem is that it will break existing code: class X(Thread, SomethingElse): def __init__(self): Thread.__init__(self) SomethingElse.__init__(self) SomethingElse.__init__ will be called twice. Is it a good reason for "old" classes to lag behind? I don't know. Perhaps some mechanism (invisible to a user) can be designed to avoid that. E.g. super() may leave a flag which should signal interpreter to "skip" all direct calls of a function and warn about it (DeprecationWarning?). Best Regards, Ilya Kulakov
You can subclass Thread just fine, you just can't have it in a multiple inheritance hierarchy except at the end of the MRO (before object). That shouldn't stop you from doing anything you want though -- you can define e.g. class MyThread(Thread): def __init__(self, *args, **kwds): Thread.__init__(self, *args, **kwds) super(Thread, self).__init__(*args, **kwds) and use this class instead of Thread everywhere. (You'll have to decide which arguments to pass on and which ones to ignore, but that's not specific to the issue of Thread.) Of course you're probably better off not trying to be so clever. On Fri, Oct 27, 2017 at 1:59 PM, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On Fri, 27 Oct 2017 13:59:01 -0700 Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
Since one of the legit use-cases of using the Thread class is subclassing, I think it's __init__ should call super() to support cooperative inheritance.
Not to derail this thread, but I find it much clearer to use the functional form of the Thread class, i.e. to pass the `target` and `args` constructor parameters. Regards Antoine.
On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:
Are you talking about threading.Thread or some other Thread? If you are talking about threading.Thread, its only superclass is object, so why bother calling super().__init__? To be successful, it would need to strip out all the parameters and just call: super().__init__() with no args, as object.__init__() takes no parameters. And that does nothing, so what's the point? I'm afraid I don't see why you think that threading.Thread needs to call super. Can you explain? -- Steve
On Friday, October 27, 2017 at 8:05:17 PM UTC-4, Steven D'Aprano wrote:
The way cooperative multiple inheritance works is that if someone defines class SomeClass(Thread): def __init__(self, **kwargs): super().__init() they expect this will initialize the base class Thread as desired. Now, if they add another base class: class SomeBase: def __init__(self, base_x): self.base_x = base_x then they need to pass up the arguments: class SomeClass(SomeBase, Thread): def __init__(self, **kwargs): super().__init(**kwargs) Unfortunately, if the order of base classes is reversed, this no longer works because Thread doesn't call super: class SomeClass(Thread, SomeBase): def __init__(self, **kwargs): super().__init(**kwargs) # SomeBase is not initialized! As things get more complicated it's not always possible to ensure that Thread is the last class in the inheritance, e.g., if there are two classes like Thread that don't call super.
On Sat, Oct 28, 2017 at 12:14:31AM -0700, Neil Girdhar wrote:
I didn't realise that Ilya was talking about *multiple* inheritance, since he didn't use the word, only "cooperative". But since you are talking about multiple inheritence:
That won't work, since you misspelled __init__ and neglected to pass any arguments :-) You need: super().__init__(**kwargs) otherwise Thread will not be initialised.
That's not going to work either, because you're passing arguments to SomeBase that it doesn't understand: all the args that Thread expects. And of course its going to doubly not work, since SomeBase fails to call super, so Thread.__init__ still doesn't get called. If you fix all those problems, you still have another problem: in Thread, if you call super().__init__(**kwargs) then object.__init__ will fail, as I mentioned in my earlier post; but if you call: super().__init__() then object is satisfied, but SomeBase.__init__ gets called with no arguments. You can't satisfy both at the same time with a single call to super.
You can't just add classes willy-nilly into the superclass list. That's why it is called *cooperative* multiple inheritence: the superclasses all have to be designed to work together. You *can't* have two classes that don't call super -- and you must have one class just ahead of object, to prevent object from receiving args it can't do anything with. And that class might as well be Thread. At least, I can't think of any reason why it shouldn't be Thread. -- Steve
On Sat, Oct 28, 2017 at 7:15 AM Steven D'Aprano <steve@pearwood.info> wrote:
That's totally fine. That's how cooperative multiple inheritance works in Python. SomeBase is supposed to pass along everything to its superclass init through kwargs just like I illustrated in my corrected code.
No, this works fine. The idea is that each class consumes the keyword arguments it wants and passes along the rest. By the time you get to object, there are none left and object.__init__ doesn't complain. If there are extra arguments, then object.__init__ raises.
You don't need "one class just ahead of object". Every class calls super passing along its arguments via kwargs. And it's very hard when combining various mixins to ensure that a given order is maintained.
And that class might as well be Thread. At least, I can't think of any reason why it shouldn't be Thread.
I'm sorry, but I don't agree with this. Unfortunately, there are some various oversights in Python when it comes to cooperative multiple inheritance. This is only one of them that I've also run into.
Neil, thank you for doing much better job explaining the problem. Generally, I'm cool with Python's standard library classes not calling super(), as many of them are not designed for subclassing. But those which are should do that. E.g. take a look at more recent asyncio's Protocol and Transport classes: they all properly call super(). One potential problem is that it will break existing code: class X(Thread, SomethingElse): def __init__(self): Thread.__init__(self) SomethingElse.__init__(self) SomethingElse.__init__ will be called twice. Is it a good reason for "old" classes to lag behind? I don't know. Perhaps some mechanism (invisible to a user) can be designed to avoid that. E.g. super() may leave a flag which should signal interpreter to "skip" all direct calls of a function and warn about it (DeprecationWarning?). Best Regards, Ilya Kulakov
You can subclass Thread just fine, you just can't have it in a multiple inheritance hierarchy except at the end of the MRO (before object). That shouldn't stop you from doing anything you want though -- you can define e.g. class MyThread(Thread): def __init__(self, *args, **kwds): Thread.__init__(self, *args, **kwds) super(Thread, self).__init__(*args, **kwds) and use this class instead of Thread everywhere. (You'll have to decide which arguments to pass on and which ones to ignore, but that's not specific to the issue of Thread.) Of course you're probably better off not trying to be so clever. On Fri, Oct 27, 2017 at 1:59 PM, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On Fri, 27 Oct 2017 13:59:01 -0700 Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
Since one of the legit use-cases of using the Thread class is subclassing, I think it's __init__ should call super() to support cooperative inheritance.
Not to derail this thread, but I find it much clearer to use the functional form of the Thread class, i.e. to pass the `target` and `args` constructor parameters. Regards Antoine.
participants (5)
-
Antoine Pitrou -
Guido van Rossum -
Ilya Kulakov -
Neil Girdhar -
Steven D'Aprano