Re: [Python-ideas] A way out of Meta-hell (was: A (meta)class algebra)

On 19 February 2015 at 22:22, Martin Teichmann <martin.teichmann@gmail.com> wrote:
You haven't really explained why you think it's not backportable. You've suggested a backport may require some workarounds in order to handle cooperative multiple inheritance, but that's not even close to being the same thing as not being able to backported at all. That said, if you'd like to write a competing proposal for the community's consideration, please go ahead (details on the process are in PEP 1) I'm certainly open to alternative ideas in this space, and PEP 422 is deferred specifically because I don't currently have time to explore those ideas further myself. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi,
Well, your explanation up there is actually very good at explaining why it isn't back portable: at the time of metaclass.__init__ the __class__ has not been initialized yet, so cooperative multiple inheritance will never work. This holds even true for the two parameter version of super(): as the __class__ has not yet been bound to anything, there is no way to access it from within __class_init__. You say that some workarounds would have to be done to handle cooperative multiple inheritance, yet given that there is no way to tamper with __class__ I cannot see how such a workaround should look like. I would love to be proven wrong on the above statements, as I actually like PEP 422. But back-portability, in my opinion, is actually very important. So, maybe someone out there manages to write a nice (or even ugly) backport of PEP 422?
I'll write something. Greetings Martin

On 20 February 2015 at 18:40, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
Backportability is at best a nice-to-have when it comes to designing changes to core semantics, and it certainly doesn't justify additional design complexity. That's why its better to focus on making actual *usage* simpler, and it may be that a __subclass_init__ hook is simpler overall than a __class_init__. Certainly one point in its favour is that opting in to running the subclass init on the base class with a decorator would be relatively easy, while opting out of running __class_init__ is likely to be a bit tricky. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 21 February 2015 at 23:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
*sigh* I'm already seeing a definite problem with remembering whether its init_class vs class_init, especially since the older Zope version uses the latter name :P Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Speaking of which, I just checked out Daniel's code which was proposed as an implementation of PEP 422. To my astonishment, it does not do what I (and others) had expected, it does not add new functionality to type.__new__ or type.__init__, instead the new machinery is in __builtins__.__build_class__. This has weird consequences, e.g. if you create a new class by writing type("Spam", (SpamBase,) {}), __class_init__ will not be called. I consider this a really unexpected and not desirable behavior. Sure, types.new_class has been modified accordingly, but still. So I think that the current version of PEP 422 unnecessarily adds some unexpected complexity, all of which could be easily avoided with my __init_subclass__ (or __subclass_init__? whatever) approach. Another detail: I think it could be useful if the additional **kwargs to the class statement were forwarded to __(sub)class_init__. Greetings Martin PS there has been some discussion about politeness on this list. I apologize for having been harsh. I hope we can all get back to discussing python and how to improve it!

On 22 Feb 2015 21:36, "Martin Teichmann" <lkb.teichmann@gmail.com> wrote:
There are some specific details that can't be implemented properly in Python 3 using a metaclass, that's one of the core points of the PEP. Specifically, the __class__ cell can only be populated *after* the object has already been created, so the new hook has to be introduced later in the class initialisation process, or it will suffer from the same limitation metaclasses do (i.e. zero-argument super() and any other references to __class__ won't work).
This has weird consequences, e.g. if you create a new class by writing type("Spam", (SpamBase,) {}), __class_init__ will not be called.
This code is inherently broken in Python 3, as it bypasses __prepare__ methods.
I consider this a really unexpected and not desirable behavior.
That ship already sailed years ago when __prepare__ was introduced.
I don't currently believe you're correct about this, but a PEP and reference implementation may persuade me that you've found a different way of achieving the various objectives of PEP 422.
No, that kind of variable signature makes cooperative multiple inheritance a nightmare to implement. The PEP 422 hook is deliberately designed to act like an inheritable class decorator that gets set from within the body of the class definition. It's *not* intended to provide the full power and flexibility offered by a metaclass - it's designed to let folks that understand class decorators and class methods combine the two concepts to define a class decorator that gets called implicitly for both the current class and all subclasses. Regards, Nick.

On 23 February 2015 at 00:25, Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah, thank you: this discussion kicked a new naming idea lose in my brain, so I just pushed an update to PEP 422 that renames the proposed __init_class__ hook to be called __autodecorate__ instead. "It's an implicitly invoked class decorator that gets inherited by subclasses" is really the best way to conceive of the proposed hook, so having "init" as part of the name not only made it hard to remember whether the hook was __init_class__ or __class_init__, it also got people thinking along completely the wrong lines. By contrast, the "auto" in "autodecorate" only makes sense as a prefix, and the inclusion of the word "decorate" in the name means it should be far more effective at triggering the reader's "class decorator" mental model rather than their "class initialisation" one. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 23 February 2015 at 01:15, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
The first reason is because the immediate question that springs to mind for me with that name is "decorate what?". The "auto" prefix hints at the implicit nature of the decoration, as well as the fact the decoration is inherited. The second is because the hook is only relevant to the new implicit decoration proposal, and not to decorators in general. Calling the hook by the more general name __decorate__ suggests that it may also play a role in explicit decorator invocation, and that's not the case. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi,
Well, your explanation up there is actually very good at explaining why it isn't back portable: at the time of metaclass.__init__ the __class__ has not been initialized yet, so cooperative multiple inheritance will never work. This holds even true for the two parameter version of super(): as the __class__ has not yet been bound to anything, there is no way to access it from within __class_init__. You say that some workarounds would have to be done to handle cooperative multiple inheritance, yet given that there is no way to tamper with __class__ I cannot see how such a workaround should look like. I would love to be proven wrong on the above statements, as I actually like PEP 422. But back-portability, in my opinion, is actually very important. So, maybe someone out there manages to write a nice (or even ugly) backport of PEP 422?
I'll write something. Greetings Martin

On 20 February 2015 at 18:40, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
Backportability is at best a nice-to-have when it comes to designing changes to core semantics, and it certainly doesn't justify additional design complexity. That's why its better to focus on making actual *usage* simpler, and it may be that a __subclass_init__ hook is simpler overall than a __class_init__. Certainly one point in its favour is that opting in to running the subclass init on the base class with a decorator would be relatively easy, while opting out of running __class_init__ is likely to be a bit tricky. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 21 February 2015 at 23:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
*sigh* I'm already seeing a definite problem with remembering whether its init_class vs class_init, especially since the older Zope version uses the latter name :P Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Speaking of which, I just checked out Daniel's code which was proposed as an implementation of PEP 422. To my astonishment, it does not do what I (and others) had expected, it does not add new functionality to type.__new__ or type.__init__, instead the new machinery is in __builtins__.__build_class__. This has weird consequences, e.g. if you create a new class by writing type("Spam", (SpamBase,) {}), __class_init__ will not be called. I consider this a really unexpected and not desirable behavior. Sure, types.new_class has been modified accordingly, but still. So I think that the current version of PEP 422 unnecessarily adds some unexpected complexity, all of which could be easily avoided with my __init_subclass__ (or __subclass_init__? whatever) approach. Another detail: I think it could be useful if the additional **kwargs to the class statement were forwarded to __(sub)class_init__. Greetings Martin PS there has been some discussion about politeness on this list. I apologize for having been harsh. I hope we can all get back to discussing python and how to improve it!

On 22 Feb 2015 21:36, "Martin Teichmann" <lkb.teichmann@gmail.com> wrote:
There are some specific details that can't be implemented properly in Python 3 using a metaclass, that's one of the core points of the PEP. Specifically, the __class__ cell can only be populated *after* the object has already been created, so the new hook has to be introduced later in the class initialisation process, or it will suffer from the same limitation metaclasses do (i.e. zero-argument super() and any other references to __class__ won't work).
This has weird consequences, e.g. if you create a new class by writing type("Spam", (SpamBase,) {}), __class_init__ will not be called.
This code is inherently broken in Python 3, as it bypasses __prepare__ methods.
I consider this a really unexpected and not desirable behavior.
That ship already sailed years ago when __prepare__ was introduced.
I don't currently believe you're correct about this, but a PEP and reference implementation may persuade me that you've found a different way of achieving the various objectives of PEP 422.
No, that kind of variable signature makes cooperative multiple inheritance a nightmare to implement. The PEP 422 hook is deliberately designed to act like an inheritable class decorator that gets set from within the body of the class definition. It's *not* intended to provide the full power and flexibility offered by a metaclass - it's designed to let folks that understand class decorators and class methods combine the two concepts to define a class decorator that gets called implicitly for both the current class and all subclasses. Regards, Nick.

On 23 February 2015 at 00:25, Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah, thank you: this discussion kicked a new naming idea lose in my brain, so I just pushed an update to PEP 422 that renames the proposed __init_class__ hook to be called __autodecorate__ instead. "It's an implicitly invoked class decorator that gets inherited by subclasses" is really the best way to conceive of the proposed hook, so having "init" as part of the name not only made it hard to remember whether the hook was __init_class__ or __class_init__, it also got people thinking along completely the wrong lines. By contrast, the "auto" in "autodecorate" only makes sense as a prefix, and the inclusion of the word "decorate" in the name means it should be far more effective at triggering the reader's "class decorator" mental model rather than their "class initialisation" one. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 23 February 2015 at 01:15, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
The first reason is because the immediate question that springs to mind for me with that name is "decorate what?". The "auto" prefix hints at the implicit nature of the decoration, as well as the fact the decoration is inherited. The second is because the hook is only relevant to the new implicit decoration proposal, and not to decorators in general. Calling the hook by the more general name __decorate__ suggests that it may also play a role in explicit decorator invocation, and that's not the case. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Joao S. O. Bueno
-
Martin Teichmann
-
Nick Coghlan