Submitting PEP 422 (Simple class initialization hook) for pronouncement
I wrote PEP 422 (simple class initialization hook) last year after PJE
pointed out that Python 3 style metaclasses made some Python 2 code
impossible to migrate (since the class body could no longer request
modifications to be made to the class after initialization was
complete).
It then languished, as I never found the time to actually implement
it. Fortunately, Daniel Urban has now agreed to be a co-author on the
PEP, and has fixed a couple of lingering technical errors in the PEP,
as well as providing a reference implementation at
http://bugs.python.org/issue17044
For those that don't recall the original discussion, the proposal is
to add a new __init_class__ hook, invoked after the class object is
created, but before the class decorators are applied. This provides a
simple approach to inherited post-creation modification of classes,
without the need for a custom metaclass.
There's currently no BDFL delegate assigned, so this is a request for
a pronouncement from Guido.
Regards,
Nick.
=======================================
PEP: 422
Title: Simple class initialisation hook
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan
On Sun, 10 Feb 2013 22:32:50 +1000
Nick Coghlan
Replaces many use cases for dynamic setting of ``__metaclass__`` -----------------------------------------------------------------
For use cases that don't involve completely replacing the defined class, Python 2 code that dynamically set ``__metaclass__`` can now dynamically set ``__init_class__`` instead. For more advanced use cases, introduction of an explicit metaclass (possibly made available as a required base class) will still be necessary in order to support Python 3.
So, what are the use cases? I probably write metaclasses once a year, I wonder how much incentive there is to bring an additional complication to the already complicated class construction process. Regards Antoine.
On Sun, Feb 10, 2013 at 10:47 PM, Antoine Pitrou
On Sun, 10 Feb 2013 22:32:50 +1000 Nick Coghlan
wrote: Replaces many use cases for dynamic setting of ``__metaclass__`` -----------------------------------------------------------------
For use cases that don't involve completely replacing the defined class, Python 2 code that dynamically set ``__metaclass__`` can now dynamically set ``__init_class__`` instead. For more advanced use cases, introduction of an explicit metaclass (possibly made available as a required base class) will still be necessary in order to support Python 3.
At least PEAK uses it for interface declarations, I believe Zope do something similar. However, just as I championed PEP 414 to make porting Unicode-aware projects easier, I'm championing this one because I don't consider it acceptable for Python 3 to be a less capable language than Python 2 in this regard. Having come up with what I consider to be an elegant design to restore that capability, I'm not especially interested in passing judgment on the worthiness of what PJE is doing with Python 2 metaclasses that cannot currently be replicated in Python 3.
So, what are the use cases? I probably write metaclasses once a year, I wonder how much incentive there is to bring an additional complication to the already complicated class construction process.
One of the main goals of the PEP is actually to give people even *less* reason to write custom metaclasses, as you won't need to write one any more if all you want to do is adjust the class attributes after execution of the class body completes, and automatically do the same for any subclasses. (The latter part is key - if you don't want inheritance, then class decorators fill the role just fine. That's why the design in the PEP is the way it us - __init_class__ effectively becomes the innermost class decorator, which is called even for subclasses) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Feb 10, 2013 at 2:32 PM, Nick Coghlan
For those that don't recall the original discussion, the proposal is to add a new __init_class__ hook, invoked after the class object is created, but before the class decorators are applied. This provides a simple approach to inherited post-creation modification of classes, without the need for a custom metaclass.
Couldn't one just write a metaclass that calls __init_class__ for you? Schiavo Simon
On Sun, 10 Feb 2013 23:17:00 +1000
Nick Coghlan
On Sun, Feb 10, 2013 at 10:47 PM, Antoine Pitrou
wrote: On Sun, 10 Feb 2013 22:32:50 +1000 Nick Coghlan
wrote: Replaces many use cases for dynamic setting of ``__metaclass__`` -----------------------------------------------------------------
For use cases that don't involve completely replacing the defined class, Python 2 code that dynamically set ``__metaclass__`` can now dynamically set ``__init_class__`` instead. For more advanced use cases, introduction of an explicit metaclass (possibly made available as a required base class) will still be necessary in order to support Python 3.
At least PEAK uses it for interface declarations, I believe Zope do something similar.
zope.interface has been ported to Python 3, so the annoyance can't be very blocking.
However, just as I championed PEP 414 to make porting Unicode-aware projects easier, I'm championing this one because I don't consider it acceptable for Python 3 to be a less capable language than Python 2 in this regard.
This sounds like a theoretical concern rather than a practical one.
Having come up with what I consider to be an elegant design to restore that capability, I'm not especially interested in passing judgment on the worthiness of what PJE is doing with Python 2 metaclasses that cannot currently be replicated in Python 3.
It may be locally elegant, but it still brings additional complication to the global model. Reading your proposal reminds me of the various ways of influencing the pickle protocol: http://docs.python.org/dev/library/pickle.html#pickling-class-instances Nobody can claim this is simple and easy to wrap your head around. It is a maintenance burden, and it discourages understanding of the underlying model by anyone but language experts. Regards Antoine.
On Sun, Feb 10, 2013 at 11:33 PM, Simon Cross
On Sun, Feb 10, 2013 at 2:32 PM, Nick Coghlan
wrote: For those that don't recall the original discussion, the proposal is to add a new __init_class__ hook, invoked after the class object is created, but before the class decorators are applied. This provides a simple approach to inherited post-creation modification of classes, without the need for a custom metaclass.
Couldn't one just write a metaclass that calls __init_class__ for you?
And, indeed, that's what Zope did more than 10 years ago. The problem is that you're now locked in to that metaclass - metaclass conflicts become a permanent risk. You also can't *add* a metaclass to public classes, the risk of introducing metaclass conflicts means that such a change is always backwards incompatible. Adding __init_class__ is fine though, because that's no higher risk than adding a __new__ or __init__ method (it's actually lower risk, since the signature is always consistent). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Feb 10, 2013 at 11:34 PM, Antoine Pitrou
Nobody can claim this is simple and easy to wrap your head around. It is a maintenance burden, and it discourages understanding of the underlying model by anyone but language experts.
You think anyone but language experts fully understands the metaclass mechanism now? *I* got the description of some details wrong in the PEP, and I'm probably in the top half dozen people on the planet when it comes to understanding how Python 3's class model works. Plenty of people understand decorators though, and that's all __init_class__ is: a special class decorator that is applied before any other decorators, and is automatically inherited by subclasses. It massively lowers the barrier to entry for inherited post-modification of classes. Will it mean people will only bother to understand metaclasses if they actually *need* metaclass.__prepare__ or to control the identity of the object returned by metaclass.__call__? Absolutely. However, I don't see that as any different from the fact that vastly more people understand how to use instance __init__ methods correctly than understand the ins and outs of using __new__ to control the *creation* of the object, rather than merely initialising it after it is already created. In some respects, cleanly separating out the two steps of controlling initialisation and controlling creation can actually make the two easier to grasp (since you can tackle them individually as distinct concepts, rather than having to wrap your head around both of them at the same time). As far as the maintenance burden goes, the patch to enable PEP 422 for types.new_class() is trivial: - return meta(name, bases, ns, **kwds) + cls = meta(name, bases, ns, **kwds) + try: + initcl = cls.__init_class__ + except AttributeError: + pass + else: + initcl() + return cls The change in the builtin.__build_class__ implementation is similarly trivial (and has the same semantics), it's just more verbose due to its being written in C. The documentation changes are quite straightforward (moving __init_class__ and decorator invocation out to their own subsection), and the tests Daniel has provided are extensive. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 11 Feb 2013 00:09:55 +1000
Nick Coghlan
As far as the maintenance burden goes, the patch to enable PEP 422 for types.new_class() is trivial:
- return meta(name, bases, ns, **kwds) + cls = meta(name, bases, ns, **kwds) + try: + initcl = cls.__init_class__ + except AttributeError: + pass + else: + initcl() + return cls
I didn't know types.new_class(). I suppose that's because I'm not part of the top half dozen people on the planet :-) I've stopped trying to understand when I saw about the __build_class__ additions and whatnot. I now consider that part of the language totally black magic. I'm not sure it's a good thing when even some maintainers of the language don't understand some of its (important) corners.
The change in the builtin.__build_class__ implementation is similarly trivial (and has the same semantics), it's just more verbose due to its being written in C.
Sure, every little addition is "trivial". At the end you have a scary monster made of many little trivial additions along the years, and everyone has to take care not to break it. Regards Antoine.
Antoine Pitrou, 10.02.2013 15:28:
On Mon, 11 Feb 2013 00:09:55 +1000 Nick Coghlan wrote:
As far as the maintenance burden goes, the patch to enable PEP 422 for types.new_class() is trivial:
- return meta(name, bases, ns, **kwds) + cls = meta(name, bases, ns, **kwds) + try: + initcl = cls.__init_class__ + except AttributeError: + pass + else: + initcl() + return cls
I didn't know types.new_class(). I suppose that's because I'm not part of the top half dozen people on the planet :-) I've stopped trying to understand when I saw about the __build_class__ additions and whatnot. I now consider that part of the language totally black magic.
I'm not sure it's a good thing when even some maintainers of the language don't understand some of its (important) corners.
I, for one, only learned most of this stuff when we implemented it for Cython and I'm sure there are lots of little details that I'm not aware of. Use cases are too rare to fully learn it and the matter is certainly complex.
The change in the builtin.__build_class__ implementation is similarly trivial (and has the same semantics), it's just more verbose due to its being written in C.
Sure, every little addition is "trivial". At the end you have a scary monster made of many little trivial additions along the years, and everyone has to take care not to break it.
FWIW, I agree with Nick that this actually kills some use cases of metaclasses and could thus lower the barrier for "doing stuff to classes" for those who don't want to dig into metaclasses for one reason or another. However, it's hard to say if this "new way of doing it" doesn't come with its own can of worms. For example, would cooperative calls to "__init_class__" work if a superclass already defines it? Do implementors have to remember that? And is it clear how this should be done, e.g. what should normally go first, my own code or the superclass call? Supporting cooperative __init_class__() calls properly might actually be a good thing. Currently, there's only one metaclass, plus a sequence of decorators, which makes the setup rather static and sometimes tedious for subclasses that need to do certain things by themselves, but in addition to what happens already. So, the way to explain it to users would be 1) don't use it, 2) if you really need to do something to a class, use a decorator, 3) if you need to decide dynamically what to do, define __init_class__() and 4) don't forget to call super's __init_class__() in that case, and 5) only if you need to do something substantially more involved and know what you're doing, use a metaclass. Sounds a bit smoother than the current metaclass hole that we throw people into directly when a decorator won't catch it for them. It's usually not a good idea to add complexity for something that's essentially a rare use case. Only the assumption that it might actually make it easier for users to handle certain other use cases where they'd currently end up with their own home-grown metaclass is a point that makes it worth considering for me. But then again, maybe these use cases are really rare enough to just let people figure out how to do it with metaclasses, or even a combination of metaclasses and decorators. Stefan
On Sun, Feb 10, 2013 at 5:48 PM, Stefan Behnel
However, it's hard to say if this "new way of doing it" doesn't come with its own can of worms. For example, would cooperative calls to "__init_class__" work if a superclass already defines it? Do implementors have to remember that? And is it clear how this should be done, e.g. what should normally go first, my own code or the superclass call? Supporting cooperative __init_class__() calls properly might actually be a good thing. Currently, there's only one metaclass, plus a sequence of decorators, which makes the setup rather static and sometimes tedious for subclasses that need to do certain things by themselves, but in addition to what happens already.
Cooperative multiple inheritance of __init_class__ would work exactly the same way as it works e.g., for __init__ of any other method (actually it is easier, since as Nick mentioned, the signature is always the same): __init_class__ can simply use the zero argument form of super. There is a simple example in the tests at http://bugs.python.org/issue17044. Daniel
On Sun, Feb 10, 2013 at 7:32 AM, Nick Coghlan
class Example: @classmethod def __init_class__(cls):
Is the @classmethod required? What happens if it's not present? Second, will type have a default __init_class__? (IMO, it should, otherwise it will be impossible to write co-operative __init_class__ functions.) Only other comment is that the PEP could use a more concrete use case, e.g.: class Record: __fields = {} @classmethod def __init_class__(cls): cls.__fields = dict(cls.__fields) # inherited fields cls.__fields.update({attr:val for attr, val in cls.__dict__.iteritems() if isinstance(val, Field)}) super().__init_class__() # be co-operative # ...other methods that use the __fields attribute class SomeRecord(Record): foo = Field(int) bar = Field(str) Putting something like this early on might help to demonstrate the usefulness of the feature on its own merits, independent of the porting issue, etc. ;-)
On Sun, Feb 10, 2013 at 11:48 AM, Stefan Behnel
So, the way to explain it to users would be 1) don't use it, 2) if you really need to do something to a class, use a decorator, 3) if you need to decide dynamically what to do, define __init_class__() and 4) don't forget to call super's __init_class__() in that case, and 5) only if you need to do something substantially more involved and know what you're doing, use a metaclass.
I'd revise that to: 1) if there's no harm in forgetting to decorate a subclass, use a class decorator 2) if you want to ensure that a modification is applied to every subclass of a single common base class, define __init_class__ (and always call its super) 3) If you need to make the class object *act* differently (not just initialize it or trigger some other side-effect at creation time), or if you want the class suite to return some other kind of object, you'll need a metaclass. Essentially, this change fixes a hole in class decorators that doesn't exist with function decorators: if you need the decoration applied to subclasses, you can end up with silent failures right now. Conversely, if you try prevent such failures using a metaclass, you not only have a big hill to climb, but the resulting code will be vulnerable to metaclass conflicts. The proposed solution neatly fixes both of these problems, providing One Obvious Way to do subclass initialization. (An alternative, I suppose, would be to let you do something like @@someClassDecorator to have inheritable class decorators, but as Nick has pointed out, implementing inheritable decorators is a lot more complicated.)
On Sun, 10 Feb 2013 16:27:49 -0500
PJ Eby
On Sun, Feb 10, 2013 at 7:32 AM, Nick Coghlan
wrote: class Example: @classmethod def __init_class__(cls):
Is the @classmethod required? What happens if it's not present?
Second, will type have a default __init_class__? (IMO, it should, otherwise it will be impossible to write co-operative __init_class__ functions.)
Only other comment is that the PEP could use a more concrete use case, e.g.:
class Record: __fields = {}
@classmethod def __init_class__(cls): cls.__fields = dict(cls.__fields) # inherited fields cls.__fields.update({attr:val for attr, val in cls.__dict__.iteritems() if isinstance(val, Field)}) super().__init_class__() # be co-operative
# ...other methods that use the __fields attribute
class SomeRecord(Record): foo = Field(int) bar = Field(str)
Putting something like this early on might help to demonstrate the usefulness of the feature on its own merits, independent of the porting issue, etc. ;-)
Can you explain what the example does / is supposed to do?
On Mon, Feb 11, 2013 at 7:41 AM, PJ Eby
On Sun, Feb 10, 2013 at 11:48 AM, Stefan Behnel
wrote: So, the way to explain it to users would be 1) don't use it, 2) if you really need to do something to a class, use a decorator, 3) if you need to decide dynamically what to do, define __init_class__() and 4) don't forget to call super's __init_class__() in that case, and 5) only if you need to do something substantially more involved and know what you're doing, use a metaclass.
I'd revise that to:
1) if there's no harm in forgetting to decorate a subclass, use a class decorator 2) if you want to ensure that a modification is applied to every subclass of a single common base class, define __init_class__ (and always call its super) 3) If you need to make the class object *act* differently (not just initialize it or trigger some other side-effect at creation time), or if you want the class suite to return some other kind of object, you'll need a metaclass.
I like that. Perhaps the PEP should propose some additional guidance in PEP 8 regarding class based metaprogramming?
Essentially, this change fixes a hole in class decorators that doesn't exist with function decorators: if you need the decoration applied to subclasses, you can end up with silent failures right now. Conversely, if you try prevent such failures using a metaclass, you not only have a big hill to climb, but the resulting code will be vulnerable to metaclass conflicts.
The proposed solution neatly fixes both of these problems, providing One Obvious Way to do subclass initialization.
I also realised last night that one significant benefit of cleanly separating class creation from class initialisation (as __new__ and __init__ separate instance creation and initialisation) is the ability to create a shared metaclass that just changes the namespace type with __prepare__, and then use __init_class__ to control what you do with it. Here's the more extended example I'm now considering adding to the PEP in order to show the improved composability the PEP offers (writing the below example with only metaclasses would be... challenging). It's still a toy example, but I don't believe there is any non-toy use case for metaclass composition that is going to be short enough to fit in a PEP: # Define a metaclass as in Python 3.3 and earlier import collections class OrderedMeta(type): def __prepare__(self, *args, **kwds): return collections.OrderedDict() # Won't be needed if we add a noop __init_class__ to type def __init_class__(cls): pass class OrderedClass(metaclass=OrderedMeta): pass # Easily supplement the metaclass behaviour in a class definition class SimpleRecord(OrderedClass): """Simple ordered record type (inheritance not supported)""" @classmethod def __init_class__(cls): super().__init_class__() cls.__fields = fields = [] for attr, obj in cls.__dict__.items(): if attr.startswith("_") or callable(obj): continue fields.append(attr) def __init__(self, *values): super().__init__(*values) for attr, obj in zip(self.__fields, values): setattr(self, attr, obj) def to_dict(self): fields = ((k, getattr(self, k)) for k in self.__fields) return collections.OrderedDict(fields) # Supplement the metaclass differently in another class definition class InheritableRecord(OrderedClass): """More complex record type that supports inheritance""" @classmethod def __init_class__(cls): super().__init_class__() cls.__fields = fields = [] for mro_cls in cls.mro(): for attr, obj in cls.__dict__.items(): if attr.startswith("_") or callable(obj): continue fields.append(attr) def __init__(self, *values): super().__init__(*values) for attr, obj in zip(self.__fields, values): setattr(self, attr, obj) def to_dict(self): fields = ((k, getattr(self, k)) for k in self.__fields) return collections.OrderedDict(fields) # Compared to custom metaclasses, composition is much simpler class ConfusedRecord(InheritableRecord, SimpleRecord): """Odd record type, only included to demonstrate composition""" # to_dict is inherited from InheritableRecord def to_simple_dict(self): return SimpleRecord.to_dict(self) Perhaps it would sweeten the deal if the PEP also provided types.OrderedMeta and types.OrderedClass, such that inheriting from types.OrderedClass and defining __init_class__ became the one-obvious-way to do order dependent class bodies? (I checked, we can make types depend on collections without a circular dependency. We would need to fix the increasingly inaccurate docstring, though) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Feb 10, 2013, at 02:34 PM, Antoine Pitrou wrote:
zope.interface has been ported to Python 3, so the annoyance can't be very blocking.
The syntax is different, but I actually prefer the Python 3-compatible syntax better. It uses a class decorator instead of a magic class attribute, so it's much clearer IMHO. Works in Python 2, too, fsvo. -Barry
On Feb 10, 2013, at 03:28 PM, Antoine Pitrou wrote:
Sure, every little addition is "trivial". At the end you have a scary monster made of many little trivial additions along the years, and everyone has to take care not to break it.
Why Antoine, that surely isn't the case with the import system! <sarcastic wink>. -Barry
On Feb 11, 2013, at 08:33 PM, Nick Coghlan wrote:
I like that. Perhaps the PEP should propose some additional guidance in PEP 8 regarding class based metaprogramming?
I wouldn't put it in PEP 8, since it'll glaze the eyes of all but 6 people on the planet. Probably better as a HOWTO in the documentation. Cheers, -Barry
Le Mon, 11 Feb 2013 10:15:36 -0500,
Barry Warsaw
On Feb 10, 2013, at 03:28 PM, Antoine Pitrou wrote:
Sure, every little addition is "trivial". At the end you have a scary monster made of many little trivial additions along the years, and everyone has to take care not to break it.
Why Antoine, that surely isn't the case with the import system! <sarcastic wink>.
Nor with pythonrun.c. I wouldn't even mention typeobject.c or the build system ;-) Regards Antoine.
Hi Nick, I think this will make a fine addition to the language. I agree that it is superior to the alternatives and fulfills a real (if rare) need. I only have a few nits/questions/suggestions. - With PJE, I think __init_class__ should automatically be a class method. The same way that __new__ is automatically a class method. - Would it make any sense to require that __init_class__ *returns* the new class object (to complete the similarity with class decorators)? Frankly, I lost track of things when you were discussing the exact point in time where __class__ is added to the class. (This may place me outside the set of people who fully understand Python's metaclasses. So be it. :-) Perhaps some explanation of the existing mechanism would be in place? It would be nice if you explained reason for why __class__ cannot be set earlier in the game (otherwise the response to this argument would be "so fix it", but somehow I doubt that is possible). I think it may also be unclear at all times whether __class__, when referenced, is meant to refer to the metaclass or to the class under construction. - I wouldn't add the OrderedMeta/OrderedClass to the PEP. Keep it focused on enabling. - Adding more examples would be great. -- --Guido van Rossum (python.org/~guido)
On Mon, Feb 11, 2013 at 12:44 PM, Guido van Rossum
Hi Nick,
I think this will make a fine addition to the language. I agree that it is superior to the alternatives and fulfills a real (if rare) need.
I only have a few nits/questions/suggestions.
- With PJE, I think __init_class__ should automatically be a class method.
Actually, I didn't say that as such, because I'm not sure how the heck we'd implement that. ;-) For example, at what point is it converted to a classmethod? Is it going to be a slot method with special C-level handling? Handled by the compiler? What happens if somebody makes it a
The same way that __new__ is automatically a class method.
Actually, isn't it automatically a staticmethod? Oh crap. Now that I'm thinking about it, doesn't this *have* to be a static method, explicitly passing in the class? I mean, otherwise, won't calling super().__init_class__() invoke it on the base class, rather than the current class? ISTM that EIBTI argues for the __new__/staticmethod approach, especially if you're returning the class (per below)
- Would it make any sense to require that __init_class__ *returns* the new class object (to complete the similarity with class decorators)?
It would certainly be quite useful to do so, but in that case, perhaps the method should be named __decorate_class__? And in that event the standard usage would look like: def __decorate_class__(cls): cls = super().__decorate_class__(cls) # do stuff return cls On the other hand, one could just drop the super() requirement and make the usage even simpler by having the class machinery walk the MRO and pass each method the result of invoking the previous one. Then the methods are short and sweet, and super() and __class__ don't come into it. (Though I guess the class machinery could keep setting __class__ to whatever the last-returned class was.) In his first draft, Nick implemented inheritable decorators instead, using a __decorators__ attribute in the class body, or something like that. While that approach had an issue or two of its own, it's possible that just going with a single __decorate_class__ method would work out better.
On Mon, Feb 11, 2013 at 12:57 PM, PJ Eby
On Mon, Feb 11, 2013 at 12:44 PM, Guido van Rossum
wrote: Hi Nick,
I think this will make a fine addition to the language. I agree that it is superior to the alternatives and fulfills a real (if rare) need.
I only have a few nits/questions/suggestions.
- With PJE, I think __init_class__ should automatically be a class method.
Actually, I didn't say that as such, because I'm not sure how the heck we'd implement that. ;-)
For example, at what point is it converted to a classmethod? Is it going to be a slot method with special C-level handling? Handled by the compiler? What happens if somebody makes it a
The same way that __new__ is automatically a class method.
Actually, isn't it automatically a staticmethod? Oh crap. Now that I'm thinking about it, doesn't this *have* to be a static method, explicitly passing in the class? I mean, otherwise, won't calling super().__init_class__() invoke it on the base class, rather than the current class?
ISTM that EIBTI argues for the __new__/staticmethod approach, especially if you're returning the class (per below)
Let's see what Nick and the implementer say.
- Would it make any sense to require that __init_class__ *returns* the new class object (to complete the similarity with class decorators)?
It would certainly be quite useful to do so, but in that case, perhaps the method should be named __decorate_class__? And in that event the standard usage would look like:
def __decorate_class__(cls): cls = super().__decorate_class__(cls) # do stuff return cls
On the other hand, one could just drop the super() requirement and make the usage even simpler by having the class machinery walk the MRO and pass each method the result of invoking the previous one. Then the methods are short and sweet, and super() and __class__ don't come into it. (Though I guess the class machinery could keep setting __class__ to whatever the last-returned class was.)
In his first draft, Nick implemented inheritable decorators instead, using a __decorators__ attribute in the class body, or something like that. While that approach had an issue or two of its own, it's possible that just going with a single __decorate_class__ method would work out better.
Half-baked idea: Maybe the base class __decorate_class__ would call the class decorators? Or does that not make sense? -- --Guido van Rossum (python.org/~guido)
On 12 Feb 2013 07:44, "Guido van Rossum"
On Mon, Feb 11, 2013 at 12:57 PM, PJ Eby
wrote: On Mon, Feb 11, 2013 at 12:44 PM, Guido van Rossum
wrote:
Hi Nick,
I think this will make a fine addition to the language. I agree that it is superior to the alternatives and fulfills a real (if rare) need.
I only have a few nits/questions/suggestions.
- With PJE, I think __init_class__ should automatically be a class method.
Actually, I didn't say that as such, because I'm not sure how the heck we'd implement that. ;-)
For example, at what point is it converted to a classmethod? Is it going to be a slot method with special C-level handling? Handled by the compiler? What happens if somebody makes it a
The same way that __new__ is automatically a class method.
Actually, isn't it automatically a staticmethod? Oh crap. Now that I'm thinking about it, doesn't this *have* to be a static method, explicitly passing in the class? I mean, otherwise, won't calling super().__init_class__() invoke it on the base class, rather than the current class?
ISTM that EIBTI argues for the __new__/staticmethod approach, especially if you're returning the class (per below)
Let's see what Nick and the implementer say.
I think these are some interesting ideas and it's going to take me a while to digest them and update the PEP :) A few random thoughts: 1. I like the idea of a metaprogramming "howto" that provides advice on choosing a suitable level of metaprogramming (with the default choice being "use existing decorators", then escalating through creating custom decorators all the way to creating custom metaclasses). I don't think the PEP needs to be conditional on writing that, but I will at least add PJE's list to the PEP itself. 2. I see the new method as more analogous to__init__ than to__new__, so the __decorate_class__ idea makes me nervous, as it's closer to a __new__ method. Composition gets a *lot* harder when your parent class can switch out the object on you. 3. I'm trying to avoid any custom magic specific to this method, but making it implicitly a static or class method is fairly easy if we so choose - the standard retrieval code during class creation can just bypass the descriptor machinery, and wrap it in staticmethod or classmethod if it isn't already. Given that __new__ is already implicitly static, it may be easier to follow that precedent here rather than trying to explain why an explicit @classmethod is needed in one case but not the other. 4.__class__ is already bound as soon as we have a class object to bind it to, so we can't move it any earlier. However, it's already early enough to let references to it from the new method (including the implied one in zero-arg super) work properly. The black magic that is zero-arg super also deals with PJE's concern about propagating the actual class up the MRO (as it is equivalent to "super(__class__, first_argument)"). 5. Implicitly walking the MRO bothers me, as it becomes a special case for people to learn. We don't do that for __init__ or __new__, so I don't think it makes sense to do it here. We can include a recommended structure in the docs, where the first step is calling the parent through. As PJE suggested, a no-op method on type will make that simple and fairly safe (people using a metaclass hierarchy not anchored on type can figure out their own equivalent) Cheers, Nick.
- Would it make any sense to require that __init_class__ *returns* the new class object (to complete the similarity with class decorators)?
It would certainly be quite useful to do so, but in that case, perhaps the method should be named __decorate_class__? And in that event the standard usage would look like:
def __decorate_class__(cls): cls = super().__decorate_class__(cls) # do stuff return cls
On the other hand, one could just drop the super() requirement and make the usage even simpler by having the class machinery walk the MRO and pass each method the result of invoking the previous one. Then the methods are short and sweet, and super() and __class__ don't come into it. (Though I guess the class machinery could keep setting __class__ to whatever the last-returned class was.)
In his first draft, Nick implemented inheritable decorators instead, using a __decorators__ attribute in the class body, or something like that. While that approach had an issue or two of its own, it's possible that just going with a single __decorate_class__ method would work out better.
Half-baked idea: Maybe the base class __decorate_class__ would call the
class decorators? Or does that not make sense?
-- --Guido van Rossum (python.org/~guido)
On Mon, Feb 11, 2013 at 2:29 PM, Nick Coghlan
On 12 Feb 2013 07:44, "Guido van Rossum"
wrote: On Mon, Feb 11, 2013 at 12:57 PM, PJ Eby
wrote: On Mon, Feb 11, 2013 at 12:44 PM, Guido van Rossum
wrote:
Hi Nick,
I think this will make a fine addition to the language. I agree that it is superior to the alternatives and fulfills a real (if rare) need.
I only have a few nits/questions/suggestions.
- With PJE, I think __init_class__ should automatically be a class method.
Actually, I didn't say that as such, because I'm not sure how the heck we'd implement that. ;-)
For example, at what point is it converted to a classmethod? Is it going to be a slot method with special C-level handling? Handled by the compiler? What happens if somebody makes it a
The same way that __new__ is automatically a class method.
Actually, isn't it automatically a staticmethod? Oh crap. Now that I'm thinking about it, doesn't this *have* to be a static method, explicitly passing in the class? I mean, otherwise, won't calling super().__init_class__() invoke it on the base class, rather than the current class?
ISTM that EIBTI argues for the __new__/staticmethod approach, especially if you're returning the class (per below)
Let's see what Nick and the implementer say.
I think these are some interesting ideas and it's going to take me a while to digest them and update the PEP :)
Heh. :-)
A few random thoughts:
1. I like the idea of a metaprogramming "howto" that provides advice on choosing a suitable level of metaprogramming (with the default choice being "use existing decorators", then escalating through creating custom decorators all the way to creating custom metaclasses). I don't think the PEP needs to be conditional on writing that, but I will at least add PJE's list to the PEP itself.
Sure.
2. I see the new method as more analogous to__init__ than to__new__, so the __decorate_class__ idea makes me nervous, as it's closer to a __new__ method. Composition gets a *lot* harder when your parent class can switch out the object on you.
Fair enough.
3. I'm trying to avoid any custom magic specific to this method, but making it implicitly a static or class method is fairly easy if we so choose - the standard retrieval code during class creation can just bypass the descriptor machinery, and wrap it in staticmethod or classmethod if it isn't already. Given that __new__ is already implicitly static, it may be easier to follow that precedent here rather than trying to explain why an explicit @classmethod is needed in one case but not the other.
Also fair enough.
4.__class__ is already bound as soon as we have a class object to bind it to, so we can't move it any earlier. However, it's already early enough to let references to it from the new method (including the implied one in zero-arg super) work properly. The black magic that is zero-arg super also deals with PJE's concern about propagating the actual class up the MRO (as it is equivalent to "super(__class__, first_argument)").
So where is it not bound when a metaclass runs? I guess in the metaclass's __new__()? Because in the metaclass's __init__() it should exist IIUC.
5. Implicitly walking the MRO bothers me, as it becomes a special case for people to learn. We don't do that for __init__ or __new__, so I don't think it makes sense to do it here. We can include a recommended structure in the docs, where the first step is calling the parent through. As PJE suggested, a no-op method on type will make that simple and fairly safe (people using a metaclass hierarchy not anchored on type can figure out their own equivalent)
Agreed. -- --Guido van Rossum (python.org/~guido)
11.02.2013 23:29, Nick Coghlan wrote:
3. I'm trying to avoid any custom magic specific to this method, but making it implicitly a static or class method is fairly easy if we so choose - the standard retrieval code during class creation can just bypass the descriptor machinery, and wrap it in staticmethod or classmethod if it isn't already. Given that __new__ is already implicitly static, it may be easier to follow that precedent here rather than trying to explain why an explicit @classmethod is needed in one case but not the other.
Though __new__ is implicitly a *static* rather than a *class* method (so we can use it e.g. by calling object.__new__(MyClass), though -- besides -- in Py3k unbound methods have gone so the difference between static and non-static-and-non-class-methods is smaller than in Py2.x), in case of __init_class__ + super() it'd have to be called: super().__init_class__(__class__) ...and it seems to me a bit awkward. And making it implicitly a *class* rather than a *static* method whould make *impossible* to do calls such as: ExplicitAncestor.__init_class__(ExplicitDescendant) ...though I'm not sure we'd ever need such a call. If not -- implicit *class* method may be a good idea, but if we would? *** On the margin: is that middle underscore in '__init_class__' necessary? We had __metaclass__, not __meta_class__... OK, it's one world, but still we also have __getattr__, __getattribute__, __getitem__, __instancecheck__, __qualname__, __truediv__ etc. (not __get_attr__, __instance_check__ etc.). [I remember only one exception: __reduce_ex__, rather rarely used, and easy to defend against weird __reduceex__]. Wouldn't __initclass__ be readable enough? IMHO it could spare users trouble with remembering special case. Cheers. *j
On 12/02/13 10:56, Jan Kaliszewski wrote:
Wouldn't __initclass__ be readable enough? IMHO it could spare users trouble with remembering special case.
+1 I approve of the colour of this bikeshed. __init_class__ has too many underscores. -- Steven
On Tue, Feb 12, 2013 at 8:35 AM, Guido van Rossum
On Mon, Feb 11, 2013 at 2:29 PM, Nick Coghlan
wrote: 4.__class__ is already bound as soon as we have a class object to bind it to, so we can't move it any earlier. However, it's already early enough to let references to it from the new method (including the implied one in zero-arg super) work properly. The black magic that is zero-arg super also deals with PJE's concern about propagating the actual class up the MRO (as it is equivalent to "super(__class__, first_argument)").
So where is it not bound when a metaclass runs? I guess in the metaclass's __new__()? Because in the metaclass's __init__() it should exist IIUC.
I stand corrected, we don't *quite* bind it as soon as the class object exists - we bind it as soon as the call to the metaclass returns. So even though the class object exists when __init__ runs, it isn't bound to __class__ yet. Given that the class construction machinery treats the metaclass constructor as an opaque callable, trying to get the cell binding to happen between the call to __new__ and the one to __init__ would be rather challenging (Translation: I'm not sure where you would even start...). Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (10)
-
Antoine Pitrou
-
Barry Warsaw
-
Daniel Urban
-
Guido van Rossum
-
Jan Kaliszewski
-
Nick Coghlan
-
PJ Eby
-
Simon Cross
-
Stefan Behnel
-
Steven D'Aprano