super() does not work during class initialization
Hi list, while a class is being initialized in a metaclass, it is not always possible to call classmethods of the class, as they might use super(), which in turn uses __class__, which is not initialized yet. I know that this is a known issue, but well, sometimes it even makes sense to fix already known issues... so I wrote a patch that moves the initialization of __class__ into type.__new__, so that one may use super() in a class once it starts existing. It's issue 23722 on the issue tracker. To illustrate what the problem is, the following code raises a RuntimeError: class Meta(type): def __init__(self, name, bases, dict): super().__init__(name, bases, dict) self.f() class A(metaclass=Meta): @classmethod def f(self): super() it works fine with my patch applied. Technically, my patch slightly changes the semantics of python if a metaclass returns something different in its __new__ than what type.__new__ returns. But I could not find any documentation of the current behavior, and also the tests don't test for it, and I consider the current behavior actually buggy. As an example let me give the following simple Singleton metaclass: class Singleton(type): def __new__(cls, name, bases, dict): return super().__new__(cls, name, bases, dict)() class A(metaclass=Singleton): def test(self): assert(isinstance(__class__, type)) A.test() The current python fails the assertion, while with my patch everything is fine, and I personally think __class__ should always actually refer to the class being defined, which means at the minimum that it is actually, well, a class. Greetings Martin
On 21 March 2015 at 00:03, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
The current python fails the assertion, while with my patch everything is fine, and I personally think __class__ should always actually refer to the class being defined, which means at the minimum that it is actually, well, a class.
For folks that haven't looked at the tracker issue: I personally like the change, but it does involve storing the cell object in a dunder-variable in the class namespace while it's being defined (until type.__new__ executes and both populates it and removes it from the class namespace). Since it introduces new behaviour that's visible to Python level code, I've suggested that Martin roll the suggestion into his current PEP 487 (which adds __init_subclass__ to similarly tidy up a few challenges with the way classes are currently initialised). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
For folks that haven't looked at the tracker issue: I personally like the change, but it does involve storing the cell object in a dunder-variable in the class namespace while it's being defined (until type.__new__ executes and both populates it and removes it from the class namespace).
I think I had the same weird feelings like you when writing the code. It feels a bit dodgy to abuse the class namespace to transport information from the compiler to type.__new__. Only when I saw that __qualname__ already does it, I decided it was probably not such a bad idea. The current implementation, setting the __class__ cell at the end of __build_class__ doesn't feel dodgy to me, it simply feels wrong. Nothing of real importance should happen there, as it is just an implementation detail. If we are fearing that we clutter namespace too much, we might call the entries @qualname and @cell, then we are sure they will never mask a user's class member. Most of my bad feelings actually only come from the fact that we don't know what we actually put our entries into, as __prepare__ might return something weird that doesn't do what we expect. Given Eric Snow's comment that class namespaces will be ordered by default soon anyways, we might deprecate __prepare__ altogether, eliminating most of the bad feelings. Speaking of which, instead of having OrderedDict re-implemented in C, maybe we could just change the compiler to leave the order in which things are defined in a class in the class namespace, say as a member __order__? Then we could use plain-old dicts for the class namespace, and we would not slow down class creation (not that it matters much), as determining the order would happen at compile time.
Since it introduces new behaviour that's visible to Python level code, I've suggested that Martin roll the suggestion into his current PEP 487 (which adds __init_subclass__ to similarly tidy up a few challenges with the way classes are currently initialised).
I thought about that, but while I stumbled over this issue while working on PEP 487, it is actually unrelated. I didn't want people to think that this change is needed to implement PEP 487. And while "and now for something completely different" might be a nice Monty Python reference, it is mostly frowned upon in PEPs as per PEP 1. Alternatively, I could write a new PEP. But I actually think that the change is not worth a PEP, because the changes will move the CPython implementation actually closer to what is documented, so I feel it is more a bugfix than an extension. I have no problem to pursue any of the above paths, how do other people feel about it? Greetings Martin
Martin Teichmann wrote:
maybe we could just change the compiler to leave the order in which things are defined in a class in the class namespace, say as a member __order__? Then we could use plain-old dicts for the class namespace, and we would not slow down class creation (not that it matters much), as determining the order would happen at compile time.
I don't think the compiler can determine the order in all cases. Consider: class Spam: if moon_is_full: alpha = 1 beta = 2 else: beta = 2 alpha = 1 -- Greg
On 24 March 2015 at 08:22, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Martin Teichmann wrote:
maybe we could just change the compiler to leave the order in which things are defined in a class in the class namespace, say as a member __order__? Then we could use plain-old dicts for the class namespace, and we would not slow down class creation (not that it matters much), as determining the order would happen at compile time.
I don't think the compiler can determine the order in all cases. Consider:
class Spam:
if moon_is_full: alpha = 1 beta = 2 else: beta = 2 alpha = 1
This is also expected to work in class namespaces: locals()["alpha"] = 1 The language reference suggests it isn't, there's an open tracker issue I filed some time ago to suggest clarifying it but haven't found the time to actually sit down and come up with readable wording: http://bugs.python.org/issue17960 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I don't think the compiler can determine the order in all cases. Consider:
class Spam:
if moon_is_full: alpha = 1 beta = 2 else: beta = 2 alpha = 1
This is also expected to work in class namespaces:
locals()["alpha"] = 1
The language reference suggests it isn't, there's an open tracker issue I filed some time ago to suggest clarifying it but haven't found the time to actually sit down and come up with readable wording: http://bugs.python.org/issue17960
Well, for sure the compiler cannot deduce things happening at runtime, but it still has an order in which things appear at compile time. And for nearly all use cases that's by far enough. We cannot stop people from doing weird things, but unless there is a use case for those weird things, we don't need to support them. And I think that tampering with locals() is not really a good idea. In your issue you mention that you want that so that you can create enums programatically. This is already doable by writing: from enum import Enum from types import new_class def cb(ns): ns.update({"a{}".format(i): i for i in range(100)}) return ns Calc = new_class("Calc", (Enum,), None, cb) I think this is simple enough that we don't need another way of doing it. Btw, going on-topic again, what should I do to get my patch to make super() work during class initialization into python? Greetings Martin
participants (3)
-
Greg Ewing -
Martin Teichmann -
Nick Coghlan