__metaclass__ problem
Hi there, first of all I'd like to introduce myself, because I'm new to this list. If I did wrong to post here, please be patient... The reason for my posting is my previous work with __metaclass__ and advice.py, which is nice to use. While working with __metaclass__ I found situations, where I could not explain the resulting output/behaviour of type.__new__ (super(...,...).__new__) using the available documentation. This is why I've been posting bug no. 1164631 - super(...).__new__( ... ) behaves "unexpected". After having checked the C code of 'typeobject.c', I think I might explain the (undocumented) behaviour now (I've been adding a comment to bug no. 1164631). It is - in short - the following: Lines 1602 to 1626 of 'typeobject.c' decide the "winning" metaclass within any class hierarchy. The effekt I noticed - and still consider to be a bug - is introduced in lines 1611 to 1614: The metaclass of the current class will always be the type of class which is at most subclassed from the metatype previously named as "winner". In consequence a programmer only is in control of the "metaclass" of his class, if he decides it to be a subtype of all former metaclasses he used in his class hierarchy, or if he uses the same metaclass as the superclass does. The reasons, why I consider this a bug is: 1. This feature is undocumented. If there is a documentation of it, i might not have found it - or maybe it was not detailed enough to make a programmer (like me: just starting with metaclasses) understand that point. In either cases it would be great to complete the documentation. 2. The class code using __metaclass__, produced by a programmer sets clear directives for the way the resulting Product (the class) has to be and/or to behave. If instead a product even might behave in a completely other way, because it just intends to, this either is a M$ Product ;-) or a bug. 3. If the intention is fixed by a specification, the bug is not the products/programs behaviour then. Instead the bug is a missing Exception, which should inform the programmer of a programming error which violates the specifications. E.g.: TypeError( "__metaclass__ is not of the expected type." ) (or whatever...) 4. Apart the points 1 to 3 and without knowledge of the discussions you probably had while developing this __metaclass__ thing, I'd call it a pitty, if using supertypes of metaclasses as metaclass in a subtype of a class is prohibited. This would make the whole thing somehow incomplete, because it would result in the need of a programmer to know, all metaclasses that have been used by all classes down the mro to 'object'. ...ok - these are my thoughts to what I called the "__metaclass problem" in the subject of this mail. Of course, I'm not into python as long as you are, so: 1. I might have tried something completely stupid 2. I did not take into account the discussions you already might have had about that. Above all I'm not a native english speaker... ;-) ... please don't flame :-) Any feedback (or bugfixes? :-) welcome. Dirk Brenckmann -- "Happy ProMail" bis 24. M�rz: http://www.gmx.net/de/go/promail Zum 6. Geburtstag gibt's GMX ProMail jetzt 66 Tage kostenlos!
Dirk Brenckmann wrote:
In consequence a programmer only is in control of the "metaclass" of his class, if he decides it to be a subtype of all former metaclasses he used in his class hierarchy, or if he uses the same metaclass as the superclass does.
The behaviour is intentional, but you are correct that it is not fully documented in the official documentation [1]. Some of the 'wrinkles' described in Guido's original Python 2.2 essay [2] may need to be transferred to the docs. For instance: """For new-style metaclasses, there is a constraint that the chosen metaclass is equal to, or a subclass of, each of the metaclasses of the bases. Consider a class C with two base classes, B1 and B2. Let's say M = C.__class__, M1 = B1.__class__, M2 = B2.__class__. Then we require issubclass(M, M1) and issubclass(M, M2). (This is because a method of B1 should be able to call a meta-method defined in M1 on self.__class__, even when self is an instance of a subclass of B1.)""" If you are not getting an exception when breaking this rule, my guess would be that your metaclasses are not inheriting from 'type', or else are not invoking type's __new__ method. The logic to trigger the exception lives in type's __new__ method - if that doesn't get invoked, you won't get the exception. (Note that this also addresses your final objection: if you genuinely don't like the behaviour imposed by using 'type' as the metaclass, then don't use 'type' as the metaclass!) Cheers, Nick. [1] http://www.python.org/dev/doc/devel/ref/metaclasses.html [2] http://www.python.org/2.2/descrintro.html#metaclasses -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net
Nick Coghlan wrote:
If you are not getting an exception when breaking this rule, my guess would be that your metaclasses are not inheriting from 'type', or else are not invoking type's __new__ method. The logic to trigger the exception lives in type's __new__ method - if that doesn't get invoked, you won't get the exception.
OK, I actually read the bug report - I think the 'invalid metaclass' exception
should also be getting thrown in the case described there.
Py> class Meta1(type): pass
...
Py> class Meta2(Meta1): pass
...
Py> class MetaA(type): pass
...
Py> class C1(object):
... __metaclass__ = Meta1
...
Py> class C2(C1):
... __metaclass__ = MetaA
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: Error when calling the metaclass bases
metaclass conflict: the metaclass of a derived class must be a (non-strict)
subclass of the metaclasses of all its bases
Py> class C2(C1):
... __metaclass__ = Meta2
...
Py> class C3(C2):
... __metaclass__ = Meta1
...
Py> type(C3)
At 10:11 AM 3/19/05 +1000, Nick Coghlan wrote:
Nick Coghlan wrote:
If you are not getting an exception when breaking this rule, my guess would be that your metaclasses are not inheriting from 'type', or else are not invoking type's __new__ method. The logic to trigger the exception lives in type's __new__ method - if that doesn't get invoked, you won't get the exception.
OK, I actually read the bug report - I think the 'invalid metaclass' exception should also be getting thrown in the case described there.
Py> class Meta1(type): pass ... Py> class Meta2(Meta1): pass ... Py> class MetaA(type): pass ... Py> class C1(object): ... __metaclass__ = Meta1 ... Py> class C2(C1): ... __metaclass__ = Meta2 ... Py> class C3(C2): ... __metaclass__ = Meta1 ... Py> type(C3)
Py> 'Meta1' is NOT a subclass of 'Meta2', yet the exception is not thrown. Instead, the explicitly requested metaclass has been silently replaced with a subclass. I think the OP is justified in calling that 'suprising'.
This is precisely the documented (in Guido's essay) behavior. That is, type.__new__ uses the "most derived" of the explicit metaclass and the __class__ attributes of the bases.
Phillip J. Eby wrote:
At 10:11 AM 3/19/05 +1000, Nick Coghlan wrote:
'Meta1' is NOT a subclass of 'Meta2', yet the exception is not thrown. Instead, the explicitly requested metaclass has been silently replaced with a subclass. I think the OP is justified in calling that 'suprising'.
This is precisely the documented (in Guido's essay) behavior. That is, type.__new__ uses the "most derived" of the explicit metaclass and the __class__ attributes of the bases.
Hmm, you're right. From Guido's essay [1]: """However, if one of the base metaclasses satisfies the constraint (including the explicitly given __metaclass__, if any), the first base metaclass found satisfying the constraint will be used as the metaclass.""" I missed this when re-reading it earlier. Unfortunately, that means an explicitly set __metaclass__ may be ignored, if a base class has a subclass of the explicitly supplied type as its metaclass (the exact problem the OP was objecting to). IOW, I was extremely surprised to learn that "('__metaclass__' in vars(C)) and (type(C) is C.__metaclass__)" is NOT an invariant Python supports for new-style classes (it breaks for C3 in my example). I have no objection to the standard rule when __metaclass__ is not given, or if it is __metaclass__ that satisifies the constraint. It's only when __metaclass__ *is* given, but doesn't meet the constraint, that I would expect an exception rather than for Python to choose to use one of the base metaclasses instead. Anyway, assuming Guido is happy with the status quo, it just means the above text needs to be included when the __metaclass__ documentation is updated. Cheers, Nick. [1] http://www.python.org/2.2/descrintro.html#metaclasses -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net
participants (3)
-
Dirk Brenckmann
-
Nick Coghlan
-
Phillip J. Eby