Simpler Customization of Class Creation - PEP 487

Hi List, about a year ago I started a discussion on how to simplify metaclasses, which led to PEP 487. I got some good ideas from this list, but couldn't follow up on this because I was bound in other projects. In short, metaclasses are often not used as they are considered very complicated. Indeed they are, especially if you need to use two of them at the same time in a multiple inheritance context. Most metaclasses, however, serve only some of the following three purposes: a) run some code after a class is created b) initialize descriptors of a class or c) keep the order in which class attributes have been defined. PEP 487 now proposes to put a metaclass into the standard library, which can be used for all those three purposes. If now libraries start to use this metaclass, we won't need any metaclass mixing anymore. What has changed since the last time I posted PEP 487? Firstly, I re-wrote large parts of the PEP to make it easier to read. Those who liked the old text, that's still existing in PEP 422. Secondly, I modified the proposal following suggestions from this list: I added the descriptor initialization (purpose b)), as this was considered particularly useful, even if it could in principle be done using purpose a) from above. The order-keeping of the class attributes is the leftover from a much more ambitious previous idea that would have allowed for custom namespaces during class creation. But this additional feature would have rendered the most common usecase - getting the order of attributes - much more complicated, so I opted for usability over flexibility. I have put the new version of the PEP here: https://github.com/tecki/metaclasses/blob/pep487/pep-0487.txt and also added it to this posting. An implementation of this PEP can be found at: https://pypi.python.org/pypi/metaclass Greetings Martin PEP: 487 Title: Simpler customisation of class creation Version: $Revision$ Last-Modified: $Date$ Author: Martin Teichmann <lkb.teichmann@gmail.com>, Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Feb-2015 Python-Version: 3.6 Post-History: 27-Feb-2015, 5-Feb-2016 Replaces: 422 Abstract ======== Currently, customising class creation requires the use of a custom metaclass. This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts. This PEP proposes to instead support a wide range of customisation scenarios through a new ``__init_subclass__`` hook in the class body, a hook to initialize descriptors, and a way to keep the order in which attributes are defined. Those hooks should at first be defined in a metaclass in the standard library, with the option that this metaclass eventually becomes the default ``type`` metaclass. The new mechanism should be easier to understand and use than implementing a custom metaclass, and thus should provide a gentler introduction to the full power Python's metaclass machinery. Background ========== Metaclasses are a powerful tool to customize class creation. They have, however, the problem that there is no automatic way to combine metaclasses. If one wants to use two metaclasses for a class, a new metaclass combining those two needs to be created, typically manually. This need often occurs as a surprise to a user: inheriting from two base classes coming from two different libraries suddenly raises the necessity to manually create a combined metaclass, where typically one is not interested in those details about the libraries at all. This becomes even worse if one library starts to make use of a metaclass which it has not done before. While the library itself continues to work perfectly, suddenly every code combining those classes with classes from another library fails. Proposal ======== While there are many possible ways to use a metaclass, the vast majority of use cases falls into just three categories: some initialization code running after class creation, the initalization of descriptors and keeping the order in which class attributes were defined. Those three use cases can easily be performed by just one metaclass. If this metaclass is put into the standard library, and all libraries that wish to customize class creation use this very metaclass, no combination of metaclasses is necessary anymore. The three use cases are achieved as follows: 1. The metaclass contains an ``__init_subclass__`` hook that initializes all subclasses of a given class, 2. the metaclass calls an ``__init_descriptor__`` hook for all descriptors defined in the class, and 3. an ``__attribute_order__`` tuple is left in the class in order to inspect the order in which attributes were defined. For ease of use, a base class ``SubclassInit`` is defined, which uses said metaclass and contains an empty stub for the hook described for use case 1. As an example, the first use case looks as follows:: class SpamBase(SubclassInit): # this is implicitly a @classmethod def __init_subclass__(cls, **kwargs): # This is invoked after a subclass is created, but before # explicit decorators are called. # The usual super() mechanisms are used to correctly support # multiple inheritance. # **kwargs are the keyword arguments to the subclasses' # class creation statement super().__init_subclass__(cls, **kwargs) class Spam(SpamBase): pass # the new hook is called on Spam The base class ``SubclassInit`` contains an empty ``__init_subclass__`` method which serves as an endpoint for cooperative multiple inheritance. Note that this method has no keyword arguments, meaning that all methods which are more specialized have to process all keyword arguments. This general proposal is not a new idea (it was first suggested for inclusion in the language definition `more than 10 years ago`_, and a similar mechanism has long been supported by `Zope's ExtensionClass`_), but the situation has changed sufficiently in recent years that the idea is worth reconsidering for inclusion. The second part of the proposal adds an ``__init_descriptor__`` initializer for descriptors. Descriptors are defined in the body of a class, but they do not know anything about that class, they do not even know the name they are accessed with. They do get to know their owner once ``__get__`` is called, but still they do not know their name. This is unfortunate, for example they cannot put their associated value into their object's ``__dict__`` under their name, since they do not know that name. This problem has been solved many times, and is one of the most important reasons to have a metaclass in a library. While it would be easy to implement such a mechanism using the first part of the proposal, it makes sense to have one solution for this problem for everyone. To give an example of its usage, imagine a descriptor representing weak referenced values (this is an insanely simplified, yet working example):: import weakref class WeakAttribute: def __get__(self, instance, owner): return instance.__dict__[self.name] def __set__(self, instance, value): instance.__dict__[self.name] = weakref.ref(value) # this is the new initializer: def __init_descriptor__(self, owner, name): self.name = name The third part of the proposal is to leave a tuple called ``__attribute_order__`` in the class that contains the order in which the attributes were defined. This is a very common usecase, many libraries use an ``OrderedDict`` to store this order. This is a very simple way to achieve the same goal. Key Benefits ============ Easier inheritance of definition time behaviour ----------------------------------------------- Understanding Python's metaclasses requires a deep understanding of the type system and the class construction process. This is legitimately seen as challenging, due to the need to keep multiple moving parts (the code, the metaclass hint, the actual metaclass, the class object, instances of the class object) clearly distinct in your mind. Even when you know the rules, it's still easy to make a mistake if you're not being extremely careful. Understanding the proposed implicit class initialization hook only requires ordinary method inheritance, which isn't quite as daunting a task. The new hook provides a more gradual path towards understanding all of the phases involved in the class definition process. Reduced chance of metaclass conflicts ------------------------------------- One of the big issues that makes library authors reluctant to use metaclasses (even when they would be appropriate) is the risk of metaclass conflicts. These occur whenever two unrelated metaclasses are used by the desired parents of a class definition. This risk also makes it very difficult to *add* a metaclass to a class that has previously been published without one. By contrast, adding an ``__init_subclass__`` method to an existing type poses a similar level of risk to adding an ``__init__`` method: technically, there is a risk of breaking poorly implemented subclasses, but when that occurs, it is recognised as a bug in the subclass rather than the library author breaching backwards compatibility guarantees. A path of introduction into Python ================================== Most of the benefits of this PEP can already be implemented using a simple metaclass. For the ``__init_subclass__`` hook this works all the way down to Python 2.7, while the attribute order needs Python 3.0 to work. Such a class has been `uploaded to PyPI`_. The only drawback of such a metaclass are the mentioned problems with metaclasses and multiple inheritance. Two classes using such a metaclass can only be combined, if they use exactly the same such metaclass. This fact calls for the inclusion of such a class into the standard library, let's call it ``SubclassMeta``, with the base class using it called ``SubclassInit``. Once all users use this standard library metaclass, classes from different packages can easily be combined. But still such classes cannot be easily combined with other classes using other metaclasses. Authors of metaclasses should bear that in mind and inherit from the standard metaclass if it seems useful for users of the metaclass to add more functionality. Ultimately, if the need for combining with other metaclasses is strong enough, the proposed functionality may be introduced into Python's ``type``. Those arguments strongly hint to the following procedure to include the proposed functionality into Python: 1. The metaclass implementing this proposal is put onto PyPI, so that it can be used and scrutinized. 2. Once the code is properly mature, it can be added to the Python standard library. There should be a new module called ``metaclass`` which collects tools for metaclass authors, as well as a documentation of the best practices of how to write metaclasses. 3. If the need of combining this metaclass with other metaclasses is strong enough, it may be included into Python itself. While the metaclass is still in the standard library and not in the language, it may still clash with other metaclasses. The most prominent metaclass in use is probably ABCMeta. It is also a particularly good example for the need of combining metaclasses. For users who want to define a ABC with subclass initialization, we should support a ``ABCSubclassInit`` class, or let ABCMeta inherit from this PEP's metaclass. Extensions written in C or C++ also often define their own metaclass. It would be very useful if those could also inherit from the metaclass defined here, but this is probably not possible. New Ways of Using Classes ========================= This proposal has many usecases like the following. In the examples, we still inherit from the ``SubclassInit`` base class. This would become unnecessary once this PEP is included in Python directly. Subclass registration --------------------- Especially when writing a plugin system, one likes to register new subclasses of a plugin baseclass. This can be done as follows:: class PluginBase(SubclassInit): subclasses = [] def __init_subclass__(cls, **kwargs): super().__init_subclass__(**kwargs) cls.subclasses.append(cls) One should note that this also works nicely as a mixin class. Trait descriptors ----------------- There are many designs of Python descriptors in the wild which, for example, check boundaries of values. Often those "traits" need some support of a metaclass to work. This is how this would look like with this PEP:: class Trait: def __get__(self, instance, owner): return instance.__dict__[self.key] def __set__(self, instance, value): instance.__dict__[self.key] = value def __init_descriptor__(self, owner, name): self.key = name class Int(Trait): def __set__(self, instance, value): # some boundary check code here super().__set__(instance, value) Rejected Design Options ======================= Calling the hook on the class itself ------------------------------------ Adding an ``__autodecorate__`` hook that would be called on the class itself was the proposed idea of PEP 422. Most examples work the same way or even better if the hook is called on the subclass. In general, it is much easier to explicitly call the hook on the class in which it is defined (to opt-in to such a behavior) than to opt-out, meaning that one does not want the hook to be called on the class it is defined in. This becomes most evident if the class in question is designed as a mixin: it is very unlikely that the code of the mixin is to be executed for the mixin class itself, as it is not supposed to be a complete class on its own. The original proposal also made major changes in the class initialization process, rendering it impossible to back-port the proposal to older Python versions. Other variants of calling the hook ---------------------------------- Other names for the hook were presented, namely ``__decorate__`` or ``__autodecorate__``. This proposal opts for ``__init_subclass__`` as it is very close to the ``__init__`` method, just for the subclass, while it is not very close to decorators, as it does not return the class. Requiring an explicit decorator on ``__init_subclass__`` -------------------------------------------------------- One could require the explicit use of ``@classmethod`` on the ``__init_subclass__`` decorator. It was made implicit since there's no sensible interpretation for leaving it out, and that case would need to be detected anyway in order to give a useful error message. This decision was reinforced after noticing that the user experience of defining ``__prepare__`` and forgetting the ``@classmethod`` method decorator is singularly incomprehensible (particularly since PEP 3115 documents it as an ordinary method, and the current documentation doesn't explicitly say anything one way or the other). Defining arbitrary namespaces ----------------------------- PEP 422 defined a generic way to add arbitrary namespaces for class definitions. This approach is much more flexible than just leaving the definition order in a tuple. The ``__prepare__`` method in a metaclass supports exactly this behavior. But given that effectively the only use cases that could be found out in the wild were the ``OrderedDict`` way of determining the attribute order, it seemed reasonable to only support this special case. The metaclass described in this PEP has been designed to be very simple such that it could be reasonably made the default metaclass. This was especially important when designing the attribute order functionality: This was a highly demanded feature and has been enabled through the ``__prepare__`` method of metaclasses. This method can be abused in very weird ways, making it hard to correctly maintain this feature in CPython. This is why it has been proposed to deprecated this feature, and instead use ``OrderedDict`` as the standard namespace, supporting the most important feature while dropping most of the complexity. But this would have meant that ``OrderedDict`` becomes a language builtin like dict and set, and not just a standard library class. The choice of the ``__attribute_order__`` tuple is a much simpler solution to the problem. A more ``__new__``-like hook ---------------------------- In PEP 422 the hook worked more like the ``__new__`` method than the ``__init__`` method, meaning that it returned a class instead of modifying one. This allows a bit more flexibility, but at the cost of much harder implementation and undesired side effects. History ======= This used to be a competing proposal to PEP 422 by Nick Coughlan and Daniel Urban. It shares both most of the PEP text and proposed code, but has major differences in how to achieve its goals. In the meantime, PEP 422 has been withdrawn favouring this approach. References ========== .. _published code: http://mail.python.org/pipermail/python-dev/2012-June/119878.html .. _more than 10 years ago: http://mail.python.org/pipermail/python-dev/2001-November/018651.html .. _Zope's ExtensionClass: http://docs.zope.org/zope_secrets/extensionclass.html .. _uploaded to PyPI: https://pypi.python.org/pypi/metaclass Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

Two small things. On Friday, February 5, 2016 1:21 PM, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
How is c) not served by just defining a __prepare__ method that returns an OrderedDict? That seems easier to use than a separate dict and __attribute_order__ tuple. You later mention that many people already do this, but don't explain what's wrong with it or why your idea is better. It's not like a one-line __prepare__ is difficult to write, or like OrderedDict is too slow (especially in 3.6), so you're just adding a cost (less convenient to use) for no benefit that I can see. Also:
It sounds like you're trying to have it both ways here: it seems like if your proposal really is better because it works with Python 2.7 rather than just 3.x, then your proposal being in the stdlib is probably a bad idea, because it will mislead rather than help people trying to write version-straddling code.

Hi Andrew, Hi List, thanks for the quick review. You brought up very interesting points which should be discussed here.
In my implementation this is exactly what I do: I create a OrderedDict in __prepare__. The question is just: how dow we get that information to the user? The __prepare__d namespace only exists during class creation, type.__new__ cripples it into a normal dict. (that is done here: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L2343) An older version of the PEP introduced a namespace parameter to the __init_subclass__ hook which would still get the OrderedDict. This is a neat way of doing so, but I did not want to exclude the option that this metaclass once becomes the default metaclass. And then suddenly OrderedDict turns from just another standard library class into a type built into the language. Maybe this is not a big deal, maybe it is, what are other people thinking about that?
It sounds like you're trying to have it both ways here: it seems like if your proposal really is better because it works with Python 2.7 rather than just 3.x, then your proposal being in the stdlib is probably a bad idea, because it will mislead rather than help people trying to write version-straddling code.
This is a very complicated issue. In order for my proposal to make any sense, everyone has to use exactly the same metaclass, not just a look-alike. I don't see another way to do that than putting it into the standard library, or directly into the Python language. While Python 2 compatibility is not such a big deal anymore (this is 2016, and I figure 2015 really has been the year of Python 3), the problem is that this proposal is targetting authors of libraries, not just applications. Libraries are, for a good reason, very reluctant to drop backwards compatibility. So trying to convince them to do something which is not backwards portable won't fly. My idea is that libraries just use the PyPI implementation of this PEP, which will use the standard library implementation once it is existing. But library developers are also very reluctant to add any additional dependencies, my module is certainly no exception. Instead they have the option to copy my library into theirs. That means that they won't have the benefit of compatibility with other libraries while the metaclass is not in the standard library, but once it is, they are automatically compatible with the rest of the world. The best option for compatiblity probably is that this metaclass is also added to compatibility libraries like six, so that different libraries are at least compatible if they use the same compatibility library (and are automatically compatible with everyone else once the metaclass is in the standard library). Greetings Martin

On Feb 6, 2016, at 03:02, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
OK, now this makes sense. The PyPI module does something like this: try: from types import Meta except ImportError: pass So, in 2.7 or 3.5, this has no effect, and you get the Meta defined in the module, but if 3.6 or 3.7 adds types.Meta, people using the PyPI module get the stdlib type, so they get the benefits of a shared metaclass. And if 3.7 or 3.8 replaces type with your metaclass, and just has types.Meta = type, the PyPI module now gets the builtin if possible, the stdlib type as a fallback, and the PyPI type as a double-fallback. Assuming that understanding was right, this does seem like the best solution to the problem--not perfect, but as good as anything could possibly be. One thing on the six idea: if some libraries get the type from six and others from your module, they'll have incompatible metaclasses for no good reason (when used on 3.5 and earlier). It seems like if this is going to end up in six, it should be in six as soon as possible, and direct use of the separate library should be strongly discouraged.

On 7 February 2016 at 03:47, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
One thing on the six idea: if some libraries get the type from six and others from your module, they'll have incompatible metaclasses for no good reason (when used on 3.5 and earlier). It seems like if this is going to end up in six, it should be in six as soon as possible, and direct use of the separate library should be strongly discouraged.
The daisy chain of fallbacks approach holds - six would just become another stepping stone on the path to the standard library (and eventually the type builtin). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 6 February 2016 at 07:20, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
Thanks for taking this up again!
I like this updated approach in general - more detailed comments are inline below.
I have put the new version of the PEP here:
https://github.com/tecki/metaclasses/blob/pep487/pep-0487.txt
I also just pushed this version to the PEPs repo.
While you do cover it later, it's worth mentioning up front that there's a reasonable case to be made that type should just work this way by default. However, changing type *again* is difficult if we decide we made a mistake, so the currently proposed plan is: 1. Introduce a PyPI package (metaclass) for initial iteration on the API 2. Introduce a stdlib metaclass as a provisional in Python 3.6 3. Consider this as possible default behaviour for type in Python 3.7. If type changes, the old type name will just become a legacy alias for type Steps 2 & 3 would be similar to the way the set datatype was first introduced as sets.Set, and only later made a builtin type (with a slightly different API) based on wider experience with the sets module. Step 2 mostly serves as a signalling mechanism that unequivocally blesses the PyPI module created in 1 as setting the future direction of the default behaviour of the builtin "type".
"__init_descriptor__" confused me, as it wasn't clear to me until much later in the PEP that it's a proposed addition to the *descriptor* API, rather than something you implement on the metaclass you're defining. It's also not really restricted to descriptors - as a new hook, it could be implemented by any attribute, regardless of whether it supported any other part of the descriptor protocol. As such, a possible way to go here is to instead call this a new "attribute ownership protocol", and make the hook name "__set_owner__". It should also be called out that implementations of __set_owner__ will need to handle the case of attribute re-use, and handle things appropriately if the owner has already been set (e.g. in the simplest case, by throwing a RuntimeError indicating that shared ownership isn't supported).
You should specify the metaclass name here as well. Given the three-fold difference in behaviour, naming the new metaclass and class after only one of those behaviours seems misleading. On the other hand, "OrderedAttributeOwningSubclassInitialisingMeta" would be silly, so it might be worth instead calling them something like "ProvisionalMeta" and "ProvisionalClass" - that is, you're opting in to the provisional future behaviour of the "type" and "object" builtins, without specifying exactly what the current differences are. I'd also suggest putting the new types in the existing "types" module, rather than defining a new module for them (aside from the module on PyPI).
This example isn't particularly clear, since the __init_subclass__ doesn't *do* anything. An example that preserves the class keyword arguments as an attribute may be more comprehensible:
This is the part I suggest renaming as an "attribute ownership protocol", with the hook name as "__set_owner__".
Similar to the __subclass_init__ case, a more meaningful usage example may help here, such as allowing owning classes to define a hook that gets called when the weak reference goes away, while still having useful default behaviour. For example (untested): class WeakAttribute: def __init__(self): self._owner = None self._name = None self._callback = None def __get__(self, instance, owner): if instance is None: return self return instance.__dict__[self.name] def __set__(self, instance, value): instance.__dict__[self.name] = weakref.proxy(value, self._callback) def __set_owner__(self, owner, attr): if self._owner is not None: raise RuntimeError("{!r} already owned by {!r}".format(self, self._owner()) self._owner = weakref.ref(owner) self._name = attr callback = getattr(owner, "attribute_collected", None) if callback is not None: self._callback = functools.partial(callback, attr) class Example(metaclass.SubclassInit): proxy = WeakAttribute() def attribute_collected(self, attr): print("{} was garbage collected".format())
This should spell out the underlying mechanism here - the new metaclass will *also* use OrderedDict to preserve the order during class construction, so the extra bit the provisional metaclass adds over the DIY __prepare__ method is taking the original ordered dicts keys and saving them in an attribute, while __dict__ itself will remain an ordinary unordered dict.
This should explain that the difference between this and just calling PluginBase.__subclasses__() is that this example flattens the inheritance tree into a simple list.
This doesn't show the descriptor subclass making use of the state set up by the new hook on the parent class, so the subclass ends up making the example more confusing, rather than improving it.
This should be elaborated on: * we *do* want to change the default behaviour of type in the future * we *also* want to be able to validate the usability of those changes before we make them Unlike PEP 422, this PEP lets us take two preliminary steps (library on PyPI, provisional API in the standard library) *before* making any changes to type itself.
Coghlan :)
Daniel Urban. It shares both most of the PEP text and proposed code,
I think the code and text have diverged significantly now, but the major shared aspect was always common *goals*, rather than any of the technical details. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi Nick, Hi List, Thanks for the good comments, I tried to incorporate all of them into the new version of my PEP, which can be found here: https://github.com/tecki/peps/blob/pep487/pep-0487.txt I'm not sure whether it is a good idea to post the entire PEP here all the time, but actually your inline comments were a very helpful technique. I'm not sure how to proceed. Just some comment about the changes I made:
I had already figured that SubclassInit is not such a great naming idea, but couldn't find anything better. Calling something Provisional gives bad Karma, nothing sticks around longer than something marked "Provisional". So, I propose to call the metaclass "types.Type" and the base class "types.Object". Maybe we can even go for "types.type" and "types.object"? I thought this would be a bit, well, blunt, so I chose the former.
While I think that you're correct that a better example would be a good idea, you example for me hints to that __set_owner__ does an initialization of the attribute at object creation time, but it does it at class creation time. I'll sleep over it, maybe I find a nice example. It's also important to note that the new thing about the metaclass is not that it supplies the attribute with the owner, but that it tell the attribute about its name. The owner was already shipped to __get__ and __set__, but not the name within that class. I am not aware of a way to find out the attribute name from within a descriptor. (That's not true: a year ago some ideas how to do that were posted here, but they used undocumented internal features of CPython, not necessarily a reasonable way). This is why the name __set_owner__ also might be misleading, but __set_name__ also doesn't sound right to me, any better calls? For the meantime I stick with __set_owner__.
Greetings Martin Teichmann

On Mon, Feb 08, 2016 at 10:28:17AM +0100, Martin Teichmann wrote:
It would cause backwards-incompatibility, but would it not be an option to call __get__ and __set__ (and __del__) with an extra parameter: the name of the attribute that is supposed to be accessed? (This can probably be worked around by inspecting the signature of __get__/... to see how many arguments it takes). Because right now, if the descriptor itself does not maintain any state, it is possible to assign the same descriptor to multiple class attributes, maybe even to multiple classes. (On the other hand, in that case it would probably not need `__set_owner__` either). The question to me is: can __set_owner__ give us something that can not be done by adding a name parameter to __get__/... . The weakref proxy (with callback) looks to be one of those cases, but is not really: def __set__(self, owner, value, attr): def _cb(arg): callback = getattr(owner, 'attribute_collected') if callback is note None: callback(attr) instance.__dict__[self.name] = weakref.proxy(value, callback) Calling desc.__set__(owner, value, attr) instead of desc.__set__(owner, value) can probably also be implemented by a custom metaclass which walks over all the values in the generated namespace, and replaces a descriptor with a NameCallingDescriptor(val, attr). for attr, val in namespace.items(): if is_descriptor(val): if descriptor_wants_attribute_name(val): namespace[attr] = NameCallingDescriptor(val, attr) Is there anything that __set_owner__ brings that can not be done by changing __get__/__set__/...?

Hi Sjoerd, Hi List,
I think the backwards-incompatibility is a show stopper for this idea. I actually think it's a good idea, but too late. I checked PEP 252, which invented __get__, and it also states that every attribute should have a __name__ and __objclass__. But it is silent on how an attribute should acquire that. __set_owner__ fills this silence. We could try to actually implement PEP 252 and set __name__ and __objclass__. I guess this would give a big backwards compatibility headache. We could require descriptors to inherit from a class types.Descriptor, and set the __name__ and __objclass__ only on them. That's a little less flexible than the current proposal.
Well, __set_owner__ gets called much earlier. This means we might avoid having to write some lazy initialization code. I know, that is a weak argument, but I think it's actually nicer to have a __set_owner__ method over having an overloaded __get__. But this "nice" certainly is a question of taste. Greetings Martin

Could you help me better understand the 3 purposes you described in the context of multiple metaclasses? On Fri, Feb 5, 2016 at 4:20 PM Martin Teichmann <lkb.teichmann@gmail.com> wrote:
This new ProvisionalMeta will allow multiple inheritance of metaclasses using standard left-to-right priority (or the MRO for more depth). In the case of two metaclasses, the code from Left will run immediately after the class is created and the code from Right will go next. Right? (joking. I should say "Correct?") If the user doesn't want to learn about the metaclasses inside the hierarchy, as per your explanation of the trouble with metaclass conflicts, will they be able to make the decision about which should be Left and which should be Right? What about cases where Left and Right interfere with each other? This reminds me of version control merge conflicts. It's frustrating that the automatic merge fails, but it's better than allowing mistakes. Or am I veering too much towards Java-land?

No. The upshot is: metaclasses are complicated, especially if you want to combine two of them. But most uses of the usecases fall into one of the three mentioned categories. So we just write one metaclass that serves those use cases, everyone can simply use the very same metaclass, so the are no conflicts anymore as there *is* only one metaclass. Eventually, this one metaclass shall become the builtin type, and so we don't need metaclasses anymore in most usecases. The proposed metaclass (current working title: types.Type) will call a hook for users to customize its behavior. This hook will use the normal Python cooperative multiple inheritance scheme, which also has its very own problems, but known and solvable problems. The current metaclasses, on the other hand, once you use two of them open a complete new box of pandora. Greetings Martin

Hi List, I thought it was a good idea to show that projects can benefit from PEP 487 out there in the real world. As a showcase example I chose IPython traitlets, and it was a nice success. With only little modifications to the traitlets code it uses a PEP 487 style metaclass, and all tests pass unmodified with both Python 2 and 3. You can find the PEP 487 style IPython traitlets here: https://github.com/tecki/traitlets/tree/pep487 Greetings Martin

Nice. I am glad to see movement and practical usage in that area. Metaclasses have been a headache for a long time. We introduced them into our production for a short period of time just to remove them later. As the PEP expected, ABCMeta was the issue. We will see if the proposal will solve that. On 15.02.2016 09:31, Martin Teichmann wrote:

On Fri, Feb 5, 2016 at 2:20 PM, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
Note that I've already been working on making OrderedDict the default class definition namespace type. [1] The patch to do so is actually super simple (if I remember correctly). This was actually one of my main motivations for re-implementing OrderedDict in C (the other was preserving the order of **kwargs). Also note that we're thinking along the same lines regarding __attribute_order__. :) In the tracker issue I referenced you'll see that I proposed roughly the same thing, calling it __definition_order__ (which I prefer as the spelling). I've brought up the idea a couple of times on this list and got a positive response (or so I remember <wink>) and was still planning on pursuing the matter in time for Python 3.7. My point is: this part of your proposal stands on its own and is already being worked on (even if slowly). FYI, one problem with bundling related ideas in a PEP is that the concept of partial acceptance isn't a natural part of the PEP process. There have been a few PEPs where part of the proposal made the cut but didn't go anywhere because the whole PEP was rejected. Anyway, keep the above in mind as it has bearing on your PEP. -eric [1] http://bugs.python.org/issue24254

Hi Eric, Hi List,
I vaguely remembered your work when I was writing my PEP, but couldn't find the discussion about it anymore, thanks for pointing me to it, I'll update that in my PEP.
I agree, __definition_order__ sounds better. I'm not a native speaker, so I need some help on naming things properly...
I am not very attached to the idea of __definition_order__, so if the general consensus here is to take it out, I will do that. In an old version of the PEP (i.e. PEP 422) there was a way to open the full __prepare__ scheme through the new metaclass. This was certainly bundling too many ideas in one PEP, so I took that out. I left the __definition_order__ in as it seemed a very simple idea to me, especially because once the new metaclass becomes the standard metaclass, this can be implemented without OrderedDict, and thus we don't have the need to make OrderedDict a builtin type. I'm not sure how the general feeling is here on the list: should I * take out the __definition_order__ and later do the same with an OrderedDict, * or should I leave that in because making OrderedDict a builtin is too much a burden? I'm looking forward to comments on that! Btw, how I want to proceed: I am currently looking how other projects could benefit from PEP 487 (beside my owns...), and got good feedback from IPython. Then I will implement a version for the standard library. As a reminder, there is already a version on PyPI: https://pypi.python.org/pypi/metaclass Greetings Martin

On Wed, Mar 2, 2016 at 12:43 PM, Martin Teichmann <lkb.teichmann@gmail.com> wrote:
I agree, __definition_order__ sounds better. I'm not a native speaker, so I need some help on naming things properly...
No worries. Glad we agree. :)
I am not very attached to the idea of __definition_order__, so if the general consensus here is to take it out, I will do that.
I'm not suggesting that you take it out, necessarily. I just wanted to make sure you could make an informed decision.
I don't think there's any harm in leaving it in, regardless of what I'm trying to achieve. It would be sensible to reference issue24254 in the PEP as there's overlap (even if not in the implementation).
* or should I leave that in because making OrderedDict a builtin is too much a burden?
As of 3.6 OrderedDict *is* a builtin type (just not part of the builtins module). So there's zero burden. :)
Good plan.
As a reminder, there is already a version on PyPI:
Cool. I'll take a look. -eric
participants (7)
-
Andrew Barnert
-
Eric Snow
-
Martin Teichmann
-
Michael Selik
-
Nick Coghlan
-
Sjoerd Job Postmus
-
Sven R. Kunze