[Python-checkins] peps: Create PEP 487 from Martin Teichmann's submission
chris.angelico
python-checkins at python.org
Fri Feb 27 20:08:58 CET 2015
https://hg.python.org/peps/rev/c45defff808a
changeset: 5716:c45defff808a
user: Chris Angelico <rosuav at gmail.com>
date: Sat Feb 28 06:07:44 2015 +1100
summary:
Create PEP 487 from Martin Teichmann's submission
files:
pep-0487.txt | 485 +++++++++++++++++++++++++++++++++++++++
1 files changed, 485 insertions(+), 0 deletions(-)
diff --git a/pep-0487.txt b/pep-0487.txt
new file mode 100644
--- /dev/null
+++ b/pep-0487.txt
@@ -0,0 +1,485 @@
+PEP: 487
+Title: Simpler customisation of class creation
+Version: $Revision$
+Last-Modified: $Date$
+Author: Martin Teichmann <lkb.teichmann at gmail.com>,
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 27-Feb-2015
+Python-Version: 3.5
+Post-History: 27-Feb-2015
+Replaces: 422
+
+
+Abstract
+========
+
+Currently, customising class creation requires the use of a custom metaclass.
+This custom metaclass then persists for the entire lifecycle of the class,
+creating the potential for spurious metaclass conflicts.
+
+This PEP proposes to instead support a wide range of customisation
+scenarios through a new ``namespace`` parameter in the class header, and
+a new ``__init_subclass__`` hook in the class body.
+
+The new mechanism should be easier to understand and use than
+implementing a custom metaclass, and thus should provide a gentler
+introduction to the full power Python's metaclass machinery.
+
+
+Connection to other PEP
+=======================
+
+This is a competing proposal to PEP 422 by Nick Coughlan and Daniel Urban.
+It shares both most of the PEP text and proposed code, but has major
+differences in how to achieve its goals.
+
+Background
+==========
+
+For an already created class ``cls``, the term "metaclass" has a clear
+meaning: it is the value of ``type(cls)``.
+
+*During* class creation, it has another meaning: it is also used to refer to
+the metaclass hint that may be provided as part of the class definition.
+While in many cases these two meanings end up referring to one and the same
+object, there are two situations where that is not the case:
+
+* If the metaclass hint refers to an instance of ``type``, then it is
+ considered as a candidate metaclass along with the metaclasses of all of
+ the parents of the class being defined. If a more appropriate metaclass is
+ found amongst the candidates, then it will be used instead of the one
+ given in the metaclass hint.
+* Otherwise, an explicit metaclass hint is assumed to be a factory function
+ and is called directly to create the class object. In this case, the final
+ metaclass will be determined by the factory function definition. In the
+ typical case (where the factory functions just calls ``type``, or, in
+ Python 3.3 or later, ``types.new_class``) the actual metaclass is then
+ determined based on the parent classes.
+
+It is notable that only the actual metaclass is inherited - a factory
+function used as a metaclass hook sees only the class currently being
+defined, and is not invoked for any subclasses.
+
+In Python 3, the metaclass hint is provided using the ``metaclass=Meta``
+keyword syntax in the class header. This allows the ``__prepare__`` method
+on the metaclass to be used to create the ``locals()`` namespace used during
+execution of the class body (for example, specifying the use of
+``collections.OrderedDict`` instead of a regular ``dict``).
+
+In Python 2, there was no ``__prepare__`` method (that API was added for
+Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__``
+attribute, and the class creation process would extract that value from the
+class namespace to use as the metaclass hint. There is `published code`_ that
+makes use of this feature.
+
+Another new feature in Python 3 is the zero-argument form of the ``super()``
+builtin, introduced by PEP 3135. This feature uses an implicit ``__class__``
+reference to the class being defined to replace the "by name" references
+required in Python 2. Just as code invoked during execution of a Python 2
+metaclass could not call methods that referenced the class by name (as the
+name had not yet been bound in the containing scope), similarly, Python 3
+metaclasses cannot call methods that rely on the implicit ``__class__``
+reference (as it is not populated until after the metaclass has returned
+control to the class creation machinery).
+
+Finally, when a class uses a custom metaclass, it can pose additional
+challenges to the use of multiple inheritance, as a new class cannot
+inherit from parent classes with unrelated metaclasses. This means that
+it is impossible to add a metaclass to an already published class: such
+an addition is a backwards incompatible change due to the risk of metaclass
+conflicts.
+
+
+Proposal
+========
+
+This PEP proposes that a new mechanism to customise class creation be
+added to Python 3.5 that meets the following criteria:
+
+1. Integrates nicely with class inheritance structures (including mixins and
+ multiple inheritance),
+2. Integrates nicely with the implicit ``__class__`` reference and
+ zero-argument ``super()`` syntax introduced by PEP 3135,
+3. Can be added to an existing base class without a significant risk of
+ introducing backwards compatibility problems, and
+4. Restores the ability for class namespaces to have some influence on the
+ class creation process (above and beyond populating the namespace itself),
+ but potentially without the full flexibility of the Python 2 style
+ ``__metaclass__`` hook.
+
+Those goals can be achieved by adding two functionalities:
+
+1. A ``__init_subclass__`` hook that initializes all subclasses of a
+ given class, and
+2. A new keyword parameter ``namespace`` to the class creation statement,
+ that gives an initialization of the namespace.
+
+As an example, the first proposal looks as follows::
+
+ class SpamBase:
+ # this is implicitly a @classmethod
+ def __init_subclass__(cls, ns, **kwargs):
+ # This is invoked after a subclass is created, but before
+ # explicit decorators are called.
+ # The usual super() mechanisms are used to correctly support
+ # multiple inheritance.
+ # ns is the classes namespace
+ # **kwargs are the keyword arguments to the subclasses'
+ # class creation statement
+ super().__init_subclass__(cls, ns, **kwargs)
+
+ class Spam(SpamBase):
+ pass
+ # the new hook is called on Spam
+
+To simplify the cooperative multiple inheritance case, ``object`` will gain
+a default implementation of the hook that does nothing::
+
+ class object:
+ def __init_subclass__(cls, ns):
+ pass
+
+Note that this method has no keyword arguments, meaning that all
+methods which are more specialized have to process all keyword
+arguments.
+
+This general proposal is not a new idea (it was first suggested for
+inclusion in the language definition `more than 10 years ago`_, and a
+similar mechanism has long been supported by `Zope's ExtensionClass`_),
+but the situation has changed sufficiently in recent years that
+the idea is worth reconsidering for inclusion.
+
+The second part of the proposal is to have a ``namespace`` keyword
+argument to the class declaration statement. If present, its value
+will be called without arguments to initialize a subclasses
+namespace, very much like a metaclass ``__prepare__`` method would
+do.
+
+In addition, the introduction of the metaclass ``__prepare__`` method
+in PEP 3115 allows a further enhancement that was not possible in
+Python 2: this PEP also proposes that ``type.__prepare__`` be updated
+to accept a factory function as a ``namespace`` keyword-only argument.
+If present, the value provided as the ``namespace`` argument will be
+called without arguments to create the result of ``type.__prepare__``
+instead of using a freshly created dictionary instance. For example,
+the following will use an ordered dictionary as the class namespace::
+
+ class OrderedBase(namespace=collections.OrderedDict):
+ pass
+
+ class Ordered(OrderedBase):
+ # cls.__dict__ is still a read-only proxy to the class namespace,
+ # but the underlying storage is an OrderedDict instance
+
+
+.. note::
+
+ This PEP, along with the existing ability to use __prepare__ to share a
+ single namespace amongst multiple class objects, highlights a possible
+ issue with the attribute lookup caching: when the underlying mapping is
+ updated by other means, the attribute lookup cache is not invalidated
+ correctly (this is a key part of the reason class ``__dict__`` attributes
+ produce a read-only view of the underlying storage).
+
+ Since the optimisation provided by that cache is highly desirable,
+ the use of a preexisting namespace as the class namespace may need to
+ be declared as officially unsupported (since the observed behaviour is
+ rather strange when the caches get out of sync).
+
+
+Key Benefits
+============
+
+
+Easier use of custom namespaces for a class
+-------------------------------------------
+
+Currently, to use a different type (such as ``collections.OrderedDict``) for
+a class namespace, or to use a pre-populated namespace, it is necessary to
+write and use a custom metaclass. With this PEP, using a custom namespace
+becomes as simple as specifying an appropriate factory function in the
+class header.
+
+
+Easier inheritance of definition time behaviour
+-----------------------------------------------
+
+Understanding Python's metaclasses requires a deep understanding of
+the type system and the class construction process. This is legitimately
+seen as challenging, due to the need to keep multiple moving parts (the code,
+the metaclass hint, the actual metaclass, the class object, instances of the
+class object) clearly distinct in your mind. Even when you know the rules,
+it's still easy to make a mistake if you're not being extremely careful.
+
+Understanding the proposed implicit class initialization hook only requires
+ordinary method inheritance, which isn't quite as daunting a task. The new
+hook provides a more gradual path towards understanding all of the phases
+involved in the class definition process.
+
+
+Reduced chance of metaclass conflicts
+-------------------------------------
+
+One of the big issues that makes library authors reluctant to use metaclasses
+(even when they would be appropriate) is the risk of metaclass conflicts.
+These occur whenever two unrelated metaclasses are used by the desired
+parents of a class definition. This risk also makes it very difficult to
+*add* a metaclass to a class that has previously been published without one.
+
+By contrast, adding an ``__init_subclass__`` method to an existing type poses
+a similar level of risk to adding an ``__init__`` method: technically, there
+is a risk of breaking poorly implemented subclasses, but when that occurs,
+it is recognised as a bug in the subclass rather than the library author
+breaching backwards compatibility guarantees.
+
+
+Integrates cleanly with \PEP 3135
+---------------------------------
+
+Given that the method is called on already existing classes, the new
+hook will be able to freely invoke class methods that rely on the
+implicit ``__class__`` reference introduced by PEP 3135, including
+methods that use the zero argument form of ``super()``.
+
+
+Replaces many use cases for dynamic setting of ``__metaclass__``
+----------------------------------------------------------------
+
+For use cases that don't involve completely replacing the defined
+class, Python 2 code that dynamically set ``__metaclass__`` can now
+dynamically set ``__init_subclass__`` instead. For more advanced use
+cases, introduction of an explicit metaclass (possibly made available
+as a required base class) will still be necessary in order to support
+Python 3.
+
+
+A path of introduction into Python
+==================================
+
+Most of the benefits of this PEP can already be implemented using
+a simple metaclass. For the ``__init_subclass__`` hook this works
+all the way down to python 2.7, while the namespace needs python 3.0
+to work. Such a class has been `uploaded to PyPI`_.
+
+The only drawback of such a metaclass are the mentioned problems with
+metaclasses and multiple inheritance. Two classes using such a
+metaclass can only be combined, if they use exactly the same such
+metaclass. This fact calls for the inclusion of such a class into the
+standard library, let's call it ``SubclassMeta``, with a base class
+using it called ``SublassInit``. Once all users use this standard
+library metaclass, classes from different packages can easily be
+combined.
+
+But still such classes cannot be easily combined with other classes
+using other metaclasses. Authors of metaclasses should bear that in
+mind and inherit from the standard metaclass if it seems useful
+for users of the metaclass to add more functionality. Ultimately,
+if the need for combining with other metaclasses is strong enough,
+the proposed functionality may be introduced into python's ``type``.
+
+Those arguments strongly hint to the following procedure to include
+the proposed functionality into python:
+
+1. The metaclass implementing this proposal is put onto PyPI, so that
+ it can be used and scrutinized.
+2. Once the code is properly mature, it can be added to the python
+ standard library. There should be a new module called
+ ``metaclass`` which collects tools for metaclass authors, as well
+ as a documentation of the best practices of how to write
+ metaclasses.
+3. If the need of combining this metaclass with other metaclasses is
+ strong enough, it may be included into python itself.
+
+
+New Ways of Using Classes
+=========================
+
+This proposal has many usecases like the following. In the examples,
+we still inherit from the ``SubclassInit`` base class. This would
+become unnecessary once this PEP is included in Python directly.
+
+Subclass registration
+---------------------
+
+Especially when writing a plugin system, one likes to register new
+subclasses of a plugin baseclass. This can be done as follows::
+
+ class PluginBase(SubclassInit):
+ subclasses = []
+
+ def __init_subclass__(cls, ns, **kwargs):
+ super().__init_subclass__(ns, **kwargs)
+ cls.subclasses.append(cls)
+
+One should note that this also works nicely as a mixin class.
+
+Trait descriptors
+-----------------
+
+There are many designs of python descriptors in the wild which, for
+example, check boundaries of values. Often those "traits" need some support
+of a metaclass to work. This is how this would look like with this
+PEP::
+
+ class Trait:
+ def __get__(self, instance, owner):
+ return instance.__dict__[self.key]
+
+ def __set__(self, instance, value):
+ instance.__dict__[self.key] = value
+
+ class Int(Trait):
+ def __set__(self, instance, value):
+ # some boundary check code here
+ super().__set__(instance, value)
+
+ class HasTraits(SubclassInit):
+ def __init_subclass__(cls, ns, **kwargs):
+ super().__init_subclass__(ns, **kwargs)
+ for k, v in ns.items():
+ if isinstance(v, Trait):
+ v.key = k
+
+The new ``namespace`` keyword in the class header enables a number of
+interesting options for controlling the way a class is initialised,
+including some aspects of the object models of both Javascript and Ruby.
+
+
+Order preserving classes
+------------------------
+
+::
+
+ class OrderedClassBase(namespace=collections.OrderedDict):
+ pass
+
+ class OrderedClass(OrderedClassBase):
+ a = 1
+ b = 2
+ c = 3
+
+
+Prepopulated namespaces
+-----------------------
+
+::
+
+ seed_data = dict(a=1, b=2, c=3)
+ class PrepopulatedClass(namespace=seed_data.copy):
+ pass
+
+
+Cloning a prototype class
+-------------------------
+
+::
+
+ class NewClass(namespace=Prototype.__dict__.copy):
+ pass
+
+
+Rejected Design Options
+=======================
+
+
+Calling the hook on the class itself
+------------------------------------
+
+Adding an ``__autodecorate__`` hook that would be called on the class
+itself was the proposed idea of PEP 422. Most examples work the same
+way or even better if the hook is called on the subclass. In general,
+it is much easier to explicitly call the hook on the class in which it
+is defined (to opt-in to such a behavior) than to opt-out, meaning
+that one does not want the hook to be called on the class it is
+defined in.
+
+This becomes most evident if the class in question is designed as a
+mixin: it is very unlikely that the code of the mixin is to be
+executed for the mixin class itself, as it is not supposed to be a
+complete class on its own.
+
+The original proposal also made major changes in the class
+initialization process, rendering it impossible to back-port the
+proposal to older python versions.
+
+
+Other variants of calling the hook
+----------------------------------
+
+Other names for the hook were presented, namely ``__decorate__`` or
+``__autodecorate__``. This proposal opts for ``__init_subclass__`` as
+it is very close to the ``__init__`` method, just for the subclass,
+while it is not very close to decorators, as it does not return the
+class.
+
+
+Requiring an explicit decorator on ``__init_subclass__``
+--------------------------------------------------------
+
+One could require the explicit use of ``@classmethod`` on the
+``__init_subclass__`` decorator. It was made implicit since there's no
+sensible interpretation for leaving it out, and that case would need
+to be detected anyway in order to give a useful error message.
+
+This decision was reinforced after noticing that the user experience of
+defining ``__prepare__`` and forgetting the ``@classmethod`` method
+decorator is singularly incomprehensible (particularly since PEP 3115
+documents it as an ordinary method, and the current documentation doesn't
+explicitly say anything one way or the other).
+
+
+Passing in the namespace directly rather than a factory function
+----------------------------------------------------------------
+
+At one point, PEP 422 proposed that the class namespace be passed
+directly as a keyword argument, rather than passing a factory function.
+However, this encourages an unsupported behaviour (that is, passing the
+same namespace to multiple classes, or retaining direct write access
+to a mapping used as a class namespace), so the API was switched to
+the factory function version.
+
+
+Possible Extensions
+===================
+
+Some extensions to this PEP are imaginable, which are postponed to a
+later pep:
+
+* A ``__new_subclass__`` method could be defined which acts like a
+ ``__new__`` for classes. This would be very close to
+ ``__autodecorate__`` in PEP 422.
+* ``__subclasshook__`` could be made a classmethod in a class instead
+ of a method in the metaclass.
+
+References
+==========
+
+.. _published code:
+ http://mail.python.org/pipermail/python-dev/2012-June/119878.html
+
+.. _more than 10 years ago:
+ http://mail.python.org/pipermail/python-dev/2001-November/018651.html
+
+.. _Zope's ExtensionClass:
+ http://docs.zope.org/zope_secrets/extensionclass.html
+
+.. _uploaded to PyPI:
+ https://pypi.python.org/pypi/metaclass
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list