Mailman 3 February 2015 - Python-checkins

cpython: Line missed in last checkin
by raymond.hettinger Feb. 28, 2015

Feb. 28, 2015

https://hg.python.org/cpython/rev/de6d278c432a changeset: 94781:de6d278c432a user: Raymond Hettinger <python(a)rcn.com> date: Fri Feb 27 16:59:29 2015 -0800 summary: Line missed in last checkin files: Modules/_collectionsmodule.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -783,7 +783,6 @@ assert(i &… [View More]

1 0

cpython: Since the index is always non-negative, use faster unsigned division and modulo.
by raymond.hettinger Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/91a1613e161c changeset: 94780:91a1613e161c user: Raymond Hettinger <python(a)rcn.com> date: Fri Feb 27 12:42:54 2015 -0800 summary: Since the index is always non-negative, use faster unsigned division and modulo. files: Modules/_collectionsmodule.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/… [View More]

2 1

cpython (merge 3.4 -> default): Merge with 3.4 for porting HOWTO tweaks
by brett.cannon Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/d5c9dd8dd5da changeset: 94779:d5c9dd8dd5da parent: 94777:28ba862036cc parent: 94778:b7945565e150 user: Brett Cannon <brett(a)python.org> date: Fri Feb 27 15:10:38 2015 -0500 summary: Merge with 3.4 for porting HOWTO tweaks files: Doc/howto/pyporting.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/… [View More]

1 0

cpython (3.4): Fix the clarification as to why division cannot be ported automatically
by brett.cannon Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/b7945565e150 changeset: 94778:b7945565e150 branch: 3.4 parent: 94776:bcc877a28219 user: Brett Cannon <brett(a)python.org> date: Fri Feb 27 15:10:03 2015 -0500 summary: Fix the clarification as to why division cannot be ported automatically files: Doc/howto/pyporting.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +… [View More]

1 0

peps: Create PEP 487 from Martin Teichmann's submission
by chris.angelico Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/peps/rev/c45defff808a changeset: 5716:c45defff808a user: Chris Angelico <rosuav(a)gmail.com> date: Sat Feb 28 06:07:44 2015 +1100 summary: Create PEP 487 from Martin Teichmann's submission files: pep-0487.txt | 485 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 485 insertions(+), 0 deletions(-) diff --git a/pep-0487.txt b/pep-0487.txt new file mode 100644 --- /dev/null +++ b/pep-0487.txt @@ -0,0 +1,485 @@ +PEP: 487 +Title: Simpler … [View More]customisation of class creation +Version: $Revision$ +Last-Modified: $Date$ +Author: Martin Teichmann <lkb.teichmann(a)gmail.com>, +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 27-Feb-2015 +Python-Version: 3.5 +Post-History: 27-Feb-2015 +Replaces: 422 + + +Abstract +======== + +Currently, customising class creation requires the use of a custom metaclass. +This custom metaclass then persists for the entire lifecycle of the class, +creating the potential for spurious metaclass conflicts. + +This PEP proposes to instead support a wide range of customisation +scenarios through a new ``namespace`` parameter in the class header, and +a new ``__init_subclass__`` hook in the class body. + +The new mechanism should be easier to understand and use than +implementing a custom metaclass, and thus should provide a gentler +introduction to the full power Python's metaclass machinery. + + +Connection to other PEP +======================= + +This is a competing proposal to PEP 422 by Nick Coughlan and Daniel Urban. +It shares both most of the PEP text and proposed code, but has major +differences in how to achieve its goals. + +Background +========== + +For an already created class ``cls``, the term "metaclass" has a clear +meaning: it is the value of ``type(cls)``. + +*During* class creation, it has another meaning: it is also used to refer to +the metaclass hint that may be provided as part of the class definition. +While in many cases these two meanings end up referring to one and the same +object, there are two situations where that is not the case: + +* If the metaclass hint refers to an instance of ``type``, then it is + considered as a candidate metaclass along with the metaclasses of all of + the parents of the class being defined. If a more appropriate metaclass is + found amongst the candidates, then it will be used instead of the one + given in the metaclass hint. +* Otherwise, an explicit metaclass hint is assumed to be a factory function + and is called directly to create the class object. In this case, the final + metaclass will be determined by the factory function definition. In the + typical case (where the factory functions just calls ``type``, or, in + Python 3.3 or later, ``types.new_class``) the actual metaclass is then + determined based on the parent classes. + +It is notable that only the actual metaclass is inherited - a factory +function used as a metaclass hook sees only the class currently being +defined, and is not invoked for any subclasses. + +In Python 3, the metaclass hint is provided using the ``metaclass=Meta`` +keyword syntax in the class header. This allows the ``__prepare__`` method +on the metaclass to be used to create the ``locals()`` namespace used during +execution of the class body (for example, specifying the use of +``collections.OrderedDict`` instead of a regular ``dict``). + +In Python 2, there was no ``__prepare__`` method (that API was added for +Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__`` +attribute, and the class creation process would extract that value from the +class namespace to use as the metaclass hint. There is `published code`_ that +makes use of this feature. + +Another new feature in Python 3 is the zero-argument form of the ``super()`` +builtin, introduced by PEP 3135. This feature uses an implicit ``__class__`` +reference to the class being defined to replace the "by name" references +required in Python 2. Just as code invoked during execution of a Python 2 +metaclass could not call methods that referenced the class by name (as the +name had not yet been bound in the containing scope), similarly, Python 3 +metaclasses cannot call methods that rely on the implicit ``__class__`` +reference (as it is not populated until after the metaclass has returned +control to the class creation machinery). + +Finally, when a class uses a custom metaclass, it can pose additional +challenges to the use of multiple inheritance, as a new class cannot +inherit from parent classes with unrelated metaclasses. This means that +it is impossible to add a metaclass to an already published class: such +an addition is a backwards incompatible change due to the risk of metaclass +conflicts. + + +Proposal +======== + +This PEP proposes that a new mechanism to customise class creation be +added to Python 3.5 that meets the following criteria: + +1. Integrates nicely with class inheritance structures (including mixins and + multiple inheritance), +2. Integrates nicely with the implicit ``__class__`` reference and + zero-argument ``super()`` syntax introduced by PEP 3135, +3. Can be added to an existing base class without a significant risk of + introducing backwards compatibility problems, and +4. Restores the ability for class namespaces to have some influence on the + class creation process (above and beyond populating the namespace itself), + but potentially without the full flexibility of the Python 2 style + ``__metaclass__`` hook. + +Those goals can be achieved by adding two functionalities: + +1. A ``__init_subclass__`` hook that initializes all subclasses of a + given class, and +2. A new keyword parameter ``namespace`` to the class creation statement, + that gives an initialization of the namespace. + +As an example, the first proposal looks as follows:: + + class SpamBase: + # this is implicitly a @classmethod + def __init_subclass__(cls, ns, **kwargs): + # This is invoked after a subclass is created, but before + # explicit decorators are called. + # The usual super() mechanisms are used to correctly support + # multiple inheritance. + # ns is the classes namespace + # **kwargs are the keyword arguments to the subclasses' + # class creation statement + super().__init_subclass__(cls, ns, **kwargs) + + class Spam(SpamBase): + pass + # the new hook is called on Spam + +To simplify the cooperative multiple inheritance case, ``object`` will gain +a default implementation of the hook that does nothing:: + + class object: + def __init_subclass__(cls, ns): + pass + +Note that this method has no keyword arguments, meaning that all +methods which are more specialized have to process all keyword +arguments. + +This general proposal is not a new idea (it was first suggested for +inclusion in the language definition `more than 10 years ago`_, and a +similar mechanism has long been supported by `Zope's ExtensionClass`_), +but the situation has changed sufficiently in recent years that +the idea is worth reconsidering for inclusion. + +The second part of the proposal is to have a ``namespace`` keyword +argument to the class declaration statement. If present, its value +will be called without arguments to initialize a subclasses +namespace, very much like a metaclass ``__prepare__`` method would +do. + +In addition, the introduction of the metaclass ``__prepare__`` method +in PEP 3115 allows a further enhancement that was not possible in +Python 2: this PEP also proposes that ``type.__prepare__`` be updated +to accept a factory function as a ``namespace`` keyword-only argument. +If present, the value provided as the ``namespace`` argument will be +called without arguments to create the result of ``type.__prepare__`` +instead of using a freshly created dictionary instance. For example, +the following will use an ordered dictionary as the class namespace:: + + class OrderedBase(namespace=collections.OrderedDict): + pass + + class Ordered(OrderedBase): + # cls.__dict__ is still a read-only proxy to the class namespace, + # but the underlying storage is an OrderedDict instance + + +.. note:: + + This PEP, along with the existing ability to use __prepare__ to share a + single namespace amongst multiple class objects, highlights a possible + issue with the attribute lookup caching: when the underlying mapping is + updated by other means, the attribute lookup cache is not invalidated + correctly (this is a key part of the reason class ``__dict__`` attributes + produce a read-only view of the underlying storage). + + Since the optimisation provided by that cache is highly desirable, + the use of a preexisting namespace as the class namespace may need to + be declared as officially unsupported (since the observed behaviour is + rather strange when the caches get out of sync). + + +Key Benefits +============ + + +Easier use of custom namespaces for a class +------------------------------------------- + +Currently, to use a different type (such as ``collections.OrderedDict``) for +a class namespace, or to use a pre-populated namespace, it is necessary to +write and use a custom metaclass. With this PEP, using a custom namespace +becomes as simple as specifying an appropriate factory function in the +class header. + + +Easier inheritance of definition time behaviour +----------------------------------------------- + +Understanding Python's metaclasses requires a deep understanding of +the type system and the class construction process. This is legitimately +seen as challenging, due to the need to keep multiple moving parts (the code, +the metaclass hint, the actual metaclass, the class object, instances of the +class object) clearly distinct in your mind. Even when you know the rules, +it's still easy to make a mistake if you're not being extremely careful. + +Understanding the proposed implicit class initialization hook only requires +ordinary method inheritance, which isn't quite as daunting a task. The new +hook provides a more gradual path towards understanding all of the phases +involved in the class definition process. + + +Reduced chance of metaclass conflicts +------------------------------------- + +One of the big issues that makes library authors reluctant to use metaclasses +(even when they would be appropriate) is the risk of metaclass conflicts. +These occur whenever two unrelated metaclasses are used by the desired +parents of a class definition. This risk also makes it very difficult to +*add* a metaclass to a class that has previously been published without one. + +By contrast, adding an ``__init_subclass__`` method to an existing type poses +a similar level of risk to adding an ``__init__`` method: technically, there +is a risk of breaking poorly implemented subclasses, but when that occurs, +it is recognised as a bug in the subclass rather than the library author +breaching backwards compatibility guarantees. + + +Integrates cleanly with \PEP 3135 +--------------------------------- + +Given that the method is called on already existing classes, the new +hook will be able to freely invoke class methods that rely on the +implicit ``__class__`` reference introduced by PEP 3135, including +methods that use the zero argument form of ``super()``. + + +Replaces many use cases for dynamic setting of ``__metaclass__`` +---------------------------------------------------------------- + +For use cases that don't involve completely replacing the defined +class, Python 2 code that dynamically set ``__metaclass__`` can now +dynamically set ``__init_subclass__`` instead. For more advanced use +cases, introduction of an explicit metaclass (possibly made available +as a required base class) will still be necessary in order to support +Python 3. + + +A path of introduction into Python +================================== + +Most of the benefits of this PEP can already be implemented using +a simple metaclass. For the ``__init_subclass__`` hook this works +all the way down to python 2.7, while the namespace needs python 3.0 +to work. Such a class has been `uploaded to PyPI`_. + +The only drawback of such a metaclass are the mentioned problems with +metaclasses and multiple inheritance. Two classes using such a +metaclass can only be combined, if they use exactly the same such +metaclass. This fact calls for the inclusion of such a class into the +standard library, let's call it ``SubclassMeta``, with a base class +using it called ``SublassInit``. Once all users use this standard +library metaclass, classes from different packages can easily be +combined. + +But still such classes cannot be easily combined with other classes +using other metaclasses. Authors of metaclasses should bear that in +mind and inherit from the standard metaclass if it seems useful +for users of the metaclass to add more functionality. Ultimately, +if the need for combining with other metaclasses is strong enough, +the proposed functionality may be introduced into python's ``type``. + +Those arguments strongly hint to the following procedure to include +the proposed functionality into python: + +1. The metaclass implementing this proposal is put onto PyPI, so that + it can be used and scrutinized. +2. Once the code is properly mature, it can be added to the python + standard library. There should be a new module called + ``metaclass`` which collects tools for metaclass authors, as well + as a documentation of the best practices of how to write + metaclasses. +3. If the need of combining this metaclass with other metaclasses is + strong enough, it may be included into python itself. + + +New Ways of Using Classes +========================= + +This proposal has many usecases like the following. In the examples, +we still inherit from the ``SubclassInit`` base class. This would +become unnecessary once this PEP is included in Python directly. + +Subclass registration +--------------------- + +Especially when writing a plugin system, one likes to register new +subclasses of a plugin baseclass. This can be done as follows:: + + class PluginBase(SubclassInit): + subclasses = [] + + def __init_subclass__(cls, ns, **kwargs): + super().__init_subclass__(ns, **kwargs) + cls.subclasses.append(cls) + +One should note that this also works nicely as a mixin class. + +Trait descriptors +----------------- + +There are many designs of python descriptors in the wild which, for +example, check boundaries of values. Often those "traits" need some support +of a metaclass to work. This is how this would look like with this +PEP:: + + class Trait: + def __get__(self, instance, owner): + return instance.__dict__[self.key] + + def __set__(self, instance, value): + instance.__dict__[self.key] = value + + class Int(Trait): + def __set__(self, instance, value): + # some boundary check code here + super().__set__(instance, value) + + class HasTraits(SubclassInit): + def __init_subclass__(cls, ns, **kwargs): + super().__init_subclass__(ns, **kwargs) + for k, v in ns.items(): + if isinstance(v, Trait): + v.key = k + +The new ``namespace`` keyword in the class header enables a number of +interesting options for controlling the way a class is initialised, +including some aspects of the object models of both Javascript and Ruby. + + +Order preserving classes +------------------------ + +:: + + class OrderedClassBase(namespace=collections.OrderedDict): + pass + + class OrderedClass(OrderedClassBase): + a = 1 + b = 2 + c = 3 + + +Prepopulated namespaces +----------------------- + +:: + + seed_data = dict(a=1, b=2, c=3) + class PrepopulatedClass(namespace=seed_data.copy): + pass + + +Cloning a prototype class +------------------------- + +:: + + class NewClass(namespace=Prototype.__dict__.copy): + pass + + +Rejected Design Options +======================= + + +Calling the hook on the class itself +------------------------------------ + +Adding an ``__autodecorate__`` hook that would be called on the class +itself was the proposed idea of PEP 422. Most examples work the same +way or even better if the hook is called on the subclass. In general, +it is much easier to explicitly call the hook on the class in which it +is defined (to opt-in to such a behavior) than to opt-out, meaning +that one does not want the hook to be called on the class it is +defined in. + +This becomes most evident if the class in question is designed as a +mixin: it is very unlikely that the code of the mixin is to be +executed for the mixin class itself, as it is not supposed to be a +complete class on its own. + +The original proposal also made major changes in the class +initialization process, rendering it impossible to back-port the +proposal to older python versions. + + +Other variants of calling the hook +---------------------------------- + +Other names for the hook were presented, namely ``__decorate__`` or +``__autodecorate__``. This proposal opts for ``__init_subclass__`` as +it is very close to the ``__init__`` method, just for the subclass, +while it is not very close to decorators, as it does not return the +class. + + +Requiring an explicit decorator on ``__init_subclass__`` +-------------------------------------------------------- + +One could require the explicit use of ``@classmethod`` on the +``__init_subclass__`` decorator. It was made implicit since there's no +sensible interpretation for leaving it out, and that case would need +to be detected anyway in order to give a useful error message. + +This decision was reinforced after noticing that the user experience of +defining ``__prepare__`` and forgetting the ``@classmethod`` method +decorator is singularly incomprehensible (particularly since PEP 3115 +documents it as an ordinary method, and the current documentation doesn't +explicitly say anything one way or the other). + + +Passing in the namespace directly rather than a factory function +---------------------------------------------------------------- + +At one point, PEP 422 proposed that the class namespace be passed +directly as a keyword argument, rather than passing a factory function. +However, this encourages an unsupported behaviour (that is, passing the +same namespace to multiple classes, or retaining direct write access +to a mapping used as a class namespace), so the API was switched to +the factory function version. + + +Possible Extensions +=================== + +Some extensions to this PEP are imaginable, which are postponed to a +later pep: + +* A ``__new_subclass__`` method could be defined which acts like a + ``__new__`` for classes. This would be very close to + ``__autodecorate__`` in PEP 422. +* ``__subclasshook__`` could be made a classmethod in a class instead + of a method in the metaclass. + +References +========== + +.. _published code: + http://mail.python.org/pipermail/python-dev/2012-June/119878.html + +.. _more than 10 years ago: + http://mail.python.org/pipermail/python-dev/2001-November/018651.html + +.. _Zope's ExtensionClass: + http://docs.zope.org/zope_secrets/extensionclass.html + +.. _uploaded to PyPI: + https://pypi.python.org/pypi/metaclass + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps [View Less]

1 0

cpython (3.4): Fix minor errors in Misc/NEWS. (Thanks for the report, Florian Bruhin!)
by larry.hastings Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/bcc877a28219 changeset: 94776:bcc877a28219 branch: 3.4 parent: 94773:0b390b5a6729 user: Larry Hastings <larry(a)hastings.org> date: Fri Feb 27 09:14:32 2015 -0800 summary: Fix minor errors in Misc/NEWS. (Thanks for the report, Florian Bruhin!) files: Misc/NEWS | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -119,8 +119,6 @@ - Issue #… [View More]

1 0

cpython (merge 3.4 -> default): Merge from 3.4.
by larry.hastings Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/28ba862036cc changeset: 94777:28ba862036cc parent: 94775:38c503c2c066 parent: 94776:bcc877a28219 user: Larry Hastings <larry(a)hastings.org> date: Fri Feb 27 09:21:06 2015 -0800 summary: Merge from 3.4. files: Misc/NEWS | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -345,8 +345,6 @@ - Issue #23366: Fixed possible integer overflow in … [View More]

1 0

cpython: Issue #22834: Drop a redundant comment and use errno instead of an
by brett.cannon Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/38c503c2c066 changeset: 94775:38c503c2c066 user: Brett Cannon <brett(a)python.org> date: Fri Feb 27 12:13:35 2015 -0500 summary: Issue #22834: Drop a redundant comment and use errno instead of an integer. Thanks to Serhiy Storchaka and Martin Panter for the suggestions. files: Lib/test/test_importlib/import_/test_path.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_importlib/import_/… [View More]

1 0

cpython (merge 3.4 -> default): Merge 3.4 (asyncio)
by victor.stinner Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/3708585125b8 changeset: 94774:3708585125b8 parent: 94772:2701e36e692f parent: 94773:0b390b5a6729 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Feb 27 17:49:19 2015 +0100 summary: Merge 3.4 (asyncio) files: Lib/asyncio/base_subprocess.py | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py --- a/Lib/asyncio/base_subprocess.… [View More]

1 0

cpython (3.4): Issue #23537: Remove 2 unused private methods of asyncio.BaseSubprocessTransport
by victor.stinner Feb. 27, 2015

Feb. 27, 2015

https://hg.python.org/cpython/rev/0b390b5a6729 changeset: 94773:0b390b5a6729 branch: 3.4 parent: 94771:056d71d7bb28 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Feb 27 17:49:09 2015 +0100 summary: Issue #23537: Remove 2 unused private methods of asyncio.BaseSubprocessTransport Methods only raise NotImplementedError and are never used. files: Lib/asyncio/base_subprocess.py | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --… [View More]

1 0