<div dir="ltr"><div style>Sorry, maybe I am too late to comment on this, but,</div><div style><br></div><div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"> >>> @singledispatch</span><br style="font-family:arial,sans-serif;font-size:12.727272033691406px">
<span style="font-family:arial,sans-serif;font-size:12.727272033691406px"> ... def fun(arg, verbose=False):</span><br style="font-family:arial,sans-serif;font-size:12.727272033691406px"><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"> ... if verbose:</span><br style="font-family:arial,sans-serif;font-size:12.727272033691406px">
<span style="font-family:arial,sans-serif;font-size:12.727272033691406px"> ... print("Let me just say,", end=" ")</span><br style="font-family:arial,sans-serif;font-size:12.727272033691406px">
<span style="font-family:arial,sans-serif;font-size:12.727272033691406px"> ... print(arg)</span><br style="font-family:arial,sans-serif;font-size:12.727272033691406px"></div><div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"><br>
</span></div><div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">It is not clear from the PEP (up until the end of the User API section at least) when, if ever, is this implementation of fun ever called. I mean, what type of 'arg' triggers a dispatch to this function body?</span></div>
<div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">I am guessing that when the arg does not match the type of any of the other registered functions, this function body is used by default. But it is only a guess, the PEP doesn't state this clearly.</span></div>
<div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"><br></span></div><div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">If my guess is true, w</span><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">ould it be reasonable to update the example "def fun" code to reflect this, e.g., to print("Warning: I do not know what to do with arg {} of type {}".format(arg, type(arg)).</span></div>
<div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"><br></span></div><div style><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">So my comment is just about clarity of the PEP text. I do not wish to interfere with pronouncement.</span></div>
<div style><br></div><div style><font face="arial, sans-serif">Thanks.</font></div><div style><font face="arial, sans-serif"><br></font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, May 31, 2013 at 10:46 AM, Łukasz Langa <span dir="ltr"><<a href="mailto:lukasz@langa.pl" target="_blank">lukasz@langa.pl</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello python-dev,<br>
<br>
PEP 443 is ready for final review. I'm attaching the latest<br>
version below for convenience. The full history of changes<br>
is available here: <a href="http://hg.python.org/peps/log/tip/pep-0443.txt" target="_blank">http://hg.python.org/peps/log/tip/pep-0443.txt</a><br>
<br>
A reference implementation for PEP 443 is available at:<br>
<a href="http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363" target="_blank">http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363</a><br>
<br>
with relevant tests here:<br>
<a href="http://hg.python.org/features/pep-443/file/tip/Lib/test/test_functools.py#l855" target="_blank">http://hg.python.org/features/pep-443/file/tip/Lib/test/test_functools.py#l855</a><br>
<br>
and documentation here:<br>
<a href="http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l189" target="_blank">http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l189</a><br>
<br>
There's also an official backport for 2.6 - 3.3 already up:<br>
<a href="https://pypi.python.org/pypi/singledispatch" target="_blank">https://pypi.python.org/pypi/singledispatch</a><br>
<br>
<br>
<br>
PEP: 443<br>
Title: Single-dispatch generic functions<br>
Version: $Revision$<br>
Last-Modified: $Date$<br>
Author: Łukasz Langa <<a href="mailto:lukasz@langa.pl">lukasz@langa.pl</a>><br>
Discussions-To: Python-Dev <<a href="mailto:python-dev@python.org">python-dev@python.org</a>><br>
Status: Draft<br>
Type: Standards Track<br>
Content-Type: text/x-rst<br>
Created: 22-May-2013<br>
Post-History: 22-May-2013, 25-May-2013, 31-May-2013<br>
Replaces: 245, 246, 3124<br>
<br>
<br>
Abstract<br>
========<br>
<br>
This PEP proposes a new mechanism in the ``functools`` standard library<br>
module that provides a simple form of generic programming known as<br>
single-dispatch generic functions.<br>
<br>
A **generic function** is composed of multiple functions implementing<br>
the same operation for different types. Which implementation should be<br>
used during a call is determined by the dispatch algorithm. When the<br>
implementation is chosen based on the type of a single argument, this is<br>
known as **single dispatch**.<br>
<br>
<br>
Rationale and Goals<br>
===================<br>
<br>
Python has always provided a variety of built-in and standard-library<br>
generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``,<br>
``copy.copy()``, and most of the functions in the ``operator`` module.<br>
However, it currently:<br>
<br>
1. does not have a simple or straightforward way for developers to<br>
create new generic functions,<br>
<br>
2. does not have a standard way for methods to be added to existing<br>
generic functions (i.e., some are added using registration<br>
functions, others require defining ``__special__`` methods, possibly<br>
by monkeypatching).<br>
<br>
In addition, it is currently a common anti-pattern for Python code to<br>
inspect the types of received arguments, in order to decide what to do<br>
with the objects. For example, code may wish to accept either an object<br>
of some type, or a sequence of objects of that type.<br>
<br>
Currently, the "obvious way" to do this is by type inspection, but this<br>
is brittle and closed to extension. Abstract Base Classes make it easier<br>
to discover present behaviour, but don't help adding new behaviour.<br>
A developer using an already-written library may be unable to change how<br>
their objects are treated by such code, especially if the objects they<br>
are using were created by a third party.<br>
<br>
Therefore, this PEP proposes a uniform API to address dynamic<br>
overloading using decorators.<br>
<br>
<br>
User API<br>
========<br>
<br>
To define a generic function, decorate it with the ``@singledispatch``<br>
decorator. Note that the dispatch happens on the type of the first<br>
argument, create your function accordingly::<br>
<br>
>>> from functools import singledispatch<br>
>>> @singledispatch<br>
... def fun(arg, verbose=False):<br>
... if verbose:<br>
... print("Let me just say,", end=" ")<br>
... print(arg)<br>
<br>
To add overloaded implementations to the function, use the<br>
``register()`` attribute of the generic function. It is a decorator,<br>
taking a type parameter and decorating a function implementing the<br>
operation for that type::<br>
<br>
>>> @fun.register(int)<br>
... def _(arg, verbose=False):<br>
... if verbose:<br>
... print("Strength in numbers, eh?", end=" ")<br>
... print(arg)<br>
...<br>
>>> @fun.register(list)<br>
... def _(arg, verbose=False):<br>
... if verbose:<br>
... print("Enumerate this:")<br>
... for i, elem in enumerate(arg):<br>
... print(i, elem)<br>
<br>
To enable registering lambdas and pre-existing functions, the<br>
``register()`` attribute can be used in a functional form::<br>
<br>
>>> def nothing(arg, verbose=False):<br>
... print("Nothing.")<br>
...<br>
>>> fun.register(type(None), nothing)<br>
<br>
The ``register()`` attribute returns the undecorated function which<br>
enables decorator stacking, pickling, as well as creating unit tests for<br>
each variant independently::<br>
<br>
>>> @fun.register(float)<br>
... @fun.register(Decimal)<br>
... def fun_num(arg, verbose=False):<br>
... if verbose:<br>
... print("Half of your number:", end=" ")<br>
... print(arg / 2)<br>
...<br>
>>> fun_num is fun<br>
False<br>
<br>
When called, the generic function dispatches on the type of the first<br>
argument::<br>
<br>
>>> fun("Hello, world.")<br>
Hello, world.<br>
>>> fun("test.", verbose=True)<br>
Let me just say, test.<br>
>>> fun(42, verbose=True)<br>
Strength in numbers, eh? 42<br>
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)<br>
Enumerate this:<br>
0 spam<br>
1 spam<br>
2 eggs<br>
3 spam<br>
>>> fun(None)<br>
Nothing.<br>
>>> fun(1.23)<br>
0.615<br>
<br>
Where there is no registered implementation for a specific type, its<br>
method resolution order is used to find a more generic implementation.<br>
To check which implementation will the generic function choose for<br>
a given type, use the ``dispatch()`` attribute::<br>
<br>
>>> fun.dispatch(float)<br>
<function fun_num at 0x104319058><br>
>>> fun.dispatch(dict)<br>
<function fun at 0x103fe4788><br>
<br>
To access all registered implementations, use the read-only ``registry``<br>
attribute::<br>
<br>
>>> fun.registry.keys()<br>
dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,<br>
<class 'decimal.Decimal'>, <class 'list'>,<br>
<class 'float'>])<br>
>>> fun.registry[float]<br>
<function fun_num at 0x1035a2840><br>
>>> fun.registry[object]<br>
<function fun at 0x103170788><br>
<br>
The proposed API is intentionally limited and opinionated, as to ensure<br>
it is easy to explain and use, as well as to maintain consistency with<br>
existing members in the ``functools`` module.<br>
<br>
<br>
Implementation Notes<br>
====================<br>
<br>
The functionality described in this PEP is already implemented in the<br>
``pkgutil`` standard library module as ``simplegeneric``. Because this<br>
implementation is mature, the goal is to move it largely as-is. The<br>
reference implementation is available on <a href="http://hg.python.org" target="_blank">hg.python.org</a> [#ref-impl]_.<br>
<br>
The dispatch type is specified as a decorator argument. An alternative<br>
form using function annotations has been considered but its inclusion<br>
has been deferred. As of May 2013, this usage pattern is out of scope<br>
for the standard library [#pep-0008]_ and the best practices for<br>
annotation usage are still debated.<br>
<br>
Based on the current ``pkgutil.simplegeneric`` implementation and<br>
following the convention on registering virtual subclasses on Abstract<br>
Base Classes, the dispatch registry will not be thread-safe.<br>
<br>
Abstract Base Classes<br>
---------------------<br>
<br>
The ``pkgutil.simplegeneric`` implementation relied on several forms of<br>
method resultion order (MRO). ``@singledispatch`` removes special<br>
handling of old-style classes and Zope's ExtensionClasses. More<br>
importantly, it introduces support for Abstract Base Classes (ABC).<br>
<br>
When a generic function implementation is registered for an ABC, the<br>
dispatch algorithm switches to a mode of MRO calculation for the<br>
provided argument which includes the relevant ABCs. The algorithm is as<br>
follows::<br>
<br>
def _compose_mro(cls, haystack):<br>
"""Calculates the MRO for a given class `cls`, including relevant<br>
abstract base classes from `haystack`."""<br>
bases = set(cls.__mro__)<br>
mro = list(cls.__mro__)<br>
for regcls in haystack:<br>
if regcls in bases or not issubclass(cls, regcls):<br>
continue # either present in the __mro__ or unrelated<br>
for index, base in enumerate(mro):<br>
if not issubclass(base, regcls):<br>
break<br>
if base in bases and not issubclass(regcls, base):<br>
# Conflict resolution: put classes present in __mro__<br>
# and their subclasses first.<br>
index += 1<br>
mro.insert(index, regcls)<br>
return mro<br>
<br>
In its most basic form, it returns the MRO for the given type::<br>
<br>
>>> _compose_mro(dict, [])<br>
[<class 'dict'>, <class 'object'>]<br>
<br>
When the haystack consists of ABCs that the specified type is a subclass<br>
of, they are inserted in a predictable order::<br>
<br>
>>> _compose_mro(dict, [Sized, MutableMapping, str,<br>
... Sequence, Iterable])<br>
[<class 'dict'>, <class 'collections.abc.MutableMapping'>,<br>
<class 'collections.abc.Iterable'>, <class 'collections.abc.Sized'>,<br>
<class 'object'>]<br>
<br>
While this mode of operation is significantly slower, all dispatch<br>
decisions are cached. The cache is invalidated on registering new<br>
implementations on the generic function or when user code calls<br>
``register()`` on an ABC to register a new virtual subclass. In the<br>
latter case, it is possible to create a situation with ambiguous<br>
dispatch, for instance::<br>
<br>
>>> from collections import Iterable, Container<br>
>>> class P:<br>
... pass<br>
>>> Iterable.register(P)<br>
<class '__main__.P'><br>
>>> Container.register(P)<br>
<class '__main__.P'><br>
<br>
Faced with ambiguity, ``@singledispatch`` refuses the temptation to<br>
guess::<br>
<br>
>>> @singledispatch<br>
... def g(arg):<br>
... return "base"<br>
...<br>
>>> g.register(Iterable, lambda arg: "iterable")<br>
<function <lambda> at 0x108b49110><br>
>>> g.register(Container, lambda arg: "container")<br>
<function <lambda> at 0x108b491c8><br>
>>> g(P())<br>
Traceback (most recent call last):<br>
...<br>
RuntimeError: Ambiguous dispatch: <class 'collections.abc.Container'><br>
or <class 'collections.abc.Iterable'><br>
<br>
Note that this exception would not be raised if ``Iterable`` and<br>
``Container`` had been provided as base classes during class definition.<br>
In this case dispatch happens in the MRO order::<br>
<br>
>>> class Ten(Iterable, Container):<br>
... def __iter__(self):<br>
... for i in range(10):<br>
... yield i<br>
... def __contains__(self, value):<br>
... return value in range(10)<br>
...<br>
>>> g(Ten())<br>
'iterable'<br>
<br>
<br>
Usage Patterns<br>
==============<br>
<br>
This PEP proposes extending behaviour only of functions specifically<br>
marked as generic. Just as a base class method may be overridden by<br>
a subclass, so too may a function be overloaded to provide custom<br>
functionality for a given type.<br>
<br>
Universal overloading does not equal *arbitrary* overloading, in the<br>
sense that we need not expect people to randomly redefine the behavior<br>
of existing functions in unpredictable ways. To the contrary, generic<br>
function usage in actual programs tends to follow very predictable<br>
patterns and registered implementations are highly-discoverable in the<br>
common case.<br>
<br>
If a module is defining a new generic operation, it will usually also<br>
define any required implementations for existing types in the same<br>
place. Likewise, if a module is defining a new type, then it will<br>
usually define implementations there for any generic functions that it<br>
knows or cares about. As a result, the vast majority of registered<br>
implementations can be found adjacent to either the function being<br>
overloaded, or to a newly-defined type for which the implementation is<br>
adding support.<br>
<br>
It is only in rather infrequent cases that one will have implementations<br>
registered in a module that contains neither the function nor the<br>
type(s) for which the implementation is added. In the absence of<br>
incompetence or deliberate intention to be obscure, the few<br>
implementations that are not registered adjacent to the relevant type(s)<br>
or function(s), will generally not need to be understood or known about<br>
outside the scope where those implementations are defined. (Except in<br>
the "support modules" case, where best practice suggests naming them<br>
accordingly.)<br>
<br>
As mentioned earlier, single-dispatch generics are already prolific<br>
throughout the standard library. A clean, standard way of doing them<br>
provides a way forward to refactor those custom implementations to use<br>
a common one, opening them up for user extensibility at the same time.<br>
<br>
<br>
Alternative approaches<br>
======================<br>
<br>
In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution<br>
with overloading based on arbitrary rule sets (with the default<br>
implementation dispatching on argument types), as well as interfaces,<br>
adaptation and method combining. PEAK-Rules [#peak-rules]_ is<br>
a reference implementation of the concepts described in PJE's PEP.<br>
<br>
Such a broad approach is inherently complex, which makes reaching<br>
a consensus hard. In contrast, this PEP focuses on a single piece of<br>
functionality that is simple to reason about. It's important to note<br>
this does not preclude the use of other approaches now or in the future.<br>
<br>
In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents<br>
a generic function implementation that dispatches on types of all<br>
arguments on a function. The same approach was chosen in Andrey Popp's<br>
``generic`` package available on PyPI [#pypi-generic]_, as well as David<br>
Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_.<br>
<br>
While this seems desirable at first, I agree with Fredrik Lundh's<br>
comment that "if you design APIs with pages of logic just to sort out<br>
what code a function should execute, you should probably hand over the<br>
API design to someone else". In other words, the single argument<br>
approach proposed in this PEP is not only easier to implement but also<br>
clearly communicates that dispatching on a more complex state is an<br>
anti-pattern. It also has the virtue of corresponding directly with the<br>
familiar method dispatch mechanism in object oriented programming. The<br>
only difference is whether the custom implementation is associated more<br>
closely with the data (object-oriented methods) or the algorithm<br>
(single-dispatch overloading).<br>
<br>
PyPy's RPython offers ``extendabletype`` [#pairtype]_, a metaclass which<br>
enables classes to be externally extended. In combination with<br>
``pairtype()`` and ``pair()`` factories, this offers a form of<br>
single-dispatch generics.<br>
<br>
<br>
Acknowledgements<br>
================<br>
<br>
Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and<br>
PEAK-Rules, influences include Paul Moore's original issue<br>
[#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part<br>
of the ``functools`` API, Guido van Rossum's article on multimethods<br>
[#artima2005]_, and discussions with Raymond Hettinger on a general<br>
pprint rewrite. Huge thanks to Nick Coghlan for encouraging me to create<br>
this PEP and providing initial feedback.<br>
<br>
<br>
References<br>
==========<br>
<br>
.. [#ref-impl]<br>
<a href="http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359" target="_blank">http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359</a><br>
<br>
.. [#pep-0008] PEP 8 states in the "Programming Recommendations"<br>
section that "the Python standard library will not use function<br>
annotations as that would result in a premature commitment to<br>
a particular annotation style".<br>
(<a href="http://www.python.org/dev/peps/pep-0008" target="_blank">http://www.python.org/dev/peps/pep-0008</a>)<br>
<br>
.. [#pep-3124] <a href="http://www.python.org/dev/peps/pep-3124/" target="_blank">http://www.python.org/dev/peps/pep-3124/</a><br>
<br>
.. [#peak-rules] <a href="http://peak.telecommunity.com/DevCenter/PEAK_2dRules" target="_blank">http://peak.telecommunity.com/DevCenter/PEAK_2dRules</a><br>
<br>
.. [#artima2005]<br>
<a href="http://www.artima.com/weblogs/viewpost.jsp?thread=101605" target="_blank">http://www.artima.com/weblogs/viewpost.jsp?thread=101605</a><br>
<br>
.. [#pypi-generic] <a href="http://pypi.python.org/pypi/generic" target="_blank">http://pypi.python.org/pypi/generic</a><br>
<br>
.. [#gnosis-multimethods]<br>
<a href="http://gnosis.cx/publish/programming/charming_python_b12.html" target="_blank">http://gnosis.cx/publish/programming/charming_python_b12.html</a><br>
<br>
.. [#pairtype]<br>
<a href="https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py" target="_blank">https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py</a><br>
<br>
.. [#issue-5135] <a href="http://bugs.python.org/issue5135" target="_blank">http://bugs.python.org/issue5135</a><br>
<br>
<br>
Copyright<br>
=========<br>
<br>
This document has been placed in the public domain.<br>
<br>
<br>
..<br>
Local Variables:<br>
mode: indented-text<br>
indent-tabs-mode: nil<br>
sentence-end-double-space: t<br>
fill-column: 70<br>
coding: utf-8<br>
End:<br>
<br>
<br>
<br>
--<br>
Best regards,<br>
Łukasz Langa<br>
<br>
WWW: <a href="http://lukasz.langa.pl/" target="_blank">http://lukasz.langa.pl/</a><br>
Twitter: @llanga<br>
IRC: ambv on #python-dev<br>
<br>
_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/python-dev" target="_blank">http://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com" target="_blank">http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Gustavo J. A. M. Carneiro<br>"The universe is always one step beyond logic." -- Frank Herbert
</div>