[Python-3000] PEP 3124 - Overloading, Generic Functions, Interfaces, etc.
monpublic at gmail.com
Wed May 9 03:58:13 CEST 2007
On 4/30/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> This is just the first draft (also checked into SVN), and doesn't include
> the details of how the extension API works (so that third-party interfaces
> and generic functions can interoperate using the same decorators,
> annotations, etc.).
> Comments and questions appreciated, as it'll help drive better
> of both the design and rationales. I'm usually not that good at guessing
> what other people will want to know (or are likely to misunderstand) until
> I get actual questions.
> PEP: 3124
> Title: Overloading, Generic Functions, Interfaces, and Adaptation
> Version: $Revision: 55029 $
> Last-Modified: $Date: 2007-04-30 18:48:06 -0400 (Mon, 30 Apr 2007) $
> Author: Phillip J. Eby <pje at telecommunity.com>
> Discussions-To: Python 3000 List <python-3000 at python.org>
> Status: Draft
> Type: Standards Track
> Requires: 3107, 3115, 3119
> Replaces: 245, 246
> Content-Type: text/x-rst
> Created: 28-Apr-2007
> Post-History: 30-Apr-2007
> "Before" and "After" Methods
> In addition to the simple next-method chaining shown above, it is
> sometimes useful to have other ways of combining methods. For
> example, the "observer pattern" can sometimes be implemented by adding
> extra methods to a function, that execute before or after the normal
> To support these use cases, the ``overloading`` module will supply
> ``@before``, ``@after``, and ``@around`` decorators, that roughly
> correspond to the same types of methods in the Common Lisp Object
> System (CLOS), or the corresponding "advice" types in AspectJ.
> Like ``@when``, all of these decorators must be passed the function to
> be overloaded, and can optionally accept a predicate as well::
> def begin_transaction(db):
> print "Beginning the actual transaction"
> def check_single_access(db: SingletonDB):
> if db.inuse:
> raise TransactionError("Database already in use")
> def start_logging(db: LoggableDB):
If we are looking at doing Design By Contract using @before and @after
(preconditions and postconditions), shouldn't there be some way of getting
at the return value in functions decorated with @after? For example, it
seems reasonable to require an extra argument, perhaps at the beginning:
return num + 1
def check_positive(num: int):
if num < 0:
raise PreconditionError("Positive integer inputs required")
def check_successor(returned, num:int):
if returned != num + 1:
raise PostconditionError("successor failed to do its job")
Or am I missing something about how @after works?
+1, BTW, on this whole idea.
``@before`` and ``@after`` methods are invoked either before or after
> the main function body, and are *never considered ambiguous*. That
> is, it will not cause any errors to have multiple "before" or "after"
> methods with identical or overlapping signatures. Ambiguities are
> resolved using the order in which the methods were added to the
> target function.
> "Before" methods are invoked most-specific method first, with
> ambiguous methods being executed in the order they were added. All
> "before" methods are called before any of the function's "primary"
> methods (i.e. normal ``@overload`` methods) are executed.
> "After" methods are invoked in the *reverse* order, after all of the
> function's "primary" methods are executed. That is, they are executed
> least-specific methods first, with ambiguous methods being executed in
> the reverse of the order in which they were added.
> The return values of both "before" and "after" methods are ignored,
> and any uncaught exceptions raised by *any* methods (primary or other)
> immediately end the dispatching process. "Before" and "after" methods
> cannot have ``__proceed__`` arguments, as they are not responsible
> for calling any other methods. They are simply called as a
> notification before or after the primary methods.
> Thus, "before" and "after" methods can be used to check or establish
> preconditions (e.g. by raising an error if the conditions aren't met)
> or to ensure postconditions, without needing to duplicate any existing
> "Around" Methods
> The ``@around`` decorator declares a method as an "around" method.
> "Around" methods are much like primary methods, except that the
> least-specific "around" method has higher precedence than the
> most-specific "before" or method.
> Unlike "before" and "after" methods, however, "Around" methods *are*
> responsible for calling their ``__proceed__`` argument, in order to
> continue the invocation process. "Around" methods are usually used
> to transform input arguments or return values, or to wrap specific
> cases with special error handling or try/finally conditions, e.g.::
> def lock_while_committing(__proceed__, db: SingletonDB):
> with db.global_lock:
> return __proceed__(db)
> They can also be used to replace the normal handling for a specific
> case, by *not* invoking the ``__proceed__`` function.
> The ``__proceed__`` given to an "around" method will either be the
> next applicable "around" method, a ``DispatchError`` instance,
> or a synthetic method object that will call all the "before" methods,
> followed by the primary method chain, followed by all the "after"
> methods, and return the result from the primary method chain.
> Thus, just as with normal methods, ``__proceed__`` can be checked for
> ``DispatchError``-ness, or simply invoked. The "around" method should
> return the value returned by ``__proceed__``, unless of course it
> wishes to modify or replace it with a different return value for the
> function as a whole.
> Custom Combinations
> The decorators described above (``@overload``, ``@when``, ``@before``,
> ``@after``, and ``@around``) collectively implement what in CLOS is
> called the "standard method combination" -- the most common patterns
> used in combining methods.
> Sometimes, however, an application or library may have use for a more
> sophisticated type of method combination. For example, if you
> would like to have "discount" methods that return a percentage off,
> to be subtracted from the value returned by the primary method(s),
> you might write something like this::
> from overloading import always_overrides, merge_by_default
> from overloading import Around, Before, After, Method, MethodList
> class Discount(MethodList):
> """Apply return values as discounts"""
> def __call__(self, *args, **kw):
> retval = self.tail(*args, **kw)
> for sig, body in self.sorted():
> retval -= retval * body(*args, **kw)
> return retval
> # merge discounts by priority
> # discounts have precedence over before/after/primary methods
> always_overrides(Discount, Before)
> always_overrides(Discount, After)
> always_overrides(Discount, Method)
> # but not over "around" methods
> always_overrides(Around, Discount)
> # Make a decorator called "discount" that works just like the
> # standard decorators...
> discount = Discount.make_decorator('discount')
> # and now let's use it...
> def price(product):
> return product.list_price
> def ten_percent_off_shoes(product: Shoe)
> return Decimal('0.1')
> Similar techniques can be used to implement a wide variety of
> CLOS-style method qualifiers and combination rules. The process of
> creating custom method combination objects and their corresponding
> decorators is described in more detail under the `Extension API`_
> Note, by the way, that the ``@discount`` decorator shown will work
> correctly with any new predicates defined by other code. For example,
> if ``zope.interface`` were to register its interface types to work
> correctly as argument annotations, you would be able to specify
> discounts on the basis of its interface types, not just classes or
> ``overloading``-defined interface types.
> Similarly, if a library like RuleDispatch or PEAK-Rules were to
> register an appropriate predicate implementation and dispatch engine,
> one would then be able to use those predicates for discounts as well,
> from somewhere import Pred # some predicate implementation
> Pred("isinstance(product,Shoe) and"
> " product.material.name=='Blue Suede'")
> def forty_off_blue_suede_shoes(product):
> return Decimal('0.4')
> The process of defining custom predicate types and dispatching engines
> is also described in more detail under the `Extension API`_ section.
> Overloading Inside Classes
> All of the decorators above have a special additional behavior when
> they are directly invoked within a class body: the first parameter
> (other than ``__proceed__``, if present) of the decorated function
> will be treated as though it had an annotation equal to the class
> in which it was defined.
> That is, this code::
> class And(object):
> # ...
> def __conjuncts(self):
> return self.conjuncts
> produces the same effect as this (apart from the existence of a
> private method)::
> class And(object):
> # ...
> def get_conjuncts_of_and(ob: And):
> return ob.conjuncts
> This behavior is both a convenience enhancement when defining lots of
> methods, and a requirement for safely distinguishing multi-argument
> overloads in subclasses. Consider, for example, the following code::
> class A(object):
> def foo(self, ob):
> print "got an object"
> def foo(__proceed__, self, ob:Iterable):
> print "it's iterable!"
> return __proceed__(self, ob)
> class B(A):
> foo = A.foo # foo must be defined in local namespace
> def foo(__proceed__, self, ob:Iterable):
> print "B got an iterable!"
> return __proceed__(self, ob)
> Due to the implicit class rule, calling ``B().foo()`` will print
> "B got an iterable!" followed by "it's iterable!", and finally,
> "got an object", while ``A().foo()`` would print only the messages
> defined in ``A``.
> Conversely, without the implicit class rule, the two "Iterable"
> methods would have the exact same applicability conditions, so calling
> either ``A().foo()`` or ``B().foo()`` would result in an
> ``AmbiguousMethods`` error.
> It is currently an open issue to determine the best way to implement
> this rule in Python 3.0. Under Python 2.x, a class' metaclass was
> not chosen until the end of the class body, which means that
> decorators could insert a custom metaclass to do processing of this
> sort. (This is how RuleDispatch, for example, implements the implicit
> class rule.)
> PEP 3115, however, requires that a class' metaclass be determined
> *before* the class body has executed, making it impossible to use this
> technique for class decoration any more.
> At this writing, discussion on this issue is ongoing.
> Interfaces and Adaptation
> The ``overloading`` module provides a simple implementation of
> interfaces and adaptation. The following example defines an
> ``IStack`` interface, and declares that ``list`` objects support it::
> from overloading import abstract, Interface
> class IStack(Interface):
> def push(self, ob)
> """Push 'ob' onto the stack"""
> def pop(self):
> """Pop a value and return it"""
> when(IStack.push, (list, object))(list.append)
> when(IStack.pop, (list,))(list.pop)
> mylist = 
> mystack = IStack(mylist)
> assert mystack.pop()==42
> The ``Interface`` class is a kind of "universal adapter". It accepts
> a single argument: an object to adapt. It then binds all its methods
> to the target object, in place of itself. Thus, calling
> ``mystack.push(42``) is the same as calling
> ``IStack.push(mylist, 42)``.
> The ``@abstract`` decorator marks a function as being abstract: i.e.,
> having no implementation. If an ``@abstract`` function is called,
> it raises ``NoApplicableMethods``. To become executable, overloaded
> methods must be added using the techniques previously described. (That
> is, methods can be added using ``@when``, ``@before``, ``@after``,
> ``@around``, or any custom method combination decorators.)
> In the example above, the ``list.append`` method is added as a method
> for ``IStack.push()`` when its arguments are a list and an arbitrary
> object. Thus, ``IStack.push(mylist, 42)`` is translated to
> ``list.append(mylist, 42)``, thereby implementing the desired
> (Note: the ``@abstract`` decorator is not limited to use in interface
> definitions; it can be used anywhere that you wish to create an
> "empty" generic function that initially has no methods. In
> particular, it need not be used inside a class.)
> Subclassing and Re-assembly
> Interfaces can be subclassed::
> class ISizedStack(IStack):
> def __len__(self):
> """Return the number of items on the stack"""
> # define __len__ support for ISizedStack
> when(ISizedStack.__len__, (list,))(list.__len__)
> Or assembled by combining functions from existing interfaces::
> class Sizable(Interface):
> __len__ = ISizedStack.__len__
> # list now implements Sizable as well as ISizedStack, without
> # making any new declarations!
> A class can be considered to "adapt to" an interface at a given
> point in time, if no method defined in the interface is guaranteed to
> raise a ``NoApplicableMethods`` error if invoked on an instance of
> that class at that point in time.
> In normal usage, however, it is "easier to ask forgiveness than
> permission". That is, it is easier to simply use an interface on
> an object by adapting it to the interface (e.g. ``IStack(mylist)``)
> or invoking interface methods directly (e.g. ``IStack.push(mylist,
> 42)``), than to try to figure out whether the object is adaptable to
> (or directly implements) the interface.
> Implementing an Interface in a Class
> It is possible to declare that a class directly implements an
> interface, using the ``declare_implementation()`` function::
> from overloading import declare_implementation
> class Stack(object):
> def __init__(self):
> self.data = 
> def push(self, ob):
> def pop(self):
> return self.data.pop()
> declare_implementation(IStack, Stack)
> The ``declare_implementation()`` call above is roughly equivalent to
> the following steps::
> when(IStack.push, (Stack,object))(lambda self, ob: self.push(ob))
> when(IStack.pop, (Stack,))(lambda self, ob: self.pop())
> That is, calling ``IStack.push()`` or ``IStack.pop()`` on an instance
> of any subclass of ``Stack``, will simply delegate to the actual
> ``push()`` or ``pop()`` methods thereof.
> For the sake of efficiency, calling ``IStack(s)`` where ``s`` is an
> instance of ``Stack``, **may** return ``s`` rather than an ``IStack``
> adapter. (Note that calling ``IStack(x)`` where ``x`` is already an
> ``IStack`` adapter will always return ``x`` unchanged; this is an
> additional optimization allowed in cases where the adaptee is known
> to *directly* implement the interface, without adaptation.)
> For convenience, it may be useful to declare implementations in the
> class header, e.g.::
> class Stack(metaclass=Implementer, implements=IStack):
> Instead of calling ``declare_implementation()`` after the end of the
> Interfaces as Type Specifiers
> ``Interface`` subclasses can be used as argument annotations to
> indicate what type of objects are acceptable to an overload, e.g.::
> def traverse(g: IGraph, s: IStack):
> g = IGraph(g)
> s = IStack(s)
> # etc....
> Note, however, that the actual arguments are *not* changed or adapted
> in any way by the mere use of an interface as a type specifier. You
> must explicitly cast the objects to the appropriate interface, as
> shown above.
> Note, however, that other patterns of interface use are possible.
> For example, other interface implementations might not support
> adaptation, or might require that function arguments already be
> adapted to the specified interface. So the exact semantics of using
> an interface as a type specifier are dependent on the interface
> objects you actually use.
> For the interface objects defined by this PEP, however, the semantics
> are as described above. An interface I1 is considered "more specific"
> than another interface I2, if the set of descriptors in I1's
> inheritance hierarchy are a proper superset of the descriptors in I2's
> inheritance hierarchy.
> So, for example, ``ISizedStack`` is more specific than both
> ``ISizable`` and ``ISizedStack``, irrespective of the inheritance
> relationships between these interfaces. It is purely a question of
> what operations are included within those interfaces -- and the
> *names* of the operations are unimportant.
> Interfaces (at least the ones provided by ``overloading``) are always
> considered less-specific than concrete classes. Other interface
> implementations can decide on their own specificity rules, both
> between interfaces and other interfaces, and between interfaces and
> Non-Method Attributes in Interfaces
> The ``Interface`` implementation actually treats all attributes and
> methods (i.e. descriptors) in the same way: their ``__get__`` (and
> ``__set__`` and ``__delete__``, if present) methods are called with
> the wrapped (adapted) object as "self". For functions, this has the
> effect of creating a bound method linking the generic function to the
> wrapped object.
> For non-function attributes, it may be easiest to specify them using
> the ``property`` built-in, and the corresponding ``fget``, ``fset``,
> and ``fdel`` attributes::
> class ILength(Interface):
> def length(self):
> """Read-only length attribute"""
> # ILength(aList).length == list.__len__(aList)
> when(ILength.length.fget, (list,))(list.__len__)
> Alternatively, methods such as ``_get_foo()`` and ``_set_foo()``
> may be defined as part of the interface, and the property defined
> in terms of those methods, but this a bit more difficult for users
> to implement correctly when creating a class that directly implements
> the interface, as they would then need to match all the individual
> method names, not just the name of the property or attribute.
> The adaptation system provided assumes that adapters are "stateless",
> which is to say that adapters have no attributes or storage apart from
> those of the adapted object. This follows the "typeclass/instance"
> model of Haskell, and the concept of "pure" (i.e., transitively
> composable) adapters.
> However, there are occasionally cases where, to provide a complete
> implementation of some interface, some sort of additional state is
> One possibility of course, would be to attach monkeypatched "private"
> attributes to the adaptee. But this is subject to name collisions,
> and complicates the process of initialization. It also doesn't work
> on objects that don't have a ``__dict__`` attribute.
> So the ``Aspect`` class is provided to make it easy to attach extra
> information to objects that either:
> 1. have a ``__dict__`` attribute (so aspect instances can be stored
> in it, keyed by aspect class),
> 2. support weak referencing (so aspect instances can be managed using
> a global but thread-safe weak-reference dictionary), or
> 3. implement or can be adapt to the ``overloading.IAspectOwner``
> interface (technically, #1 or #2 imply this)
> Subclassing ``Aspect`` creates an adapter class whose state is tied
> to the life of the adapted object.
> For example, suppose you would like to count all the times a certain
> method is called on instances of ``Target`` (a classic AOP example).
> You might do something like::
> from overloading import Aspect
> class Count(Aspect):
> count = 0
> def count_after_call(self, *args, **kw):
> Count(self).count += 1
> The above code will keep track of the number of times that
> ``Target.some_method()`` is successfully called (i.e., it will not
> count errors). Other code can then access the count using
> ``Aspect`` instances can of course have ``__init__`` methods, to
> initialize any data structures. They can use either ``__slots__``
> or dictionary-based attributes for storage.
> While this facility is rather primitive compared to a full-featured
> AOP tool like AspectJ, persons who wish to build pointcut libraries
> or other AspectJ-like features can certainly use ``Aspect`` objects
> and method-combination decorators as a base for more expressive AOP
> XXX spec out full aspect API, including keys, N-to-1 aspects, manual
> attach/detach/delete of aspect instances, and the ``IAspectOwner``
> Extension API
> TODO: explain how all of these work
> implies(o1, o2)
> declare_implementation(iface, class)
> parse_rule(ruleset, body, predicate, actiontype, localdict, globaldict)
> combine_actions(a1, a2)
> Rule objects
> ActionDef objects
> RuleSet objects
> Method objects
> MethodList objects
> Implementation Notes
> Most of the functionality described in this PEP is already implemented
> in the in-development version of the PEAK-Rules framework. In
> particular, the basic overloading and method combination framework
> (minus the ``@overload`` decorator) already exists there. The
> implementation of all of these features in ``peak.rules.core`` is 656
> lines of Python at this writing.
> ``peak.rules.core`` currently relies on the DecoratorTools and
> BytecodeAssembler modules, but both of these dependencies can be
> replaced, as DecoratorTools is used mainly for Python 2.3
> compatibility and to implement structure types (which can be done
> with named tuples in later versions of Python). The use of
> BytecodeAssembler can be replaced using an "exec" or "compile"
> workaround, given a reasonable effort. (It would be easier to do this
> if the ``func_closure`` attribute of function objects was writable.)
> The ``Interface`` class has been previously prototyped, but is not
> included in PEAK-Rules at the present time.
> The "implicit class rule" has previously been implemented in the
> RuleDispatch library. However, it relies on the ``__metaclass__``
> hook that is currently eliminated in PEP 3115.
> I don't currently know how to make ``@overload`` play nicely with
> ``classmethod`` and ``staticmethod`` in class bodies. It's not really
> clear if it needs to, however.
> This document has been placed in the public domain.
> Python-3000 mailing list
> Python-3000 at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-3000