<br><br><div><span class="gmail_quote">On 4/30/07, <b class="gmail_sendername">Phillip J. Eby</b> <<a href="mailto:pje@telecommunity.com">pje@telecommunity.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
This is just the first draft (also checked into SVN), and doesn't include<br>the details of how the extension API works (so that third-party interfaces<br>and generic functions can interoperate using the same decorators,
<br>annotations, etc.).<br><br>Comments and questions appreciated, as it'll help drive better explanations<br>of both the design and rationales. I'm usually not that good at guessing<br>what other people will want to know (or are likely to misunderstand) until
<br>I get actual questions.<br><br><br>PEP: 3124<br>Title: Overloading, Generic Functions, Interfaces, and Adaptation<br>Version: $Revision: 55029 $<br>Last-Modified: $Date: 2007-04-30 18:48:06 -0400 (Mon, 30 Apr 2007) $<br>
Author: Phillip J. Eby <<a href="mailto:pje@telecommunity.com">pje@telecommunity.com</a>><br>Discussions-To: Python 3000 List <<a href="mailto:python-3000@python.org">python-3000@python.org</a>><br>Status: Draft
<br>Type: Standards Track<br>Requires: 3107, 3115, 3119<br>Replaces: 245, 246<br>Content-Type: text/x-rst<br>Created: 28-Apr-2007<br>Post-History: 30-Apr-2007</blockquote><div><br>[snip]<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>"Before" and "After" Methods<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br><br>In addition to the simple next-method chaining shown above, it is<br>sometimes useful to have other ways of combining methods. For
<br>example, the "observer pattern" can sometimes be implemented by adding<br>extra methods to a function, that execute before or after the normal<br>implementation.<br><br>To support these use cases, the ``overloading`` module will supply
<br>``@before``, ``@after``, and ``@around`` decorators, that roughly<br>correspond to the same types of methods in the Common Lisp Object<br>System (CLOS), or the corresponding "advice" types in AspectJ.<br><br>
Like ``@when``, all of these decorators must be passed the function to<br>be overloaded, and can optionally accept a predicate as well::<br><br> def begin_transaction(db):<br> print "Beginning the actual transaction"
<br><br><br> @before(begin_transaction)<br> def check_single_access(db: SingletonDB):<br> if db.inuse:<br> raise TransactionError("Database already in use")<br><br> @after(begin_transaction)
<br> def start_logging(db: LoggableDB):<br> db.set_log_level(VERBOSE)</blockquote><div><br><br>If we are looking at doing Design By Contract using @before and @after (preconditions and postconditions), shouldn't there be some way of getting at the return value in functions decorated with @after? For example, it seems reasonable to require an extra argument, perhaps at the beginning:
<br><br>def successor(num):<br> return num + 1<br><br>@before(successor)<br>def check_positive(num: int):<br> if num < 0:<br> raise PreconditionError("Positive integer inputs required")<br><br>@after(successor)
<br>def check_successor(returned, num:int):<br> if returned != num + 1:<br> raise PostconditionError("successor failed to do its job")<br><br>Or am I missing something about how @after works?<br><br>+1, BTW, on this whole idea.
<br><br>- C<br> </div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">``@before`` and ``@after`` methods are invoked either before or after
<br>the main function body, and are *never considered ambiguous*. That<br>is, it will not cause any errors to have multiple "before" or "after"<br>methods with identical or overlapping signatures. Ambiguities are
<br>resolved using the order in which the methods were added to the<br>target function.<br><br>"Before" methods are invoked most-specific method first, with<br>ambiguous methods being executed in the order they were added. All
<br>"before" methods are called before any of the function's "primary"<br>methods (i.e. normal ``@overload`` methods) are executed.<br><br>"After" methods are invoked in the *reverse* order, after all of the
<br>function's "primary" methods are executed. That is, they are executed<br>least-specific methods first, with ambiguous methods being executed in<br>the reverse of the order in which they were added.<br><br>
The return values of both "before" and "after" methods are ignored,<br>and any uncaught exceptions raised by *any* methods (primary or other)<br>immediately end the dispatching process. "Before" and "after" methods
<br>cannot have ``__proceed__`` arguments, as they are not responsible<br>for calling any other methods. They are simply called as a<br>notification before or after the primary methods.<br><br>Thus, "before" and "after" methods can be used to check or establish
<br>preconditions (e.g. by raising an error if the conditions aren't met)<br>or to ensure postconditions, without needing to duplicate any existing<br>functionality.<br><br><br>"Around" Methods<br>~~~~~~~~~~~~~~~~
<br><br>The ``@around`` decorator declares a method as an "around" method.<br>"Around" methods are much like primary methods, except that the<br>least-specific "around" method has higher precedence than the
<br>most-specific "before" or method.<br><br>Unlike "before" and "after" methods, however, "Around" methods *are*<br>responsible for calling their ``__proceed__`` argument, in order to
<br>continue the invocation process. "Around" methods are usually used<br>to transform input arguments or return values, or to wrap specific<br>cases with special error handling or try/finally conditions, e.g.::
<br><br> @around(commit_transaction)<br> def lock_while_committing(__proceed__, db: SingletonDB):<br> with db.global_lock:<br> return __proceed__(db)<br><br>They can also be used to replace the normal handling for a specific
<br>case, by *not* invoking the ``__proceed__`` function.<br><br>The ``__proceed__`` given to an "around" method will either be the<br>next applicable "around" method, a ``DispatchError`` instance,<br>
or a synthetic method object that will call all the "before" methods,<br>followed by the primary method chain, followed by all the "after"<br>methods, and return the result from the primary method chain.
<br><br>Thus, just as with normal methods, ``__proceed__`` can be checked for<br>``DispatchError``-ness, or simply invoked. The "around" method should<br>return the value returned by ``__proceed__``, unless of course it
<br>wishes to modify or replace it with a different return value for the<br>function as a whole.<br><br><br>Custom Combinations<br>~~~~~~~~~~~~~~~~~~~<br><br>The decorators described above (``@overload``, ``@when``, ``@before``,
<br>``@after``, and ``@around``) collectively implement what in CLOS is<br>called the "standard method combination" -- the most common patterns<br>used in combining methods.<br><br>Sometimes, however, an application or library may have use for a more
<br>sophisticated type of method combination. For example, if you<br>would like to have "discount" methods that return a percentage off,<br>to be subtracted from the value returned by the primary method(s),<br>
you might write something like this::<br><br> from overloading import always_overrides, merge_by_default<br> from overloading import Around, Before, After, Method, MethodList<br><br> class Discount(MethodList):
<br> """Apply return values as discounts"""<br><br> def __call__(self, *args, **kw):<br> retval = self.tail(*args, **kw)<br> for sig, body in self.sorted
():<br> retval -= retval * body(*args, **kw)<br> return retval<br><br> # merge discounts by priority<br> merge_by_default(Discount)<br><br> # discounts have precedence over before/after/primary methods
<br> always_overrides(Discount, Before)<br> always_overrides(Discount, After)<br> always_overrides(Discount, Method)<br><br> # but not over "around" methods<br> always_overrides(Around, Discount)
<br><br> # Make a decorator called "discount" that works just like the<br> # standard decorators...<br> discount = Discount.make_decorator('discount')<br><br> # and now let's use it...
<br> def price(product):<br> return product.list_price<br><br> @discount(price)<br> def ten_percent_off_shoes(product: Shoe)<br> return Decimal('0.1')<br><br>Similar techniques can be used to implement a wide variety of
<br>CLOS-style method qualifiers and combination rules. The process of<br>creating custom method combination objects and their corresponding<br>decorators is described in more detail under the `Extension API`_<br>section.
<br><br>Note, by the way, that the ``@discount`` decorator shown will work<br>correctly with any new predicates defined by other code. For example,<br>if ``zope.interface`` were to register its interface types to work<br>
correctly as argument annotations, you would be able to specify<br>discounts on the basis of its interface types, not just classes or<br>``overloading``-defined interface types.<br><br>Similarly, if a library like RuleDispatch or PEAK-Rules were to
<br>register an appropriate predicate implementation and dispatch engine,<br>one would then be able to use those predicates for discounts as well,<br>e.g.::<br><br> from somewhere import Pred # some predicate implementation
<br><br> @discount(<br> price,<br> Pred("isinstance(product,Shoe) and"<br> " product.material.name=='Blue Suede'")<br> )<br> def forty_off_blue_suede_shoes(product):
<br> return Decimal('0.4')<br><br>The process of defining custom predicate types and dispatching engines<br>is also described in more detail under the `Extension API`_ section.<br><br><br>Overloading Inside Classes
<br>--------------------------<br><br>All of the decorators above have a special additional behavior when<br>they are directly invoked within a class body: the first parameter<br>(other than ``__proceed__``, if present) of the decorated function
<br>will be treated as though it had an annotation equal to the class<br>in which it was defined.<br><br>That is, this code::<br><br> class And(object):<br> # ...<br> @when(get_conjuncts)<br> def __conjuncts(self):
<br> return self.conjuncts<br><br>produces the same effect as this (apart from the existence of a<br>private method)::<br><br> class And(object):<br> # ...<br><br> @when(get_conjuncts)<br> def get_conjuncts_of_and(ob: And):
<br> return ob.conjuncts<br><br>This behavior is both a convenience enhancement when defining lots of<br>methods, and a requirement for safely distinguishing multi-argument<br>overloads in subclasses. Consider, for example, the following code::
<br><br> class A(object):<br> def foo(self, ob):<br> print "got an object"<br><br> @overload<br> def foo(__proceed__, self, ob:Iterable):<br> print "it's iterable!"
<br> return __proceed__(self, ob)<br><br><br> class B(A):<br> foo = A.foo # foo must be defined in local namespace<br><br> @overload<br> def foo(__proceed__, self, ob:Iterable):
<br> print "B got an iterable!"<br> return __proceed__(self, ob)<br><br>Due to the implicit class rule, calling ``B().foo([])`` will print<br>"B got an iterable!" followed by "it's iterable!", and finally,
<br>"got an object", while ``A().foo([])`` would print only the messages<br>defined in ``A``.<br><br>Conversely, without the implicit class rule, the two "Iterable"<br>methods would have the exact same applicability conditions, so calling
<br>either ``A().foo([])`` or ``B().foo([])`` would result in an<br>``AmbiguousMethods`` error.<br><br>It is currently an open issue to determine the best way to implement<br>this rule in Python 3.0. Under Python 2.x, a class' metaclass was
<br>not chosen until the end of the class body, which means that<br>decorators could insert a custom metaclass to do processing of this<br>sort. (This is how RuleDispatch, for example, implements the implicit<br>class rule.)
<br><br>PEP 3115, however, requires that a class' metaclass be determined<br>*before* the class body has executed, making it impossible to use this<br>technique for class decoration any more.<br><br>At this writing, discussion on this issue is ongoing.
<br><br><br>Interfaces and Adaptation<br>-------------------------<br><br>The ``overloading`` module provides a simple implementation of<br>interfaces and adaptation. The following example defines an<br>``IStack`` interface, and declares that ``list`` objects support it::
<br><br> from overloading import abstract, Interface<br><br> class IStack(Interface):<br> @abstract<br> def push(self, ob)<br> """Push 'ob' onto the stack"""
<br><br> @abstract<br> def pop(self):<br> """Pop a value and return it"""<br><br><br> when(IStack.push, (list, object))(list.append)<br> when(IStack.pop, (list,))(
list.pop)<br><br> mylist = []<br> mystack = IStack(mylist)<br> mystack.push(42)<br> assert mystack.pop()==42<br><br>The ``Interface`` class is a kind of "universal adapter". It accepts<br>a single argument: an object to adapt. It then binds all its methods
<br>to the target object, in place of itself. Thus, calling<br>``mystack.push(42``) is the same as calling<br>``IStack.push(mylist, 42)``.<br><br>The ``@abstract`` decorator marks a function as being abstract: i.e.,<br>having no implementation. If an ``@abstract`` function is called,
<br>it raises ``NoApplicableMethods``. To become executable, overloaded<br>methods must be added using the techniques previously described. (That<br>is, methods can be added using ``@when``, ``@before``, ``@after``,<br>``@around``, or any custom method combination decorators.)
<br><br>In the example above, the ``list.append`` method is added as a method<br>for ``IStack.push()`` when its arguments are a list and an arbitrary<br>object. Thus, ``IStack.push(mylist, 42)`` is translated to<br>``list.append(mylist, 42)``, thereby implementing the desired
<br>operation.<br><br>(Note: the ``@abstract`` decorator is not limited to use in interface<br>definitions; it can be used anywhere that you wish to create an<br>"empty" generic function that initially has no methods. In
<br>particular, it need not be used inside a class.)<br><br><br>Subclassing and Re-assembly<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br><br>Interfaces can be subclassed::<br><br> class ISizedStack(IStack):<br> @abstract
<br> def __len__(self):<br> """Return the number of items on the stack"""<br><br> # define __len__ support for ISizedStack<br> when(ISizedStack.__len__, (list,))(list.__len__)
<br><br>Or assembled by combining functions from existing interfaces::<br><br> class Sizable(Interface):<br> __len__ = ISizedStack.__len__<br><br> # list now implements Sizable as well as ISizedStack, without
<br> # making any new declarations!<br><br>A class can be considered to "adapt to" an interface at a given<br>point in time, if no method defined in the interface is guaranteed to<br>raise a ``NoApplicableMethods`` error if invoked on an instance of
<br>that class at that point in time.<br><br>In normal usage, however, it is "easier to ask forgiveness than<br>permission". That is, it is easier to simply use an interface on<br>an object by adapting it to the interface (
e.g. ``IStack(mylist)``)<br>or invoking interface methods directly (e.g. ``IStack.push(mylist,<br>42)``), than to try to figure out whether the object is adaptable to<br>(or directly implements) the interface.<br><br><br>
Implementing an Interface in a Class<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br><br>It is possible to declare that a class directly implements an<br>interface, using the ``declare_implementation()`` function::<br><br> from overloading import declare_implementation
<br><br> class Stack(object):<br> def __init__(self):<br> self.data = []<br> def push(self, ob):<br> self.data.append(ob)<br> def pop(self):<br> return self.data.pop
()<br><br> declare_implementation(IStack, Stack)<br><br>The ``declare_implementation()`` call above is roughly equivalent to<br>the following steps::<br><br> when(IStack.push, (Stack,object))(lambda self, ob: self.push
(ob))<br> when(IStack.pop, (Stack,))(lambda self, ob: self.pop())<br><br>That is, calling ``IStack.push()`` or ``IStack.pop()`` on an instance<br>of any subclass of ``Stack``, will simply delegate to the actual<br>``push()`` or ``pop()`` methods thereof.
<br><br>For the sake of efficiency, calling ``IStack(s)`` where ``s`` is an<br>instance of ``Stack``, **may** return ``s`` rather than an ``IStack``<br>adapter. (Note that calling ``IStack(x)`` where ``x`` is already an<br>
``IStack`` adapter will always return ``x`` unchanged; this is an<br>additional optimization allowed in cases where the adaptee is known<br>to *directly* implement the interface, without adaptation.)<br><br>For convenience, it may be useful to declare implementations in the
<br>class header, e.g.::<br><br> class Stack(metaclass=Implementer, implements=IStack):<br> ...<br><br>Instead of calling ``declare_implementation()`` after the end of the<br>suite.<br><br><br>Interfaces as Type Specifiers
<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br><br>``Interface`` subclasses can be used as argument annotations to<br>indicate what type of objects are acceptable to an overload, e.g.::<br><br> @overload<br> def traverse(g: IGraph, s: IStack):
<br> g = IGraph(g)<br> s = IStack(s)<br> # etc....<br><br>Note, however, that the actual arguments are *not* changed or adapted<br>in any way by the mere use of an interface as a type specifier. You
<br>must explicitly cast the objects to the appropriate interface, as<br>shown above.<br><br>Note, however, that other patterns of interface use are possible.<br>For example, other interface implementations might not support
<br>adaptation, or might require that function arguments already be<br>adapted to the specified interface. So the exact semantics of using<br>an interface as a type specifier are dependent on the interface<br>objects you actually use.
<br><br>For the interface objects defined by this PEP, however, the semantics<br>are as described above. An interface I1 is considered "more specific"<br>than another interface I2, if the set of descriptors in I1's
<br>inheritance hierarchy are a proper superset of the descriptors in I2's<br>inheritance hierarchy.<br><br>So, for example, ``ISizedStack`` is more specific than both<br>``ISizable`` and ``ISizedStack``, irrespective of the inheritance
<br>relationships between these interfaces. It is purely a question of<br>what operations are included within those interfaces -- and the<br>*names* of the operations are unimportant.<br><br>Interfaces (at least the ones provided by ``overloading``) are always
<br>considered less-specific than concrete classes. Other interface<br>implementations can decide on their own specificity rules, both<br>between interfaces and other interfaces, and between interfaces and<br>classes.<br>
<br><br>Non-Method Attributes in Interfaces<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br><br>The ``Interface`` implementation actually treats all attributes and<br>methods (i.e. descriptors) in the same way: their ``__get__`` (and
<br>``__set__`` and ``__delete__``, if present) methods are called with<br>the wrapped (adapted) object as "self". For functions, this has the<br>effect of creating a bound method linking the generic function to the
<br>wrapped object.<br><br>For non-function attributes, it may be easiest to specify them using<br>the ``property`` built-in, and the corresponding ``fget``, ``fset``,<br>and ``fdel`` attributes::<br><br> class ILength(Interface):
<br> @property<br> @abstract<br> def length(self):<br> """Read-only length attribute"""<br><br> # ILength(aList).length == list.__len__(aList)<br> when(
ILength.length.fget, (list,))(list.__len__)<br><br>Alternatively, methods such as ``_get_foo()`` and ``_set_foo()``<br>may be defined as part of the interface, and the property defined<br>in terms of those methods, but this a bit more difficult for users
<br>to implement correctly when creating a class that directly implements<br>the interface, as they would then need to match all the individual<br>method names, not just the name of the property or attribute.<br><br><br>Aspects
<br>-------<br><br>The adaptation system provided assumes that adapters are "stateless",<br>which is to say that adapters have no attributes or storage apart from<br>those of the adapted object. This follows the "typeclass/instance"
<br>model of Haskell, and the concept of "pure" (i.e., transitively<br>composable) adapters.<br><br>However, there are occasionally cases where, to provide a complete<br>implementation of some interface, some sort of additional state is
<br>required.<br><br>One possibility of course, would be to attach monkeypatched "private"<br>attributes to the adaptee. But this is subject to name collisions,<br>and complicates the process of initialization. It also doesn't work
<br>on objects that don't have a ``__dict__`` attribute.<br><br>So the ``Aspect`` class is provided to make it easy to attach extra<br>information to objects that either:<br><br>1. have a ``__dict__`` attribute (so aspect instances can be stored
<br> in it, keyed by aspect class),<br><br>2. support weak referencing (so aspect instances can be managed using<br> a global but thread-safe weak-reference dictionary), or<br><br>3. implement or can be adapt to the ``overloading.IAspectOwner``
<br> interface (technically, #1 or #2 imply this)<br><br>Subclassing ``Aspect`` creates an adapter class whose state is tied<br>to the life of the adapted object.<br><br>For example, suppose you would like to count all the times a certain
<br>method is called on instances of ``Target`` (a classic AOP example).<br>You might do something like::<br><br> from overloading import Aspect<br><br> class Count(Aspect):<br> count = 0<br><br> @after(
Target.some_method)<br> def count_after_call(self, *args, **kw):<br> Count(self).count += 1<br><br>The above code will keep track of the number of times that<br>``Target.some_method()`` is successfully called (
i.e., it will not<br>count errors). Other code can then access the count using<br>``Count(someTarget).count``.<br><br>``Aspect`` instances can of course have ``__init__`` methods, to<br>initialize any data structures. They can use either ``__slots__``
<br>or dictionary-based attributes for storage.<br><br>While this facility is rather primitive compared to a full-featured<br>AOP tool like AspectJ, persons who wish to build pointcut libraries<br>or other AspectJ-like features can certainly use ``Aspect`` objects
<br>and method-combination decorators as a base for more expressive AOP<br>tools.<br><br>XXX spec out full aspect API, including keys, N-to-1 aspects, manual<br> attach/detach/delete of aspect instances, and the ``IAspectOwner``
<br> interface.<br><br><br>Extension API<br>=============<br><br>TODO: explain how all of these work<br><br>implies(o1, o2)<br><br>declare_implementation(iface, class)<br><br>predicate_signatures(ob)<br><br>parse_rule(ruleset, body, predicate, actiontype, localdict, globaldict)
<br><br>combine_actions(a1, a2)<br><br>rules_for(f)<br><br>Rule objects<br><br>ActionDef objects<br><br>RuleSet objects<br><br>Method objects<br><br>MethodList objects<br><br>IAspectOwner<br><br><br><br>Implementation Notes
<br>====================<br><br>Most of the functionality described in this PEP is already implemented<br>in the in-development version of the PEAK-Rules framework. In<br>particular, the basic overloading and method combination framework
<br>(minus the ``@overload`` decorator) already exists there. The<br>implementation of all of these features in ``peak.rules.core`` is 656<br>lines of Python at this writing.<br><br>``peak.rules.core`` currently relies on the DecoratorTools and
<br>BytecodeAssembler modules, but both of these dependencies can be<br>replaced, as DecoratorTools is used mainly for Python 2.3<br>compatibility and to implement structure types (which can be done<br>with named tuples in later versions of Python). The use of
<br>BytecodeAssembler can be replaced using an "exec" or "compile"<br>workaround, given a reasonable effort. (It would be easier to do this<br>if the ``func_closure`` attribute of function objects was writable.)
<br><br>The ``Interface`` class has been previously prototyped, but is not<br>included in PEAK-Rules at the present time.<br><br>The "implicit class rule" has previously been implemented in the<br>RuleDispatch library. However, it relies on the ``__metaclass__``
<br>hook that is currently eliminated in PEP 3115.<br><br>I don't currently know how to make ``@overload`` play nicely with<br>``classmethod`` and ``staticmethod`` in class bodies. It's not really<br>clear if it needs to, however.
<br><br><br>Copyright<br>=========<br><br>This document has been placed in the public domain.<br><br>_______________________________________________<br>Python-3000 mailing list<br><a href="mailto:Python-3000@python.org">Python-3000@python.org
</a><br><a href="http://mail.python.org/mailman/listinfo/python-3000">http://mail.python.org/mailman/listinfo/python-3000</a><br>Unsubscribe: <a href="http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com">
http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com</a><br></blockquote></div><br>