[Python-Dev] PEP 318: How I would implement decorators

[Jim Hugunin]

I've spent a couple days reading the decorators discussion and thinking
about the issues.  That's not a lot of time, but I still thought I'd share
my opinions on implementing this feature.  This all is in my (mostly) humble
opinion.

When I started looking at PEP 318, I thought that the most important issue
was deciding on the correct syntax.  After a while I became much more
worried about semantics.

* Semantics, composition and understandable failures

One of the nice properties of C# and Java attributes is that they are
declarative rather than transformative so there are no conflicts between
different attributes placed on the same object.  This is probably too strong
of a restriction for Python; nevertheless, it's a valuable reference point.

Using the definition of returns from the PEP, I get the following behavior
if I confuse its order with classmethod, i.e.

class C:
    [classmethod(m), returns(str)]
    def m(cls): return repr(cls)

C.m()
--> TypeError: unbound method new_f() must be called with C instance as
first argument (got nothing instead)

This is a hard message to understand, but at least it does occur at the
first instance of calling the method.  Even worse is where you forget to
create an instance of a class or to call a function like returns, i.e.

class C:
    [returns]
    def m(self):
        return "hello"

c = C()
print c.m() #there's no error for this call
--> function new_f at 0x01278BB0>

This kind of error scares me a lot.  It can be extremely hard to track down
because it doesn't occur either at the point of declaration or the point of
calling the method, but only at some later stage where the result from the
method is used.  I'm also concerned that this error is caused by the person
using the attribute not the person writing it and that person may be less
sophisticated and more easily confused.

A less critical, but important issue is the propagation of doc strings and
other function attributes, i.e.

class C:
    [returns(str)]
    def m(self):
        "a friendly greeting"
        return "hello"

print C.m.__doc__
--> None

This will really mess up programs that rely on doc strings or other function
attributes.

After thinking about these issues, my first inclination was to sign up for
Raymond Hettinger's NO on 318 campaign.  Unfortunately, as my previous
message showed there are a number of ways that I'd really like to use
decorators in my Python code.

My next thought was to propose that Python's decorators be restricted to be
declarative like C# and Java attributes.  They could be used to add meta
information to functions, but other meta-tools would be needed for any
actual transformations.  I'm still open to that idea, but I've come to
believe that's not a very pythonic approach.

My current idea is that we should design this feature and any documentation
around it to strongly encourage people to write safe, composable decorators.
Specifically, it should be easier to write safe and composable decorators
than to write bad ones.  After that, we can rely on decorator developers to
do the right thing.

1. Use a "decorate" method instead of a call to avoid the worst problems
like [returns].  This has the added benefit in my eyes of making it a little
harder to write small decorators and thus encouraging people to think a
little more before writing these meta-programs.

2. Provide a small number of helper classes that make it easy to write safe
and composable decorators.  Here are my top two candidates:

class attribute:
    def __init__(self, **kwds):
          self.attrs = kwds

    def decorate(self, decl):
        if not hasattr(decl, __attrs__):
            decl.__attrs__ = []
        decl.__attrs__.append(self)
        return decl

    def __repr__(self): ...

class func_wrapper:
    """subclass this and implement a wrap function"""
    def decorate(self, func):
        assert iscallable(func)
        new_func = self.wrap(func)
        assert iscallable(new_func)
        self.copy_attributes(func, new_func)
        return new_func

    def copy_attributes(self, old_func, new_func):
        ...

func_wrapper would be used for things like synchronized and memoize, and
attribute would be used for most of the other examples in my previous
message.

3. Provide a clear set of guidelines for good decorators, i.e.
  a. decorators should not have side effects outside of the class/function
  b. decorators should return the same kind of thing they are given
  c. decorators should be commutative whenever possible (requires b)


+ sys._getframe()

There should be a consensus on whether or not decorators like 'generic' are
a good or a bad idea.  If they're a good idea then the decorate method
should get an extra argument exposing the local namespace so that
sys._getframe() won't be required.  I'd be in favor of adding that argument
to decorate.


+ classmethod and staticmethod

These two decorators are a big problem because they don't return the same
kind of thing as they are given.  This means they always have to be the last
decorator applied in any list.  That kind of restriction is never a good
thing.

Personally, I'd like to see these changed from returning a new object to
instead setting an attribute on the existing function/method object.  Then
the method object could use this attribute to implement its __get__ method
correctly.  This would make these decorators compose very nicely with all of
the other decorators; however, it would be a fairly major change to how
Python handles static and class methods and I'm doubtful that would fly.


* Syntax

I've come to see syntax as less crucial than I originally thought.  In
particular, the examples I've read on this list of using temporary variables
showed me that even if the syntax is too confining there are tolerable
solutions.  Nevertheless, I'm still strongly in favor of putting the
decorator list in front of the def/class.

Decorators/attributes are an exciting new technique that's gaining
increasing use in different programming languages.  They're important enough
to be one of the key ideas that Java-1.5 is stealing from C# (turn-about is
always fair play).  The AOP community often has vehement debates not about
the value of attributes, but about how they can be most effectively used.

Because decorators are still a relatively new idea, it's hard to know how
much room they will need.  Certainly if we're only interested in classmethod
and staticmethod it's not much.  However, I think it's important to give
decorators as much room as possible to be prepared for the future.

I have some sympathy with the arguments that a bare prefix list is already
legal Python syntax and is certainly found in existing code.  I'm quite sure
that using a bare list would break existing Python programs that have
dangling lists left over as they evolved.

I like Sam's proposal to add a '*' in front of the list.  I could also be
persuaded to go along with any of the new grouping constructing, like <| |>;
however, those all seem like too much of a change for the value.  I'm fairly
confident that the *[] form would allow for attributes to optionally be put
on the same line as the def.  This would be LL1, but perhaps the Python
parser has other restrictions that I'm unaware of, i.e.

*[java_sig("public void meth(int a, int b)"), staticmethod]
def meth(self, a, b):
    pass

or for simple cases

*[staticmethod] def meth(self, a, b):
    pass

-Jim