[Python-ideas] Proposal for new-style decorators

Christophe Schlick cschlick at gmail.com
Tue Apr 26 18:52:11 CEST 2011


Hi Jim,

The new version. Now cut-and-paste code should work...

------
Hello everybody,

My name is Christophe Schlick, from the University of Bordeaux,
France. I've been using Python for years, but it happens that this is
my first post on a Python-related mailing list. For the first time, I
feel that I may have some topic interesting enough for the Python
community, so I would be very happy to get any kind of feeback
(positive or negative) on it. Thanks in advance...

The goal of this post is to propose a new syntax for defining
decorators in Python. I would like to fully explain the rationale and
the benefits of this proposal, as I guess that the discussion might be
more fruitful if all details are written down once for all. However
this has led me to a very long post (a kind of informal PEP) that was
blocked by the moderators. So, following their recommendation, I've
divided the text in three successive parts: (1) the rationale, (2) the
proposal, (3) the proposed implementation and additional
questions/remarks.

For better clarity of the description, I will call the proposed syntax
as "new-style decorators" (NSD, for short) and rename the classic
syntax as "old-style decorators" (OSD), following the terms used some
years ago with the (re)definition of classes. By the way, the
introducing process for NSD shares many aspects with the process used
for introducing new-style classes, including the following features:

* No existing syntax is broken: the only thing required to create a
new-style decorator is to decorate itself by a newly-introduced
decorator called... "decorator" (well, this sentence is less recursive
than it might appear at the first reading).

* Every thing that can be done with OSD is possible with NSD, but NSD
offer additional more user-friendly features.

* NSD can peacefully live together with OSD in the same code. An NSD
may even decorate an OSD (and vice-versa), however some properties of
the NSD are lost with such a combination.

--------------------------------------------------
1 - Why bother with a new syntax?

To explain what I don't like with the current syntax of decorators,
let me take the example of a basic decorator (called
'old_style_repeat_fix') that simply repeats 3 times its undecorated
function, and adds some tracing to the standard output. Here is the
code:

#---
def old_style_repeat_fix(func):
  """docstring for decorating function"""
  # @wraps(func)
  def dummy_func_name_never_used(*args, **keys):
    """docstring for decorated function"""
    print "apply 'old_style_repeat_fix' on %r" % func.__name__
    for loop in range(3): func(*args, **keys)
  return dummy_func_name_never_used
#---

Even if such code snippets have become quite usual since the
introduction of decorators in Python 2.2, many people have argued (and
I am obviously one of them) that the decorator syntax is a bit
cumbersome. First, it imposes the use of nested functions, which often
reduces readability by moving the function signature and docstring too
far from the corresponding code. Second, as anonymous lambdas
expressions can usually not be employed for decorating functions, the
programmer has no other choice than to create a dummy function name
(only used for one single 'return' statement), which is never a good
coding principle, whatever the programming language. Once you have
tried to teach decorators to a bunch of students, you really
understand how much this syntax leverages the difficulty to grab the
idea.

The situation is even worse when the decorator needs some arguments:
let's create an extended  decorator (called 'old_style_repeat_var)
that includes an integer 'n' to control the number of iterations, and
a boolean 'trace' to control the tracing behavior. Here is the code:

#---
def old_style_repeat_var(n=3, trace=True):
  """docstring for decorating function"""
  def dummy_deco_name_never_used(func):
  """docstring never used"""
    # @wraps(func)
    def dummy_func_name_never_used(*args, **keys):
      """docstring for decorated function"""
      if trace:
        print "apply 'old_style_repeat_var' on %r" % func.__name__
      for loop in range(n): func(*args, **keys)
    return dummy_func_name_never_used
  return dummy_deco_name_never_used
#---

This time a two-level function nesting is required and the code needs
two dummy names for these two nested functions. Note that the
docstring of the middle nested function is even totally invisible for
introspection tools. So whether you like nested functions or not,
there is some evidence here that the current syntax is somehow
suboptimal.

Another drawback of OSD is that they do not gently collaborate with
introspection and documentation tools. For instance, let's apply our
decorator on a silly 'test' function:

#---
@old_style_repeat_var(n=5) # 'trace' keeps its default value
def test(first=0, last=0):
  """docstring for undecorated function"""
  print "test: first=%s last=%s" % (first, last)
#---

Now, if we try 'help' on it, we get the following answer:

#---
>>> help(test)
dummy_func_name_never_used(*args, **keys)
   docstring for decorated function
#---

which means that neither the name, nor the docstring, nor the
signature of the 'test' function are correct. Things are a little
better when using the 'wraps' function from the standard 'functools'
module (simply uncomment the line '@wraps(func)' in the code of
'old_style_repeat_var'):

#---
>>> help(test)
test(*args, **keys)
   """docstring for undecorated function"""
#---

'@wraps(func)' copies the name and the docstring from the undecorated
function to the decorated one, in order to get some useful piece of
information when using 'help'. However, the signature of the function
still comes from the decorated function, not the genuine one. The
reason is that signature copying is not an easy process. The only
solution is to inspect the undecorated function and then use 'exec' to
generate a wrapper with a correct signature. This is basically what is
done in the 'decorator' package (available at PyPI) written by Michele
Simionato. There has been a lengthy discussion in python-dev (in 2009
I guess, but I can't find the archive right now) whether to include or
not this package in the standard library.

As far as I know, there is currently no clear consensus whether this
is a good idea or not, because there has always been a mixed feeling
from the community about transparent copy from the undecorated to the
decorated function (even about the 'wraps' function): on one hand,
transparent copy is cool for immediate help, for automatic
documentation and for introspection tools, but on the other hand, it
totally hides the decorating process which is not always what is
wanted... or needed.

The syntax for NSD presented in this proposal tries to improve this
situation by offering two desirable features, according to the Zen of
Python:

* "flat is better than nested": no nested functions with dummy names
are required, even when parameters are passed to the decorator; only
one single decorating function has to be written by the programmer,
whatever the kind of decorator.

* "explicit is better than implicit": introspection of a decorated
function explicitely reveals the decoration process, and allows one to
get the name/signature/docstring not only for the corresponding
undecorated function, but also for any number of chained decorators
that have been applied on it.

------
to be continued in Part 2...

CS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20110426/86bb1def/attachment.html>


More information about the Python-ideas mailing list