[Python-Dev] Rejecting the J2 decorators proposal

[Guido van Rossum]

Robert and Python-dev,

I've read the J2 proposal up and down several times, pondered all the
issues, and slept on it for a night, and I still don't like it enough
to accept it. The only reason to accept it would be to pacify the
supporters of the proposal, and that just isn't a good enough reason
in language design.

However, it got pretty darn close! I'm impressed with how the
community managed to pull together and face the enormous challenge of
picking a single alternative (from more than two dozen on the Wiki!)
and arguing consistently. I expect to see more proposals like this in
the future, and I'm sure that some of them will be good enough to make
it into the language.

I've also (again) learned a lesson: dramatic changes must be discussed
with the community at large. In a large enough group there are no
uncontroversial proposals, so this will take time, but it's worth it
-- one of the main issues with the @decorator syntax was not technical
but socio-political, in the sense that it hadn't been properly
discussed outside a *very* small circle. I take the full blame for
that, and I don't want to hide behind my current lack of time which,
realistically, won't change until either my ESI stock options earn me
an early retirement, or the PSF strikes it rich and can pay me full
time :-).

So let me explain why I'm not choosing J2, and what's next.

There are two major issues and one minor that made me decide against J2.

Major issue one: the syntactic form of an indented block strongly
suggests that its contents should be a sequence of statements, but in
fact it is not -- only expressions are allowed, and there is an
implicit "collecting" of these expressions going on until they can be
applied to the subsequent function definition. To me, this is a more
serious problem than the namespace questions brought up in the
proposal (unfortunately that particular section of the proposal is its
most confused part; but even if the text had been crystal clear, the
problem remains). The best counter-argument to this I've heard is
"you'll get used to it", which is also what I'm saying of @decorators;
and many people have already testified that they indeed got used to it
and even liked it.

Major issue two: the keyword starting the line that heads a block
draws a lot of attention to it. This is true for "if", "while", "for",
"try", "def" and "class". But the "using" keyword (or any other
keyword in its place) doesn't deserve that attention; the emphasis
should be on the decorator or decorators inside the suite, since those
are the important modifiers to the function definition that follows.
When a function definition carries one or more decorators, the most
important information is not the fact that it has decorators, but the
specific decorators used. A classmethod or staticmethod decorator adds
a completely different flavor than a decorator that provides an
external linkage hint for ObjC, or one that adds synchronization, or
one that declares deprecation. I expect that at least 80% of the use
of decorators will have a single decorator per function, and it's a
pain for that decorator to be hiding behind a content-free keyword.
(This is *not* a number-of-keystrokes argument. You know I don't care
much about that.)

Minor issue: "using" is a poor choice of keyword. It resembles C#'s
"using" and perhaps Perl's "use", both of which have completely
different meanings. But there don't seem to be any better alternatives
(the best I could come up with was "transmogrify" :-).

So, what's next? In Python 2.4a3 (to be released this Thursday),
everything remains as currently in CVS. For 2.4b1, I will consider a
change of @ to some other single character, even though I think that @
has the advantage of being the same character used by a similar
feature in Java. It's been argued that it's not quite the same, since
@ in Java is used for attributes that don't change semantics. But
Python's dynamic nature makes that its syntactic elements never mean
quite the same thing as similar constructs in other languages, and
there is definitely significant overlap. Regarding the impact on 3rd
party tools: IPython's author doesn't think there's going to be much
impact; Leo's author has said that Leo will survive (although it will
cause him and his users some transitional pain). I actually expect
that picking a character that's already used elsewhere in Python's
syntax might be harder for external tools to adapt to, since parsing
will have to be more subtle in that case. But I'm frankly undecided,
so there's some wiggle room here. I don't want to consider further
syntactic alternatives at this point: the buck has to stop at some
point, everyone has had their say, and the show must go on.

In the coming years I hope that as a community we'll gain enough
experience with decorators to decide whether we need to adopt a
different syntax for Python 3000 or not. One of the difficulties with
choosing a decorator syntax has definitely been that nobody can
predict how they are going to be used predominantly. Different
alternatives look better depending on whether there are many or few
decorators per function, whether they have long argument lists or not,
and perhaps also whether their use is for transformation or for
annotation. Despite the novelty of using the @ character, I personally
feel that prefix decorators are a huge improvement over the "f =
staticmethod(f)" style of decorating.

A warning: some people have shown examples of extreme uses of
decorators. I've seen decorators proposed for argument and return type
annotations, and even one that used a decorator to create an object
that did a regular expression substitution. Those uses are cute, but I
recommend being conservative when deciding between using a decorator
or some other approach, especially in code that will see a large
audience (like 3rd party library packages). Using decorators for type
annotations in particular looks tedious, and this particular
application is so important that I expect Python 3000 will have
optional type declarations integrated into the argument list.

Thanks to everyone who read until the end of this message!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)