[Python-Dev] PEP 318: Decorators last before colon

[Guido van Rossum]

> There appears to be a strong correlation between people who have
> specific use cases for decorators, and the people who want the
> last-before-colon syntax.  Whereas, people who have few use cases
> (or don't like decorators at all) appear to favor syntaxes that move
> decorators earlier.  Whether that means the "earlier" syntaxes are
> better or worse, I don't know.  <0.5 wink>

Maybe the practitioners are so eager to have something usable that
they aren't swayed as much by esthetics.

> I'll assume your intent is to prevent decorators from biting the
> unwary -- specifically people who *don't* use decorators a lot and
> therefore are not looking for them.  I will therefore focus now on
> the issues with the "previous line" syntax that may bite people,
> with an eye to how they might be fixed.

They should still be very unlikely to accidentally create one.

With proper vertical whitespace, the fact that a decorator list
(written properly) means something special should be obvious to even
the most casual observer: there should be a blank line before the
decorator list and none between it and the 'def' line.

The human brain is a lot more flexible in picking up patterns than the
Python parser; as shown many times in this discussion, most people
have no clue about the actual syntax accepted by the Python parser,
and simply copy (and generalize!) patterns they see in examples.

My goal is to help people grasp the gist of a particular construct
without having to think too much about the exact syntax; within all
sorts of other constraints of course, like being parsable with the
simple ans stupid LL(1) parser, and being easy to grep (in some
cases).

> > > By the way, you didn't mention whether it's okay to put the
> > > decorators on the same logical line, e.g.:
> > >
> > > [classmethod] def foo(bar,baz):
> > >      # body goes here
> > >
> > > If the rationale here is that we're copying C#, I'd think that it
> > > should be permissible, even though it looks a bit ugly and tempts me
> > > to indent the body to align with the function name.
> >
> >This is much harder to do with the current parser.  (My plan would
> >be to tie the list expression and the function definition together
> >in the code generating phase, just like doc strings.)
> 
> Yeah, this is also going to now have to be a special case for
> documentation processing tools.

I hadn't thought of those, but the new situation can't be worse than
before (decorations following the 'def').  Usually these tools either
use some ad-hoc regex-based parsing, which shouldn't have any problems
picking out at least the typical forms, or they (hopefully) use the
AST produced by the compiler package -- it should be possible for that
package to modify the parse tree so that decorators appear as part of
the Function node.

> Whereas making it part of the definition syntax, it's more directly
> available in the parse tree.

Definitely, but I don't see it as a showstopper.  I've now implemented
this for the traditional compile.c; I think the effort is fairly
moderate.  (My patch is smaller than Michael Hudson's.)

> It also seems to be working against the AST branch a bit, in that I
> would expect the decorator expressions to be part of the function
> definition node, rather than in an unrelated statement.  And, it's
> also going to be interesting to document in the language reference,
> since the grammar there is going to continue to diverge from the
> "real" grammar used by the implementation.

Hardly worse than the difference between theory and practice for
assignment statements or keyword parameters.

> Another issue...  is this valid?
> 
> [classmethod]
> 
> def foo(bar,baz):
>      pass

Yes.

> How about this?
> 
> [classmethod]
> # Okay, now we're going to define something...
> def foo(bar,baz):
>      pass

Yes.

> If they *are* valid, then you can have nasty effects at a distance.

Given that there really isn't much of a use case for putting a list
display on a line by itself (without assigning it to something), I
don't see this as a likely accident.  By giving only sane examples
we'll help people write readable code (I see no other way; you can't
force people to write unobfuscated code :-).

> If they *aren't* valid, accidentally adding or removing whitespace
> or comments can silently change the meaning of the program, and
> *not* in a DWIMish way.
> 
> I personally would rather have the decorators required to be on the
> same logical line, and then use:
> 
> [classmethod] \
> def foo(bar,baz):
>      pass
> 
> for visual separation.  The backslash visually alerts that this is
> *not* a mere bare list.

Ugly, and the stupid LL(1) parser can't parse that -- it needs a
unique initial symbol for each alternative at a particular point.

> I'm not a parser guru by any stretch of the imagination, but
> wouldn't it be possible to simply create a statement-level construct
> that was something like:
> 
> liststmt: '[' [listmaker] ']' ( funcdef | restofliststmt )
> 
> and put it where it matches sooner than the expression-based versions of 
> the statement?

Maybe in SPARK, but not in Python's LL(1) parser.

> It seems like the main complexity would be the possibility of having
> to duplicate a number of levels of containing rules for
> 'restofliststmt'.  But maybe I'm completely off base here and
> there's no sensible way to define a correct 'restofliststmt'.

You can assume that's the case. :)

--Guido van Rossum (home page: http://www.python.org/~guido/)