Guido writes:

One advantage mentioned by Fredrik Lundh of this, and also of my other favorite, over "decorators-after-args" is that the decorators are easily cut-and-pasted -- it's much easier to cut whole lines than sections of a line, and cutting a multiple-lines-spanning section is even worse.

Yeah, it makes the code marginally easier to edit, but it also makes the code harder to read. Used to be that "def" and "class" would stand out in the file, being at the left edge. Now you've got these new lines, with, God help us, new syntax, cluttering up the left edge. What's even worse, these decorators come before the function, so you encounter them before you even know what function you're talking about. A reading nightmare. I still prefer "def FUNCTION (ARGS) as DECORATOR[S]:", myself. Bill

Paul Prescod writes:

The left edge is already cluttered.

a = 5 b = a() def foo(xyz): abc = 5 c = 20

It is the indentation that alerts your eye to the function definition.

Paul, most modules have global variables defined only in one place, usually well-marked, so you don't usually get the clutter that you cite here. Scripts, of course, are an exception.

Nobody reads code one line at a time from the top down. You scan for the bit that interests you and then look around for its context.

It's of course the scanning that I'm talking about. Having only "def" and "class" at the left edge makes it remarkably easy to scan for a function or class definition. Putting oddly-shaped decorators there too ruins this. But I disagree with your statement here. Everyone reads *unfamiliar* code one line at a time from the top. If you're really reading it, I mean, rather than just looking for something in it. Bill

Paul, I think you're proving my point. There's an interesting paper on www.liveink.com where they claim that one of the problems with classically formatted text is that the lines above and below the line that's currently being read make it harder to focus on that line. To me, the same seems true in your examples of the @foo decoration syntax. Bill

The left edge is already cluttered.

a = 5 b = a() def foo(xyz): abc = 5 c = 20

It is the indentation that alerts your eye to the function definition.

But the indentation alone doesn't distinguish a def from an if or while or any other indented thing. Most people would leave some space before and after the function definition: a = 5 b = a() def foo(xyz): abc = 5 c = 20 Unfortunately, leaving space between the decorators and the function would break the visual association between them. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

In article <200406250357.i5P3vRNh020599@cosc353.cosc.canterbury.ac.nz>, Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:

...

If people prefer my favorite (decorator-list-before-def, C# style) over Java-style, that's fine too... :-)

I don't really like either of them, but to my eyes your original one is definitely the least worst. I find the @-signs extremely ugly.

I wouldn't say they're pretty, but I like them better than the alternative C# list-followed-by-def-magically-becomes-decorator. And the argument about being able to cut and paste them a line at a time makes sense to me too. -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science

David Abrahams <dave@boost-consulting.com>:

You can go on introduce various syntactic abominations using the special operator methods without actually changing the language syntax, too.

...

+staticmethod +foodstyle(sandwich) def eject(tomato): ... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

At 01:26 PM 6/24/04 -0400, David Abrahams wrote:

Anthony Baxter <anthony@interlink.com.au> writes:

...

So, let the floodgates open. Remember, we _can_ change this any time up until 2.4b1, if there's a decision that the chosen form sucks. :-)

Here's thinking in a different direction altogether:

No special syntax

Instead, expose enough functionality in standard library functions that an appropriately-written pure-python "decorate" function can do it.

decorate(staticmethod, my_decorator) def f(x): whatever()

Since function definitions are executable statements, it should in principle be possible to arrange something that allows one to hook the execution of that statement. Perhaps it's already doable with the debugger hook?

Hmmm. You probably *could* create such a function in CPython, maybe even as far back as 2.1 (since all it needs is sys._getframe and the tracing hook), but it wouldn't be portable to Jython. And, boy, would it be a sick and twisted piece of code. You'd need to do something like keep a copy of the locals and on each "new line" event from the trace hook, you'd need to both call the old trace hook (so that debugging could still take place) and check to see if the locals had a new function object. As soon as the function object appeared, you could do your dirty work (after first restoring the *old* value bound to that name, so the decorators would have access to the previous binding of the name). About the only bit you couldn't do in pure Python would be decorating a function defined in a nested scope, because "fast locals" aren't accessible from pure Python. But for module-level functions and methods in classes, this might actually work. Heck, I'm rather tempted to try to write it and see if it does. At worst, it might be a fun way to learn the ins and outs of sys.settrace(). [goes away for 30 minutes or so] Hm, interesting. After a bit of twiddling, I've got a rough draft of the trace mechanism. It seems to be possible to chain tracer functions in the manner described, and I got it to work even with pdb running. That is, if the debugger was already tracing a frame, adding co-operative trace functions didn't appear to disturb the debugger's operation. However, there are certainly some "interesting" peculiarities in the trace function behavior. There seems to be a bit of voodoo necessary. Specifically, it seems as though one must call 'sys.settrace()' *and* set 'frame.f_trace' in order to change a local frame trace function, even though in theory just returning a new trace function from a 'line' event should work. Ah well. Here's a proof-of-concept version, that uses right-to-left application and simple calling for the decorators, but as you'll see, the bulk of the code is devoted to mechanism rather than policy. You'll see that it's easy to write different policies, although the way I have the trace mechanism set up, later tracers execute *before* earlier tracers. I'm not sure if this is the right way to go, but it seems to read best with this syntax. Anyway, it works with CPython 2.2. Caveat: do *not* use listcomps or any other assignment statements between your 'decorate()' invocation and the function definition, or the decorators will be applied to the bound-to variable! Personally, I think the 'decorate()' function is probably actually not that useful, compared to making specialized decoration functions like, say: classmethod_follows() def foo(klass, x, y, z): # ... Only with better names. :) Also, using this approach means you can do something like: [classmethod_follows()] def foo(klass, x, y, z): # ... which comes awfully close to Guido's preferred syntax! Anyway, concept code follows, and I'm quite tempted to add this to PyProtocols, which already has a similar function for class decoration. # ======= import sys def add_assignment_advisor(callback,depth=2): frame = sys._getframe(depth) oldtrace = [frame.f_trace] old_locals = frame.f_locals.copy() def tracer(frm,event,arg): if event=='call': if oldtrace[0]: return oldtrace[0](frm,event,arg) else: return None try: if frm is frame and event !='exception': for k,v in frm.f_locals.items(): if k not in old_locals: del frm.f_locals[k] break elif old_locals[k] is not v: frm.f_locals[k] = old_locals[k] break else: return tracer callback(frm,k,v) finally: if oldtrace[0]: oldtrace[0] = oldtrace[0](frm,event,arg) frm.f_trace = oldtrace[0] sys.settrace(oldtrace[0]) return oldtrace[0] frame.f_trace = tracer sys.settrace(tracer) def decorate(*decorators): if len(decorators)>1: decorators = list(decorators) decorators.reverse() def callback(frame,k,v): for d in decorators: v = d(v) frame.f_locals[k] = v add_assignment_advisor(callback) decorate(classmethod) def f(klass,x,y): pass print f

At 11:01 AM 6/25/04 +0100, Armin Rigo wrote:

Hello Phillip,

On Fri, Jun 25, 2004 at 01:27:03AM -0400, Phillip J. Eby wrote:

...

About the only bit you couldn't do in pure Python would be decorating a function defined in a nested scope, because "fast locals" aren't accessible from pure Python.

Would you believe it? Your implementation works just fine in this case too :-) sysmodule.c invokes PyFrame_FastToLocals and PyFrame_LocalsToFast around a call to a trace function, so that any change to f_locals from the trace function are actually correctly reflected into the frame.

Wow. You're right, it does work. The time machine strikes again. :)

Excellent. Let's put this in the standard library and close the issue ;-)

Ah, if only that *would* close the issue. It really just opens more, I'm afraid.

As an easy extension, decorate(attr=value, ...) could just assign attributes to the function object. Also, the fact that it works with other assignment statements is a bonus in my opinion, e.g.:

See, I told you it opens more issues. Among them: * The location in the stdlib of 'decorate' * The signature of 'decorate', including whether it should even be *called* 'decorate', or something else. * Whether to have a 'decorate' function at all, or whether it's better to just list specialized decorator objects/functions (I generally favor the latter, since 'decorate' is IMO a dead chicken that hasn't appeared in any widely-supported syntax proposals to date). * Which of the numerous possible syntax variations should be used/recommended: +classmethod +attrs(spam="fidget") def foo(...): ... [classmethod(), attrs(spam="fidget")] def foo(...): ... ~classmethod ~attrs(spam="fidget") def foo(...): ... -classmethod -attrs(spam="fidget") def foo(...): ... All of these are possible, and each likely to have *some* proponents. Oh, and let's not forget the question of application order of the decorations. :) I personally favor a(b(c(...))) ordering for decorations that are listed before the 'def', even though I favor a(), b(), c() ordering for decorations that come after. Anyway, I think it's probably more realistic to say that the presence of a viable implementation approach that allows numerous policies and syntax variants to be experimented with, even in existing Python versions, means it's more likely that Guido will *not* allow this in the stdlib for 2.4, and leave it for the frameworks that need this to fight it out amongst themselves. I'm not sure if this is good or bad. Personally, I'd prefer Guido to "bless" one of the above syntax alternatives as the recommended way to do these things, along with a "blessed" order in which decorators should be applied. Then, in any of my frameworks that I implement this for, I'll be able to be "forward compatible" with other frameworks that do the same. By the way, I should also mention that I think with this approach, my strongest objection to Guido's original syntax proposal goes away: if you have to *call* the decorators, it makes it more obvious that something is happening. When you could just list '[classmethod]', it's not obvious that that line *does* anything. If you have '[classmethod()]', it's apparent that something is being invoked.

As I see it, any decorator that appears on lines previous to a function impairs code readability by making the function harder to find. When editing code with the standard 4-space tab, the function name is very conveniently lined up with the code below it, and in most code I've seen, there's a blank line above it, like this: def something(foo): do_something() def somethingelse(bar): do_something_else() Now say we were to add decorators to this. Then visually, to find the function, you'd be searching through a non-predetermined set of decorators first, as opposed to searching for a solid piece of whitespace then scrolling your eyes one line down, a fixed position from the left edge of the editor. The function name would be embedded in noise, making the code more confusing to edit. This particular issue can be remedied in two ways. First off, a list of decorators with brackets, as Phillip's solution makes possible, but with some limitation to prevent multiple lines of them (or at least a prescribed coding convention prohibiting it without true enforcement). It would still take away from readability a little bit, but not in such a bad way. The brackets actually set off the decorators nicely. Something like: [decorators] def func(args): pass The other solution I see is something like what Bill said which is: def func (args) as [decorators]: The idea of bracketing them as a list is, I think, a very good one, and it would of course still have to be determined whether decorators are functions, whether they should be called, etc. I kind of like the idea of them being functions that are called on the function, and if they need additional arguments, make them higher order functions such that the decorator list element is called with func as its argument. Just my 2c, Brian On Fri, 25 Jun 2004 16:18:38 -0400, Fred L. Drake, Jr. <fdrake@acm.org> wrote:

On Friday 25 June 2004 04:09 pm, Phillip J. Eby wrote:

...

* Whether to have a 'decorate' function at all, or whether it's better to just list specialized decorator objects/functions (I generally favor the latter, since 'decorate' is IMO a dead chicken that hasn't appeared in any widely-supported syntax proposals to date).

Ooohh... time for some fun! Some proposals have included "as". Since "as" isn't really a keyword, how about:

from decorate import decorate as as

as(classmethod) def myAlternateConstructor(...): # make something interesting: return 42

Too bad this is ugly for the simple attribute decorator:

as(spam="fidget") def foo(...): send_fred_more_spam_please()

-Fred

-- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/zorander%40gmail.com

-- The years ahead pick up their dark bags. They move closer. There's a slight rise in the silence then nothing.

On Jun 25, 2004, at 5:51 PM, Jeff Bone wrote:

On Jun 25, 2004, at 3:52 PM, Brian L. wrote:

...

such a bad way. The brackets actually set off the decorators nicely. Something like:

[decorators] def func(args): pass

+1

IMHO, making decorators functions is a bad idea. Decorators are metadata about the function to which they refer. Using the same syntax for decorators as for functions conceptually weakens this distinction and encourages misuse of decorators, even to the point of potentially encouraging (or implying) side-effects as a result of decoration.

Uh, the WHOLE POINT we want this is to have side-effects. If it doesn't make the function act in a different way, it might as well live in the doc string or something. The most wanted use cases are all function transformations, not the setting attributes on function objects. -bob

At 09:23 AM 6/26/04 -0500, Jeff Bone wrote:

...

Uh, the WHOLE POINT we want this is to have side-effects. If it doesn't make the function act in a different way, it might as well live in the doc string or something.

Declarative vs. imperative. Design-time *definitional* modification of behavior vs. runtime. I don't really think you want side-effects in the literal sense.

Actually, he very well might, and a number of other people definitely do. One use case repeatedly mentioned on Python-Dev has been to register functions or classes with frameworks, therefore definitely involving a literal side-effect (i.e. modification to an unrelated data structure). (I wonder if I should submit a patch for PEP 318 to add this to the motivation section, because it seems a lot of people keep repeating this "no side-effects metadata" misconception.)

E.g. "classmethod" isn't a side-effect, it's a definitional thing. If you really want side-effecting operations on functions, you've already got that given higher-order / first-class functions.

Please read the current "Motivation" section of PEP 318. The point isn't that such operations aren't possible, it's that they're not very readable in standard Python syntax. In contrast, most other languages with first-class functions allow function definitions to be used as *expressions*, which means that they can be nested within a decoration expression, giving the reader a context for understanding the function definition. PEP 318 seeks to remedy this clarity/readability issue, by providing an alternative syntax for expressions that will "wrap" a function definition that follows.

Jeff Bone <jbone@deepfile.com> writes:

...

Uh, the WHOLE POINT we want this is to have side-effects. If it doesn't make the function act in a different way, it might as well live in the doc string or something.

Declarative vs. imperative. Design-time *definitional* modification of behavior vs. runtime. I don't really think you want side-effects in the literal sense. E.g. "classmethod" isn't a side-effect, it's a definitional thing. If you really want side-effecting operations on functions, you've already got that given higher-order / first-class functions.

I'm a big fan of declarative programs. That said, declarativeness (especially in Python) is more a matter of "notational flavor" than anything we can measure. It's certainly independent of whether there are actual side-effects. Also, I'm suspicious of any heroic efforts to prevent language features from being "abused". Some of the most powerful techniques in many languages were not envisioned by the language designers. The languages I like (by their nature) tend to encourage the invention of these unforseen techniques. Python is one of them. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Jeff Bone <jbone@place.org> writes:

NB, I'm no type Nazi.

NB: I am one. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

On Jun 25, 2004, at 3:52 PM, Brian L. wrote:

such a bad way. The brackets actually set off the decorators nicely. Something like:

[decorators] def func(args): pass

+1 IMHO, making decorators functions is a bad idea. Decorators are metadata about the function to which they refer. Using the same syntax for decorators as for functions conceptually weakens this distinction and encourages misuse of decorators, even to the point of potentially encouraging (or implying) side-effects as a result of decoration. IMHO, jb

I could also see the function call version: from decorators import decorate ... def foo (bar): decorate(bletch, flappy) ... where decorate is automatically applied to "foo". Bill

Yes, it would have to be a 'compile-time' macro of some sort. Bill

At 09:47 PM 6/25/04 -0700, Bill Janssen wrote:

...

I could also see the function call version:

from decorators import decorate ... def foo (bar): decorate(bletch, flappy) ...

where decorate is automatically applied to "foo".

That won't work, since by the time the function is called it's too late to redefine foo.

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

On Friday 25 June 2004 04:09 pm, Phillip J. Eby wrote:

...

* Whether to have a 'decorate' function at all, or whether it's better to just list specialized decorator objects/functions (I generally favor the latter, since 'decorate' is IMO a dead chicken that hasn't appeared in any widely-supported syntax proposals to date).

Ooohh... time for some fun! Some proposals have included "as". Since "as" isn't really a keyword, how about:

from decorate import decorate as as

as(classmethod) def myAlternateConstructor(...): # make something interesting: return 42

Too bad this is ugly for the simple attribute decorator:

as(spam="fidget") def foo(...): send_fred_more_spam_please()

I don't find it so ugly... but then there's always "with" ;-) -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

"Phillip J. Eby" <pje@telecommunity.com> writes:

* Which of the numerous possible syntax variations should be used/recommended:

Just a reminder, but I believe that Guido has already pronounced to the effect that there are only 3 contenders left: @staticmethod def foo(a,b,c): pass [staticmethod] def foo(a,b,c): pass and def foo(a,b,c) [staticmethod]: pass Hey - guess what? Writing them out like that, I find that my initial preference for the 3rd one has gone. To be honest, I no longer have a strong opinion between the three. In fact, if anything, I'd say I have a very mild preference for Guido's option (number 2). I dislike the ambiguity problem with it, though. Paul -- "Bother," said the Borg, "We've assimilated Pooh."

"Phillip J. Eby" <pje@telecommunity.com> writes:

...

@staticmethod def foo(a,b,c): pass

[staticmethod] def foo(a,b,c): pass

and

def foo(a,b,c) [staticmethod]: pass

Hey - guess what? Writing them out like that, I find that my initial preference for the 3rd one has gone. To be honest, I no longer have a strong opinion between the three. In fact, if anything, I'd say I have a very mild preference for Guido's option (number 2). I dislike the ambiguity problem with it, though.

IMO, the ambiguity is resolved if some kind of visible invocation occurs, e.g.:

[declare_staticmethod()] def foo(a,b,c): pass

While '[staticmethod]' by itself may appear to be a useless and ignorable statement, adding the () makes it clear that you have to know what the called function *does* before you could consider removing the statement.

When you add in the fact that this syntax can be made to work in Python 2.2 and better, not just 2.4, it makes the syntax a big winner in my book.

I would like to reverse the application order, though: Guido's patch uses the same order for syntax 2 as for syntax 3, and now that I've played with syntax 2 "for real", I think that right-to-left application should be used for any syntax where the decorators come before the 'def'.

Here's another syntactic idea that could be made to work without core language changes: [as.staticmethod] def foo(a,b,c): pass I think that one's pretty sweet. It can be extended any number of ways: [as.staticmethod, as.classmethod, ...] def foo(a,b,c): pass [as.staticmethod|classmethod| ...] def foo(a,b,c): pass -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

Barry Warsaw <barry@python.org> writes:

On Sat, 2004-06-26 at 11:27, Paul Moore wrote:

...

Just a reminder, but I believe that Guido has already pronounced to the effect that there are only 3 contenders left:

I've said before that I'm past caring which syntax is chosen. I could live with any of these three. I just want the feature in 2.4 and I trust Guido to Make The Right Decision.

A-fucking-men. Cheers, mwh (who apologies for the crudity, but would also like an apology from all the people who bugged him about this topic at EuroPython (apart from Guido :-)) -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998

At 08:26 AM 6/25/04 -0400, David Abrahams wrote:

"Phillip J. Eby" <pje@telecommunity.com> writes:

...

You'd need to do something like keep a copy of the locals and on each "new line" event from the trace hook, you'd need to both call the old trace hook (so that debugging could still take place) and check to see if the locals had a new function object. As soon as the function object appeared, you could do your dirty work (after first restoring the *old* value bound to that name, so the decorators would have access to the previous binding of the name).

I'm perfectly happy for this decorate function to work on the very next name binding; I don't think it's reasonable to expect to be able to do anything else between it and the function definition.

Keep in mind that this means code like this: decorate(foo) decorate(bar(baz=[x*2 for x in y])) def spam(): pass Will likely fail horribly, applying the 'foo' decorator to the first element of 'y' before trying to multiply by 2, or worse, possibly applying them to the "anonymous" list object that's created to hold the results of the list comprehension. Or, it might sometimes *seem* to work due to hashing order and the vagaries of when 'line' events fire, and then fail in a new release of Python. This is why I consider it somewhat of a wart in the current implementation, as I note below...

...

Anyway, it works with CPython 2.2. Caveat: do *not* use listcomps or any other assignment statements between your 'decorate()' invocation and the function definition, or the decorators will be applied to the bound-to variable!

Personally, I think the 'decorate()' function is probably actually not that useful, compared to making specialized decoration functions like, say:

classmethod_follows() def foo(klass, x, y, z): # ...

Ick!

Is it the name you dislike? Or the idea of simply invoking decorating functions directly?

...

Only with better names. :) Also, using this approach means you can do something like:

[classmethod_follows()]

Ick in brackets ;-> !