Last time I wrote in favour of removing the restriction on decorator syntax, an unfortunate offhand remark of mine about something being more Pythonic led to a tangential debate about what was Pythonic and whether concision was always good, etc., with almost everyone ignoring the issue I was actually raising. An honourable exception was Brett Cannon, who answered my points squarely (and did his best to shoot me down in flames). So please, back to the point. I argued in favour of removing the syntax restriction (i.e. allowing any expression after '@') on the following grounds, which I paraphrase from my original: 1(a)) It was inconsistent to impose a restriction on expressions in one particular context [meaning: and not in others]. 1(b)) The restriction was hard to explain [but see below]. 2) The restriction can easily be circumvented. 3) Plausible use cases exist [I gave some]. I would like to answer some of Brett's points here: I wrote: "Guido has said he has a 'gut feeling' against [removing the restriction] but has not as far as I know rationalised it." Brett wrote: "When it comes to Guido's gut, a rationalization isn't needed. Perk of being BDFL. Plus his gut is right so often it tends to not be questioned." I am sure that Guido (may-he-live-forever) would be the first to agree that saying an opinion of his should NEVER be questioned is just, well, silly. When I said in point 1(b)) above that the restriction was hard to explain, this was ambiguous. Did I mean (i) It was hard to explain why there was a restriction at all., or (ii) It was hard to explain exactly what the restriction was. ? Actually I intended (i), so my point 1(b) sort of overlaps with 1(a), but Brett in good faith anwered to (ii) and wrote "It's not difficult to explain; decorators can only be a dotted name w/ an optional method call and its corresponding arguments." OK, but I at first got the wrong idea. I was reading the book "Core Python programming" by Wesley J. Chun. I quote from the book (page 422 section 11.3.6): 'The syntax for decorators uses a leading "at-sign" ( @ ) followed by the decorator function name and optional arguments.' No mention, you see, of the fact that the function name could be a dotted name. Nor AFAIK does a dotted name occur in any of the examples in the book. In fact it didn't occur to me that it could be, until some of the correspondence included examples that used a dotted name. So while it is possible to explain what the syntax restriction is, it is also possible to mis-explain it (or explain it poorly). OK, this is probably more a criticism of the book than anything else, but it shows that an additional potential source of confusion has been introduced. In reply to my IMO plausible use cases, Brett wrote: "Plausible does not equal useful. You need to show that this actually comes up in normal coding for a decent amount of Python code to warrant tweaking the language over." Why? That would be correct if I were proposing adding a new language feature or changing the way an existing feature works and/or causing breakage of existing code. But all I am proposing is lifting a restriction which at least arguably shouldn't have been there in the first place. After all, in other contexts where an expression is expected, say on the RHS of an assignment statement, it is easy to write something nonsensical that nobody could produce a use case for, e.g. f() + f 'a' - 'b' 0/0 but nobody is suggesting that such constructs should be SYNTACTICALLY illegal, and quite right too. Look at it this way: Suppose the restriction on decorator syntax had never been imposed. Everyone who uses decorators would be using them in exactly the same way. Would there be much of a case for then adding a restriction? Would we be having this discussion at all? I doubt it. So: please speak, if you agree with me, whether for the reasons I give or for different ones, or if you disagree with me - please give reasons too. Thanks and best wishes Rob Cliffe
2009/10/2 Rob Cliffe <rob.cliffe@btinternet.com>:
So: please speak, if you agree with me, whether for the reasons I give or for different ones, or if you disagree with me - please give reasons too.
I believe that the key reason for there being restrictions is to avoid Python code looking like "line noise". That's not precise - the specific rules as given express a sensible (in my view) balance between this goal and simplicity of implementation/explanation/usability. For example, without restrictions the following would be valid (yes, it's silly, yes it's deliberately a worst case) @(a['b'].fns[1])(1,2,{3,4}) def something(): I contend that's clearly "line noise". Python has a long tradition of not just frowning on code like this, but actively forbidding such code and not implementing constructs that contribute to such. Witness the fact that C's ternary operator (?:) and the pre and post increment operators (++ and --) are not valid Python. So forbidding such code is clearly consistent with Python's traditions. @classmethod def f(...): is equally obviously *not* line noise, and it's entirely reasonable to allow it. So pick a point on the spectrum between the first and second example. That's what the current rules do - you may not agree with the precise point picked, but my contention is that not picking a point at all is contrary to Python's established design. I'm not sure there is a corresponding design principle within Python that all constructs should be fully general (which is basically the principle you're appealing to). I can't immediately find counterexamples, though, so there may be a point there. But certainly, I'd say that such a principle (if it exists) isn't as well-established as the "no line noise" one (witness comments like "if you want Perl, you know where to get it"...)
The above covers my feelings on 1(a) and 1(b). With regard to (2), circumventing the restriction isn't really the point - the whole decorator construct is a shorthand. @deco def fn(): ... vs def fn(): ... fn = deco(fn) Giving a complex expression a name increases readability, and the assignment can be kept near the decorator if it's a one-off, as follows meaningful_name = (a['b'].fns[1])(1,2,{3,4}) @meaningful_name def f(): ... I'd go so far as to say that was improving readability, rather than "circumvention". As for (3), I agree that plausible use cases exist. They *may* warrant allowing certain extra constructs in ("moving the point" as I said above) but I'm not sure about that, and to my mind they certainly don't warrant completely removing the restriction. As a final point, I should say that personally, I have never written any code which is impacted by the current restrictions on decorator syntax. So, I don't personally have any need for the relaxation you're proposing. I'm not sure if that's a point in your favour (as my view is of limited value since I don't need the feature you're proposing) or against (as I'm an example of why the proposal isn't as generally useful as you claim) but I mention it anyway, in the interests of complete disclosure :-) I hope this helps. Paul.
Paul Moore wrote:
+1 to everything Paul said (including the parts I cut) +1 to a patch that relaxes the restriction on decorations to allow subscript operations in addition to call operations. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
[multi-reply] On Fri, Oct 2, 2009 at 03:43, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
=) Glad I was an exception. [snip]
Very true, but in this case I agree with Guido's gut. If I didn't I wouldn't have brought up support for it. On Fri, Oct 2, 2009 at 08:39, Nick Coghlan <ncoghlan@gmail.com> wrote:
+1 from me as well.
+1 to a patch that relaxes the restriction on decorations to allow subscript operations in addition to call operations.
+0 from me. Like Paul, after all of the decorator usage I have done since its introduction I have never even come up against the restriction (hell, I didn't even realize there was one until this thread). But expanding to index syntax I could live with. -Brett
fwiw C's ternary operator is implemented in Python like this: x = a if b else c Gotta say I miss pre- and post-fix operators, though. I'd rather write: x = a[b++] than x = a[b] b += 1 since I only have to reference "b" once with a post-fix operator. On Fri, Oct 2, 2009 at 8:18 AM, Paul Moore <p.f.moore@gmail.com> wrote:
-- Gerald Britton
On Fri, 2 Oct 2009 13:18:03 +0100 Paul Moore <p.f.moore@gmail.com> wrote:
This principle is know in Unix circles as "POLA" - the "Principle Of Least Astonishment". I think it's an important part of what makes Python Python - when you do something that seems obvious, it very seldom astonishes you. I'm not sure why Tim didn't include it in the Zen; my gut says that it's because this is an even more fundamental part of the landscape than most of the things on the list, so he overlooked it, but I'd rather he speaks for himself. So I propose amending the Zen of Python (as from "import this") to include it, as so: Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. The obvious way should not produce astonishing results. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
[Mike Meyer]
This principle is know in Unix circles as "POLA" - the "Principle Of Least Astonishment".
In language design circles, there should be a POGA -- Principle Of Guaranteed Astonishment. No matter what design choices you make, someone, somewhere is going to be astonished. Obviousness is in the eye of the beholder. Raymond
On 2 Oct 2009, at 19:50, Mike Meyer wrote:
Seems redundant to me. If we're to consider something a "way to do it" (let alone the Obvious Way), I'd presume at the very minimum that it does what you're trying to do. Otherwise, it's just a non-obvious way to do something else.
2009/10/3 Greg Ewing <greg.ewing@canterbury.ac.nz>:
Absolutely. Unless you give f a meaningful name... I'm not trying to say that you can't write unreadable line noise in Python (look at any of my code for examples :-)) just that the design principle is to avoid actively encouraging it. There's no doubt that this is a grey area. And I'm not against allowing subscripts in decorators (I've no personal need for it but I see how it might help, so +0) so it may be that I already support allowing the monstrosity above :-) (I refuse to go and check if it violates any other syntax rules, as my point is basically that if I need to check, I don't think it should be used, whether or not it's allowed). Paul
Well, if the syntax was loosened to allow a subscript, i.e. @a.b.c[x]( ...args... ) that would be something. And allowing a dictionary ley as well - with the same syntax - would be even better; it seems hard to justify allowing one without the other. On the other hand, maybe in future someone will come up with another 'must-have' loosening of the syntax. Eventually, don't you get to the point where it's simpler to allow anything? Best wishes Rob Cliffe ----- Original Message ----- From: "Greg Ewing" <greg.ewing@canterbury.ac.nz> To: <python-ideas@python.org> Sent: Saturday, October 03, 2009 2:40 AM Subject: Re: [Python-ideas] Decorator syntax restriction
On Fri, Oct 2, 2009 at 9:40 PM, Greg Ewing <greg.ewing@canterbury.ac.nz>wrote:
I'm +0 on the whole thing, but this to me is the strongest argument advanced thus far. Seems silly to make people run otherwise reasonable, readable code through the manglifier in order to satisfy a rule that doesn't work anyway. Geremy Condra
2009/10/8 Zac Burns:
Use case:
@noDecorator if not debug else debug
One might also write: @debug if debug else lambda f: f def f.... Is that too confusing though? My instinct says it's fine, but maybe others disagree. Still, I think the restriction should be taken off. We're all adults, etc. — Carl
geremy condra writes:
On Sat, Oct 10, 2009 at 2:12 AM, Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote:
Does either of the suggestions for debug actually "work", stylistically? Specifically, consider the preparatory code required. This looks awful to me: def debug (f): return decorate_for_debug(f) debug = False # gag, retch, puke @debug if debug else lambda f: f def foo(): pass The comment may be *little* extreme, but surely that's not very beautiful in your eyes? Isn't debug = False def instrument (f): return decorate_for_debug(f) if debug else f @instrument def foo(): pass preferable? Caveat: I'm of the school that lambda isn't very beautiful itself, not in Python.
Stephen J. Turnbull:
Does either of the suggestions for debug actually "work", stylistically? Specifically, consider the preparatory code required.
I had some questions about that myself, but I don't think they're that deep. One could easily write something like: from debugger import debug_dec debug_flag = True #Or False @debug_dec if debug_flag else lambda f: f def my_func(arg1): … I was only using "debug" as both a flag and a function because it was used that way in the previously given example. I see a certain elegance in using "debug" as both a flag and a function, but there's also something a little unsettling about it that others might not like. Now, a legitimate follow up question might be, why not do this, which is legal today: from debugger import debug_dec debug_flag = True #Or False if not debug_flag: debug_dec = lambda f: f @debug_dec def my_func(arg1): … I think the counter-objection here will be, "Why should I have @debug_dec (or in Stephen's example @instrument) in front of my function if it's only sometimes a debug decorator and other times an identity function?" And I think that's a good counter-objection. The reason that we have decorators at the top of functions instead of the bottom is to improve readability. But when @debug_dec only means "sometimes this will apply a debug decorator but not necessarily" that seems very misleading to me. Why not keep the "if" at the top so it doesn't get passed over because the next programmer only read the function declaration but not the actual definition of debug_dec/instrument? At this point though, instead of debating subjective stylistic questions I really do think that something like the consenting adults clause should come into play. Why enforce debatable style decisions through arbitrary grammar restrictions? — Carl
Carl Johnson writes:
Well, that's not the point. The grammar restriction is no longer arbitrary, it's historical. The question is there enough here to warrant relaxing the restrictions, and by how much. Since decorators can accept arguments, I don't really see a need to relax it at all. TOOWTDI, and it may as well stay that way. But that's just a -0, since I don't understand the use cases for extensions that have been presented.
2009/10/2 Rob Cliffe <rob.cliffe@btinternet.com>:
So: please speak, if you agree with me, whether for the reasons I give or for different ones, or if you disagree with me - please give reasons too.
I believe that the key reason for there being restrictions is to avoid Python code looking like "line noise". That's not precise - the specific rules as given express a sensible (in my view) balance between this goal and simplicity of implementation/explanation/usability. For example, without restrictions the following would be valid (yes, it's silly, yes it's deliberately a worst case) @(a['b'].fns[1])(1,2,{3,4}) def something(): I contend that's clearly "line noise". Python has a long tradition of not just frowning on code like this, but actively forbidding such code and not implementing constructs that contribute to such. Witness the fact that C's ternary operator (?:) and the pre and post increment operators (++ and --) are not valid Python. So forbidding such code is clearly consistent with Python's traditions. @classmethod def f(...): is equally obviously *not* line noise, and it's entirely reasonable to allow it. So pick a point on the spectrum between the first and second example. That's what the current rules do - you may not agree with the precise point picked, but my contention is that not picking a point at all is contrary to Python's established design. I'm not sure there is a corresponding design principle within Python that all constructs should be fully general (which is basically the principle you're appealing to). I can't immediately find counterexamples, though, so there may be a point there. But certainly, I'd say that such a principle (if it exists) isn't as well-established as the "no line noise" one (witness comments like "if you want Perl, you know where to get it"...)
The above covers my feelings on 1(a) and 1(b). With regard to (2), circumventing the restriction isn't really the point - the whole decorator construct is a shorthand. @deco def fn(): ... vs def fn(): ... fn = deco(fn) Giving a complex expression a name increases readability, and the assignment can be kept near the decorator if it's a one-off, as follows meaningful_name = (a['b'].fns[1])(1,2,{3,4}) @meaningful_name def f(): ... I'd go so far as to say that was improving readability, rather than "circumvention". As for (3), I agree that plausible use cases exist. They *may* warrant allowing certain extra constructs in ("moving the point" as I said above) but I'm not sure about that, and to my mind they certainly don't warrant completely removing the restriction. As a final point, I should say that personally, I have never written any code which is impacted by the current restrictions on decorator syntax. So, I don't personally have any need for the relaxation you're proposing. I'm not sure if that's a point in your favour (as my view is of limited value since I don't need the feature you're proposing) or against (as I'm an example of why the proposal isn't as generally useful as you claim) but I mention it anyway, in the interests of complete disclosure :-) I hope this helps. Paul.
Paul Moore wrote:
+1 to everything Paul said (including the parts I cut) +1 to a patch that relaxes the restriction on decorations to allow subscript operations in addition to call operations. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
[multi-reply] On Fri, Oct 2, 2009 at 03:43, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
=) Glad I was an exception. [snip]
Very true, but in this case I agree with Guido's gut. If I didn't I wouldn't have brought up support for it. On Fri, Oct 2, 2009 at 08:39, Nick Coghlan <ncoghlan@gmail.com> wrote:
+1 from me as well.
+1 to a patch that relaxes the restriction on decorations to allow subscript operations in addition to call operations.
+0 from me. Like Paul, after all of the decorator usage I have done since its introduction I have never even come up against the restriction (hell, I didn't even realize there was one until this thread). But expanding to index syntax I could live with. -Brett
fwiw C's ternary operator is implemented in Python like this: x = a if b else c Gotta say I miss pre- and post-fix operators, though. I'd rather write: x = a[b++] than x = a[b] b += 1 since I only have to reference "b" once with a post-fix operator. On Fri, Oct 2, 2009 at 8:18 AM, Paul Moore <p.f.moore@gmail.com> wrote:
-- Gerald Britton
On Fri, 2 Oct 2009 13:18:03 +0100 Paul Moore <p.f.moore@gmail.com> wrote:
This principle is know in Unix circles as "POLA" - the "Principle Of Least Astonishment". I think it's an important part of what makes Python Python - when you do something that seems obvious, it very seldom astonishes you. I'm not sure why Tim didn't include it in the Zen; my gut says that it's because this is an even more fundamental part of the landscape than most of the things on the list, so he overlooked it, but I'd rather he speaks for himself. So I propose amending the Zen of Python (as from "import this") to include it, as so: Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. The obvious way should not produce astonishing results. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
[Mike Meyer]
This principle is know in Unix circles as "POLA" - the "Principle Of Least Astonishment".
In language design circles, there should be a POGA -- Principle Of Guaranteed Astonishment. No matter what design choices you make, someone, somewhere is going to be astonished. Obviousness is in the eye of the beholder. Raymond
On 2 Oct 2009, at 19:50, Mike Meyer wrote:
Seems redundant to me. If we're to consider something a "way to do it" (let alone the Obvious Way), I'd presume at the very minimum that it does what you're trying to do. Otherwise, it's just a non-obvious way to do something else.
2009/10/3 Greg Ewing <greg.ewing@canterbury.ac.nz>:
Absolutely. Unless you give f a meaningful name... I'm not trying to say that you can't write unreadable line noise in Python (look at any of my code for examples :-)) just that the design principle is to avoid actively encouraging it. There's no doubt that this is a grey area. And I'm not against allowing subscripts in decorators (I've no personal need for it but I see how it might help, so +0) so it may be that I already support allowing the monstrosity above :-) (I refuse to go and check if it violates any other syntax rules, as my point is basically that if I need to check, I don't think it should be used, whether or not it's allowed). Paul
Well, if the syntax was loosened to allow a subscript, i.e. @a.b.c[x]( ...args... ) that would be something. And allowing a dictionary ley as well - with the same syntax - would be even better; it seems hard to justify allowing one without the other. On the other hand, maybe in future someone will come up with another 'must-have' loosening of the syntax. Eventually, don't you get to the point where it's simpler to allow anything? Best wishes Rob Cliffe ----- Original Message ----- From: "Greg Ewing" <greg.ewing@canterbury.ac.nz> To: <python-ideas@python.org> Sent: Saturday, October 03, 2009 2:40 AM Subject: Re: [Python-ideas] Decorator syntax restriction
On Fri, Oct 2, 2009 at 9:40 PM, Greg Ewing <greg.ewing@canterbury.ac.nz>wrote:
I'm +0 on the whole thing, but this to me is the strongest argument advanced thus far. Seems silly to make people run otherwise reasonable, readable code through the manglifier in order to satisfy a rule that doesn't work anyway. Geremy Condra
2009/10/8 Zac Burns:
Use case:
@noDecorator if not debug else debug
One might also write: @debug if debug else lambda f: f def f.... Is that too confusing though? My instinct says it's fine, but maybe others disagree. Still, I think the restriction should be taken off. We're all adults, etc. — Carl
geremy condra writes:
On Sat, Oct 10, 2009 at 2:12 AM, Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote:
Does either of the suggestions for debug actually "work", stylistically? Specifically, consider the preparatory code required. This looks awful to me: def debug (f): return decorate_for_debug(f) debug = False # gag, retch, puke @debug if debug else lambda f: f def foo(): pass The comment may be *little* extreme, but surely that's not very beautiful in your eyes? Isn't debug = False def instrument (f): return decorate_for_debug(f) if debug else f @instrument def foo(): pass preferable? Caveat: I'm of the school that lambda isn't very beautiful itself, not in Python.
Stephen J. Turnbull:
Does either of the suggestions for debug actually "work", stylistically? Specifically, consider the preparatory code required.
I had some questions about that myself, but I don't think they're that deep. One could easily write something like: from debugger import debug_dec debug_flag = True #Or False @debug_dec if debug_flag else lambda f: f def my_func(arg1): … I was only using "debug" as both a flag and a function because it was used that way in the previously given example. I see a certain elegance in using "debug" as both a flag and a function, but there's also something a little unsettling about it that others might not like. Now, a legitimate follow up question might be, why not do this, which is legal today: from debugger import debug_dec debug_flag = True #Or False if not debug_flag: debug_dec = lambda f: f @debug_dec def my_func(arg1): … I think the counter-objection here will be, "Why should I have @debug_dec (or in Stephen's example @instrument) in front of my function if it's only sometimes a debug decorator and other times an identity function?" And I think that's a good counter-objection. The reason that we have decorators at the top of functions instead of the bottom is to improve readability. But when @debug_dec only means "sometimes this will apply a debug decorator but not necessarily" that seems very misleading to me. Why not keep the "if" at the top so it doesn't get passed over because the next programmer only read the function declaration but not the actual definition of debug_dec/instrument? At this point though, instead of debating subjective stylistic questions I really do think that something like the consenting adults clause should come into play. Why enforce debatable style decisions through arbitrary grammar restrictions? — Carl
Carl Johnson writes:
Well, that's not the point. The grammar restriction is no longer arbitrary, it's historical. The question is there enough here to warrant relaxing the restrictions, and by how much. Since decorators can accept arguments, I don't really see a need to relax it at all. TOOWTDI, and it may as well stay that way. But that's just a -0, since I don't understand the use cases for extensions that have been presented.
participants (14)
-
Adam Atlas -
Brett Cannon -
Carl Johnson -
Gerald Britton -
geremy condra -
Greg Ewing -
Mike Meyer -
Nick Coghlan -
Paul Moore -
Raymond Hettinger -
Rob Cliffe -
Stephen J. Turnbull -
Yuvgoog Greenle -
Zac Burns