SyntaxWarning for for/while/else without break or return?
Hi, a suggestion/question related to the discussion about renaming for/else and while/else: Is it easy to generate a SyntaxWarning if you use these constructs without a break or return statement in the loop? AFAICS, any such usage would be either wrong or unnecessary. (right?) This would both help prevent wrong usage because of a false intuition and, if adequately and prominently documented, would help those interested in learning about for/else while/else. Easily doable? Maybe even make it a SyntaxError? I guess it would involve using a flag while parsing loops, but that's why I am asking, since I don't have experience with Python's parser... cheers, stefan
On Thu, Oct 8, 2009 at 10:57 AM, Stefan Rank <list-ener@strank.info> wrote:
Your question is good and valid except that "return" has nothing to do with it. In order to understand the "else" clause it's better that you think of it as an "if not break" after the loop. Any way you look at it "return" behaves as expected - skips the "else" AND skips all the code that follows (much like "raise" would have). "break" is the only way to skip the "else" clause whilst still executing the code that follows the for/while loop. So the only way to the "else" clause is useful is when a "break" can skip it. If there is no "break", you might as well remove the "else" and deindent. --yuv
on 2009-10-08 11:24 Yuvgoog Greenle said the following:
You're right. In the return-but-no-break-example I had in mind:: def findit(bag): for elem in bag: if isgreat(elem): return elem else: return default the use of else, though correct and arguably readable, is still unnecessary. --strank
On 8 Oct 2009, at 11:59 , Stefan Rank wrote:
So is it in the following cases: def find_it(bag): for elem in bag: if is_great(elem): raise Whatever(elem) else: return default or def find_it(bag): flag = False for elem in bag: if is_great(elem): flag = True else: if flag: blow_up() the point of the SyntaxWarning proposal, as far as I read it, was to point out that the `else:` clause is only ever useful in the `for:break else:` case. So there's no reason to *not* generate a SyntaxWarning if there is a `return` in the loop if you generate one when there's a `raise`.
On Thu, 8 Oct 2009 08:59:21 pm Stefan Rank wrote:
We should raise UnnecessaryNotActuallyAnError then. There is lots of code which is correct and readable while still unnecessary. Should we ban these too? mylist.extend([]) flag = True and flag if (flag is True) is True: ... while False: ... if True: ... elif False: ... [item for item in seq if True] alist.sort(key=lambda x: x) and so forth. Just think, with a bit of imagination we can slow the compiler down by a factor of two, three or even ten, by prohibiting all sorts of perfectly legal code! -- Steven D'Aprano
Yuvgoog Greenle wrote:
Yup. As to the original question in this thread - definitely possible, and shouldn't be particularly difficult. The AST and the symbol table generation don't really keep track of loop nesting (as they don't care all that much), so you would defer picking this situation up to the actual compilation stage (Python/compile.c in the source tree). Conveniently, the compiler already searches for a LOOP fblock in order to generate the syntax error for using break outside a loop, so adding a "fb_break" field to the fblock struct would be a simple way to keep track of whether or not a loop contains a break statement (although I suspect you would have to reverse the current fblock search order to find the innermost loop instead of the outermost one). Then in compiler_for and compiler_while, the fb_break field on the relevant LOOP fblock could be checked whenever an OrElse clause was present in the AST. A SyntaxWarning would then be emitted whenever the else clause was present but the break flag was not set on the relevant LOOP fblock. That's just an implementation sketch obviously, and not actually tested in any way shape or form. Still, it should be fairly straightforward to add the warning. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
I'm still pretty much against the whole idea of changing this syntax or issuing warnings. I like writing: # for loop using else clause for x in y: do_something_with(x) if something_happened_with(x): break else: # only get here if nothing happened nothing_happened() #end of for loop since it visually ties the "nothing happened" to the "for" There's a logical context that one can see. An equivalent without an else clause: #for loop without else clause something_happened = False for x in y: do_something_with(x) if something_happened_with(x): something_happened = True break # end of for loop. We get here whether something happened or not if not something_happened: nothing_happened() #end of logical context lacks such visual clues (and is more verbose and introduces another variable) and has the potential for future problems, since another programmer may add something else between the end of the loop and the conditional call to nothing_happened. OTOH if said programmer put it in the else clause, he or she would hopefully think twice if it belonged there (which it might not). If the syntax were changed (snowball's chance in hades, I bet!) it would break existing programs; if a warning were issued instead, working programs would suddenly spout spurious messages. -1 On Thu, Oct 8, 2009 at 6:19 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- Gerald Britton
On 8 Oct 2009, at 15:08 , Gerald Britton wrote:
[snip]
if a warning were issued instead, working programs would suddenly spout spurious messages.
None of the proposals (as far as I can see) would emit a warning on this snippet: there is a `break` going with the `else:` clause, which is (as far as I can say from the discussion) the normal use case for `for: else:`. The SyntaxWarning proposed would only be emitted on a `for: else:` *without* a break as it's entirely equivalent to just deleting the `else:` clause and dedenting the code it contains. As far as outputting "spurious" messages in working programs, that's the point of a warning isn't it? Tell the programmer that something works, but might not do what one would like (or does nothing at all).
re warnings: The compiler is simply not smart enough to know if a program is doing what a programmer wants. If a given program is working quietly according to its specifications, we should do nothing to disturb the peace. re syntax: There are at least three ways to exit a for-loop early: break, return and raise (explicit or implicit). Would this code generate a warning? (I hope not) for x in y: if found_what_I_want(x): return True else: return False or this? for a in b: if bogus(a): raise Bogus, "Found a bad one" else: nothing_bogus_found() On Thu, Oct 8, 2009 at 9:15 AM, Masklinn <masklinn@masklinn.net> wrote:
-- Gerald Britton
On 8 Oct 2009, at 15:44 , Gerald Britton wrote: programmer to spot potential bugs and/or simplify their code. And having a `for: else:` statement without a `break` is one such case which, according to Nick Coghlan, can be recognized. So why not warn about it?
If the program is running quietly on its current runtime, it'll keep doing so as long as it isn't switched to a runtime implementing that warning won't it?
Both would, because in both cases the `else:` clause serves no purpose whatsoever. It's noise.
Masklinn writes:
Both would, because in both cases the `else:` clause serves no purpose whatsoever. It's noise.
Tut, tut. In Nick's proposal, both would raise because there is an "else:" but there is no break. I doubt the Python compiler would add the editorial comment about "noise," though.<wink>
On 8 Oct 2009, at 18:19 , Stephen J. Turnbull wrote: the `for:` clause and removing the `else:` one. Therefore the "else:" part of the `else:` clause is noise in the system, it's not signal and it's not absent.
In all my use cases, the else serves the purpose of keeping the code in a logical group. For me at least, it is a structural aid. I vote strongly against any change that would flag for/while loops with else clauses but no breaks, especially since there are at least two other ways for early exit. On Thu, Oct 8, 2009 at 12:19 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
-- Gerald Britton
On 8 Oct 2009, at 09:32, Gerald Britton wrote:
I am also against this. It's a PyLint problem, if anything. (Just for completeness, I suppose 'yield' is a third way; but like return/raise, it makes the else unnecessary) Jared
On Fri, 9 Oct 2009 06:09:19 am Jared Grubb wrote:
(Just for completeness, I suppose 'yield' is a third way; but like return/raise, it makes the else unnecessary)
That's incorrect. Code after a yield will (in general) be executed, when you call the iterator again. E.g.: def gen(y): for x in [1, 2, 3]: yield x if y == 0: break else: yield 0 yield -1 gen(1) will yield 1, 2, 3, 0, -1. -- Steven D'Aprano
On Fri, 9 Oct 2009 03:19:35 am Stephen J. Turnbull wrote:
But that's exactly what it is doing, even if it doesn't use the word "noise". It's making an editorial judgement that perfectly legal code that does exactly what it is meant to do is somehow "bad" and requires a warning. If I see a message SyntaxWarning('for...else with no break'), my immediate response is "Yes, so what?". The code runs, it runs correctly. People write code incrementally. A for...else block that ends up including a break may go through many edit-compile-run cycles before the break actually gets inserted. I shouldn't have to add a spurious "if False: break" to shut the compiler up during those incremental edits. -- Steven D'Aprano
On Thu, Oct 8, 2009 at 10:53 PM, Steven D'Aprano <steve@pearwood.info>wrote:
The fact that code is currently legal (or illegal) does not stop us from making judgments about whether it should or should not be legal- see the current discussion about relaxing the restriction on decorators if you want an example. The only difference here is that instead of attempting to outright forbid that behavior, the proposal is to issue a warning notifying the programmer that they have probably screwed up, like pretty much everybody providing code examples in the course of this discussion. If I see a message SyntaxWarning('for...else with no break'), my
immediate response is "Yes, so what?". The code runs, it runs correctly.
You're switching definitions of "what it is meant to do" from one based on the idea that the compiler is always right to one based on the idea that there's some higher definition of correctness.
So what? Suppress the warning if you want, otherwise, listen to it, it's telling you that your code probably isn't doing what you intended it to do. That's the sort of thing I'd generally like to know while in the middle of the dev cycle. Geremy Condra
On Fri, 9 Oct 2009 03:00:57 pm geremy condra wrote:
Of course. *We* do. The compiler shouldn't. It should shut up and compile, not comment on my coding style. That's what PyLint is for. [...]
I reject the position that the compiler should be guessing what I intended. The compiler should do what I say, exactly as I say it, and not editorialise with spurious warnings about things which aren't errors. Again, that's what PyLint is for. -- Steven D'Aprano
On Fri, Oct 9, 2009 at 6:09 AM, Steven D'Aprano <steve@pearwood.info> wrote:
So the compiler shouldn't guess you were wrong when you wrote a "break" not in a loop? Please do remember we prefer no silly limitations as much as we are against silly mistakes. And btw, concerning your "__debug__" example: that code won't raise a warning because the "__debug__: break" will pass the parsing stage and be optimized out in the code generation stage, so don't worry, it's taken care of. --yuv
Steven D'Aprano writes:
Tut, tut. You are confused. It is true that this perfectly legal code does exactly what *Guido* meant it to do. That is not a problem, as you correctly and oh so tediously insist. The problem is that it does *not* do what any number of Pythonistas in the street, as well as several posters to this very thread, meant it to do. That is a real problem, even if you do not suffer from it personally.
I shouldn't have to add a spurious "if False: break" to shut the compiler up during those incremental edits.
People who hire otherwise competent Python programmers who think they know what for...else does -- but are wrong -- shouldn't have to suffer buggy programs either. That's what warnings are for.
Masklinn wrote:
On 8 Oct 2009, at 15:44 , Gerald Britton wrote:
To you, but not to Gerald or me. To me, this is a formatting style issue that the *compiler* should keeps its hands off of. As I said at the begining of this thread, this issue is appropriate for separate and optional code/style checkers like pylint/pychecker. Suppose I have for i is s: do_a(): if c(i): break else: do_b do_c I decide *maybe* I do not want do_c in the function, so I comment out the last line and change 'break' to 'return'. You and the OP would force ME to also comment out or delete the 'else:' and dedent 'do_b' for no functional reason but merely to satisy YOUR esthetic style preference. To me, that attitude is pretty obnoxious. Terry Jan Reedy
On 8 Oct 2009, at 21:44 , Terry Reedy wrote:
A warning doesn't *force* anything, it merely… warns. If you want to ignore the warning, you're free to do so. Most of the time, there's even a switch to disable (or enable, if it's not enabled by default) the aforementioned warning. The point of a warning is very much that the code works and is legal, but *in most case* is not what people would want. If it's what you want, you're free to go ahead and keep the code as is, ignoring the warning (or even filtering it out).
Based upon Guido's post in this thread, I think the chances of getting such a warning are between slim and none. And, I'm very happy about that. In fact, I would be extremely unhappy if a program that has worked for 20 years (Guido's number, not mine) would suddenly start to spurt warnings just because I upgraded Python. Bad idea all round. OTOH, if you really would like such a warning, put it in pylint. Seems to me to be by far the best option. On Thu, Oct 8, 2009 at 4:31 PM, Masklinn <masklinn@masklinn.net> wrote:
-- Gerald Britton
Gerald, if this warning would have been generated you might have saved yourself the mistakes on the previous thread. Gerald wrote as an example of "for..else" being useful without a break:
If "# suite" has no break, "i" will always be 0. Either way after the loop "i" isn't going to have the number of things processed (unless "something" was empty). After being corrected Gerald wrote the following remedied example:
The last block "# we did something" is unreachable if "#do something" has no break. This isn't meant to go against Gerald here btw. I'm simply saying that if even the best Pythonistas can cause havoc with "for...else" then think about the noobs. Personally I would suggest not using the "for...break...else" idiom at all but the least we can do is give a warning when "for..else" is so obviously misused. You could allow "break" outside of loops and simply ignore it, but you don't allow it because it helps avoid bugs. Really, a warning is the LEAST python can do here. --yuv
Ach, I just whipped it up in a hurry. I didn't pull it from live code (or even test it). A better (working) version:
And before someone says _again_ that the else is not necessary here, that's not the reason I use it. I use it to structure the code so that it is obvious that the stuff in the else clause belongs to the for loop, not to code that follows it. The point is, there are valid reasons to use this construct. That's just one of many, many more. I would fight tooth-and-nail against a warning message, if I thought there was even the faintest chance of it being considered. As it is, there is essentially no chance that it will be considered for the compiler (pylint is another matter, I think that it _should_ be there). On Thu, Oct 8, 2009 at 6:52 PM, Yuvgoog Greenle <ubershmekel@gmail.com> wrote:
-- Gerald Britton
On Fri, Oct 9, 2009 at 1:35 AM, Gerald Britton <gerald.britton@gmail.com> wrote:
So you say using "else" to structure the code is a use case, to me it's just an obfuscation, we can agree to disagree on this. I would really like to hear your other "many" reasons for using the "for..else" construct where there are no "break" statements. --yuv
Gerald Britton schrieb:
Based upon Guido's post in this thread, I think the chances of getting such a warning are between slim and none.
I don't read the post that way. Changing for/else's semantics, or removing the else clause entirely won't happen, yes. But
Sorry, but that happens often enough, e.g. when new keywords are added.
Bad idea all round.
That's a very presumptuous statement. There is quite a heavy argument in favor, namely prevention of bugs for people who misunderstand how for/else works, see your initial examples. These bugs can be nasty, because they are triggered by a corner case. An example from a different language is mixing of && and || without parens, which GCC warns about. The syntax is valid C, but I assume the warning exists because the bugs it prevents weigh more than the irritation it causes in some developers. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On 9 Oct 2009, at 23:07 , Georg Brandl wrote:
Indeed, there are numerous examples of such warnings in compilers: warning about assignments in conditional contexts (which Python forbids entirely, the ultimate hand-holding), warning about mismatches between the format specifiers and the arguments of a `printf` or `scanf`, warning about two-digit year formats, warning about incomplete enumerations, warning about storing values but never reading them, warning about allocated objects which are might not get freed, warning about potential null-pointer dereferencings, warning about overlapping patterns, warning about incomplete patterns, warning about mixing tabs and spaces… Python even has the last one already. In all of those cases, the code is legal, and it might even have been written that way on purpose.
Well I guess you can read the post how you like. Based on the way Python works today, I'm sure you won't get a warning for people failing to read the documentation. In fact, Python issues few warnings at all, ever where many other compilers do. Without a significant change in philosophy, it never will. OTOH if Python did start warnings, I'd most like to see warnings for things like: 1. variables assigned but never read 2. dead code (like if False: do_something()) 3. meaningless code (like x = True or False or x == True or False) 4. try/except without any exceptions to catch 5. unused imports Still, I think the best place for these is pylint.
On Fri, Oct 9, 2009 at 7:04 PM, Gerald Britton <gerald.britton@gmail.com>wrote:
<snip> You'd do well to consider the fact that you CC'd this comment to someone with more than 2500 commits to core, and who happens to have authored our doc system to boot. He may be qualified to talk about what is or is not pythonic. Geremy Condra
Gerald Britton writes:
The point is giving warnings for constructs that are sufficiently confusing, even having read the documentation, or typoable (in Emacs, the difference between for thing in iterable: if thing.is_good(): break else: thing = default_thing() and for thing in iterable: if thing.is_good(): break else: thing = default_thing() is one DEL when typing the code). Whether it's worth doing it for this construct is marginal to me, but I find those who advocate it to be giving plausible arguments. And you really ought to stop posting and let Steven d'Aprano carry the flag.<wink> Given *your* record of repeatedly explaining the semantics incorrectly when you have clearly read the documentation, for-else qualifies on the criterion above. The others on your list don't, especially since most of them have nearly identical semantics to the preferred form, and at least one (#4) is not currently possible to implement.
Masklinn <maskl...@masklinn.net> wrote:
You could make the same case for issuing a warning every time someone uses a mutable object as a default value in a function definition. This has _certainly_ caused a lot more confusion than the loop-else construct. But recourse to naive realism isn't a valid argument here; you're in effect punishing those who have taken their time to familiarise themselves with the language to spare those who haven't from ever having to do so. The effort should be in _learning_ the language, not in continually bypassing the training wheels.
alex23 wrote:
We don't warn about mutable default arguments because in a lot of cases the compiler doesn't know if the default argument is mutable or not, and there are multiple well-established legitimate use cases that look identical to noobish mistakes. Warning about for-else usage without break is an entirely different scenario. The comparison would hold water if we were proposing to issue a warning for *any* use of for-else. We aren't - we're only proposing to issue it for cases where the lack of a break statement suggests that the programmer has misunderstood what the clause is for. Will that give the occasional false positive due to code that is in the process of being modified? Yeah, of course it will, otherwise I'd be proposing a SyntaxError. Are those occasional false positives worth the benefit of fewer buggy programs due to misunderstandings of the for/break/else construct? In my opinion, yes. For the record, there are three cases (not counting Py3k warnings) where we currently issue syntax warnings (but generate valid code anyway). 1. Using "import *" at function level (as it disables local variable optimisations):
2. Referencing a global variable before declaring it as such:
3. Assigning to a global variable before declaring it as such:
All 3 could easily be left to pylint/pychecker style tools, but were deemed worthy of being warned about by the compiler itself. Keep in mind that using a pylint/pychecker style tool at all is a comparatively advanced Python technique. For cases like this where misunderstandings are prevalent amongst both experienced and novice Pythoneers, getting the compiler itself to offer a helping hand still strikes me as a good idea. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, 10 Oct 2009 10:22:11 pm Nick Coghlan wrote:
`import *` inside a function is illegal in Python3. It's a warning in Python2.x only because of backwards compatibility. This was the primary use-case for the warning module in the first place: to warn when functionality is being deprecated and will be removed in the future. Even though `import *` at the module level is a frequent source of bugs and confusion even for experienced coders, and the usual advice is Just Don't Do It, Python does not generate a warning for that case.
2. Referencing a global variable before declaring it as such: 3. Assigning to a global variable before declaring it as such:
These are insignificant variations of the same act: using a global before declaring it. They may have two different error messages, but the underlying problem is the same.
All 3 could easily be left to pylint/pychecker style tools, but were deemed worthy of being warned about by the compiler itself.
The first could not, because it is functionality being removed. The second and third are warning about the same thing, and this is a genuine case of Python warning about a stylistic issue. -- Steven D'Aprano
On Fri, 9 Oct 2009 07:31:22 am Masklinn wrote:
It forces me to read it. It forces me to take the time to filter out in my head real warnings for real problems, and pretend warnings for non-problems, or to run Python with some switch to disable warnings -- which means I lose the ability to get real warnings I care about. Excessive warnings trains me to ignore them. "Oh, it's some stupid compiler warning, don't worry about it, it's complaining for no good reason". (Sometimes I think some coders haven't heard of the story of The Boy Who Cried Wolf.)
The point of a warning is very much that the code works and is legal, but *in most case* is not what people would want.
Can you give any examples in the Python standard library where code raises a warning on such a basis? I would be shocked if you can give any examples. -- Steven D'Aprano
Steven D'Aprano wrote:
See my other post - there are currently 3 in the 2.x branch. Two of them are pure style warnings strongly encouraging people to put their global statements at the beginning of the function, while the third is a warning issued due to the fact that using "import *" inside a function is a really bad idea (at the very least, it forces the compiler to disable all of its normal local variable optimisations, as it has no idea what local variable names are going to exist after that import statement executes). I just checked the Py3k branch, and that adds a few more: - A warning for assert statements with a non-empty tuple as the first argument (i.e. cases where the assert will never fail) - Another two extending the global statement warnings to cover nonlocal statements as well. All 6 are cases where the code is legal and the interpreter will do the right thing, but the core developers feel people should be encouraged to write their program differently. In this thread, the proposal is that we encourage people to only use the else clause on loops containing at least one break statement. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, 10 Oct 2009 10:32:55 pm Nick Coghlan wrote:
See my previous response.
That's fundamentally the same warning: don't access a global before you declare it global. Whether that access is an assignment or a lookup is trivial.
It's actually a warning that the functionality is in the process of being removed.
assert (1, 2), "error" <stdin>:1: SyntaxWarning: assertion is always true, perhaps remove
But only if the first argument is a tuple: parentheses? And ridiculously: parentheses? Why single out always-true tuples for a warning, when other always-true items are considered legitimate, even when there is no ambiguity about what the programmer meant? This sort of inconsistency makes little sense and should stand as a cautionary tale for people putting warnings in the compiler: sometimes the warning is *bad advice*:
- Another two extending the global statement warnings to cover nonlocal statements as well.
Given that Python warns about global, it makes sense to warn about nonlocal. Its all the same thing really, just the details change: whether the declaration is global or nonlocal, whether it is an assignment or a lookup, it's the same fundamental warning: don't use a non-local variable before declaring it as such. -- Steven D'Aprano
Steven D'Aprano wrote:
<from other thread>
However, this particular warning has nothing to do with Py3k, as it has been present at least since the AST branch was merged back in 2005. import * at function level is genuinely harmful because of the effect it has on the compiler on top of the effect it has on readability. It makes sense to warn about it. This was a case of generating a warning for legal syntax that the compiler could handle because the developers considered it dubious. It was just considered so dubious that the warning was upgraded to a SyntaxError for Py3k.
Actually, we don't generate a warning for this because there are significant legitimate use cases for import * at the module level that can't be handled any other way (the two that immediately come to mind are for convenience at the interactive prompt and for bringing in methods and classes from an optional acceleration module as happens in a number of standard library modules). We can hardly generate a warning for constructs that we use ourselves. A visual grouping technique that can easily be handled with vertical whitespace or comments isn't even close to being in the same league as the legitimate use cases for import * at module level.
That's why it says "perhaps remove" rather than "remove". That hint also indicates the developer error that it is actually trying to warn against: assert (longAndComplicatedAssertionCondition, "My assertion message") That's not an assert statement using parentheses purely for line continuation - it's an assertion of a 2-tuple that will never fail, and a naive user may not realise what is happening (thus getting very confused when their assert statement appears to pass, but subsequent code relying on that assertion fails anyway). You have to either use a backslash for the line continuation here, or else ensure the comma is outside the parentheses: assert longAndComplicatedAssertionCondition, \ "My assertion message" assert longAndComplicatedAssertionCondition, ( "My assertion message") In that context, the syntax warning turns what could be a very obscure and hard to debug problem into something fairly obvious (as the only parentheses present will be the ones that are causing the comma to be misinterpreted). The warning could probably be refined such that it only triggers for a 2-tuple as the sole argument to an assert statement, but how often is such a false triggering actually going to happen in real code rather than dummy examples? For the loop else clauses, the users who are going to get themselves into trouble are the ones who *think* they know what it means, but actually don't (i.e. not the sort of people that are likely to be regularly running pylint or pychecker over their code). A SyntaxWarning built directly into the compiler would hopefully give them the hint they need in order to realise that they don't actually know what is going on. ("break? What does break have to do with using else on a loop? Hmm, maybe I better look this up or ask someone about it..."). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Arnaud Delobelle writes:
Also, it would be odd to have a language feature that generates a warning whenever it is used :)
Not really. Deprecation warnings behave that way. AFAICS, a deprecation warning is just the extreme case of a syntax warning.
On Sun, 11 Oct 2009 05:36:07 am Stephen J. Turnbull wrote:
I don't see it that way at all. A deprecation warning makes sense, it's the primary use case for the warnings module. It says "Here is a feature which is changing, or being removed -- you MUST take steps to change your code, or be stuck with this version forever, because the language is changing." A syntax warning, as being suggested, is nothing like that. It says "Here is a legitimate feature, part of the language. It's not going away. You may have a good reason for doing what you want, but just in case you're ignorant or naive or stupid, I'm going to warn you that your perfectly legal code might be doing what something other than what you expect." This is, to my mind, particularly egregious because every single language feature might be doing something other than what you expect, if you are sufficiently ignorant, naive or stupid. Such warnings not only try to guess what users want, but they make an arbitrary, and inconsistent, decision as to how much ignorance they "protect" the user from. We don't, for example, protect the user from thinking that list.sort() returns the sorted list, or that arithmetic on floats is simple, or that locals() can be meaningfully modified, or that mutable defaults are recreated every time the function is called -- all errors FAR more common than misunderstanding for...else. If your aim is to protect users from their own mistakes, then there are far more serious mistakes that we just ignore. And if our attitude is "we're all adults here", then why on earth single out such an uncommon gotcha for special treatment? -- Steven D'Aprano
On Sun, 11 Oct 2009 02:30:51 am Nick Coghlan wrote:
Right. It's so dangerous that it's becoming illegal. You can't just turn language features illegal overnight, you have to go through a period of deprecation. This makes perfect sense, and is the primary use-case for warnings in Python.
Of course we can. But we shouldn't, it would be silly. The point is that there are many language features that are pointless, silly or troublesome, and we don't -- and shouldn't -- raise warnings for them. I don't see for...else to be exceptional enough to deserve a language warning. [...]
I don't believe the language should be hand-holding the naive user to that extent. What do we do, fill the compiler up with a thousand warnings for things which might confuse some naive user? What ever happened to "We're all adults here"? The solution to naive users is for the user to learn more about the language and be less naive, not for the language to hold his hand. The compiler isn't your babysitter, and Python isn't meant to be a teaching language for kiddies. The proliferation of warnings leads to complacently among naive users ("I don't need to understand this feature to use it, the compiler will warn me if I do something wrong") and, much worse, overload among non-naive users ("it's just another damn warning, ignore it, the compiler is just complaining about nothing"). In my opinion, we've already taken too many steps down this path, lets not go any further.
It's a bug -- it is output that is generated unnecessarily. Defending it on the basis that it's a rare bug is not a wise idea.
You can't protect ignorant, naive and downright stupid users against themselves. It's a never-ending problem. Perhaps we should raise a warning when the user says: for in in range(len(sequence)): SyntaxWarning: There's probably a better way to do this. or for list.insert? PerformanceWarning: This may be slow for large lists. You may think I'm exaggerating, but Python already does something similar, only worse. The built-in sum() prohibits summing strings because it may be O(N**2) -- even though in recent versions of CPython, it may not be, even though for small numbers of small strings it makes no practical difference, and even though it allows O(N**2) summation of lists! A naive user will be lulled into a false sense of security that sum(list_of_lists, []) is fine, and an advanced user who wants to use sum() as a teaching aid to show how repeated string concatenation is slow is frustrated. -- Steven D'Aprano
Steven D'Aprano writes:
Nobody (except you and Gerald) is talking about things that "might" confuse "some naive user", nor does anybody advocate proliferating warnings or even getting within a light-year of that slippery slope. We're talking about one construct that *demonstrably* confuses *people who have taken it upon themselves to explain correct usage in this thread*, including at least one who considers himself above needing such warnings. We're talking about a single construct that the BDFL has deigned to deprecate as an unfortunate choice of keywords. Please stop wandering off topic, and address the arguments that have been presented.
On Sun, 11 Oct 2009 03:36:45 pm Stephen J. Turnbull wrote:
With all respect to those people, just because somebody takes it upon themselves to explain correct usage, doesn't mean they know the correct usage. This isn't meant as a dig at anyone here, but sadly the least competent people tend to be the most sure of themselves: http://www.ncbi.nlm.nih.gov/pubmed/10626367 http://en.wikipedia.org/wiki/Illusory_superiority To quote Charles Darwin: "ignorance more frequently begets confidence than does knowledge". I don't exclude myself in this -- we all make mistakes, and those who are on comp.lang.python may have noticed me make a clanger just yesterday. Nobody here has demonstrated that for...else is a significant problem. I'd describe it as a gotcha, and Python, like every computer language, has a number of gotchas. There's no need have the compiler defensively warn people about them.
You're kidding, right? Or is anything but acquiescence to the idea of giving a warning "off-topic"? Just because somebody proposes a change, doesn't mean that opposition to that change is off-topic. -- Steven D'Aprano
Steven D'Aprano writes:
You're kidding, right?
No.
Or is anything but acquiescence to the idea of giving a warning "off-topic"?
That's not the alternative to "kidding".
Just because somebody proposes a change, doesn't mean that opposition to that change is off-topic.
"There you go again." I have no problem with opposition, and if you go back and read my posts, that's clear. It's the bogus attribution of motives and wildly exaggerated claims that need to stop, on both sides. On your side, the "proliferation of warnings" and the "warn on anything that might trip up a naive programmer who hasn't read the docs" nonsense. Neither is in question. There's *one* warning in question, the construct is legal and unambiguous as implemented. Good luck contesting those facts. So the topic is "how useful and common is the construct, correctly used?" vs. "how much code in actual use is likely to be written under the seductive spell of the meaning of 'else' in English?" (I'm open to other suggestions, within the framework of "only one warning" and "construct is unambiguous when understood.") At the moment, my assessment is "almost useless" and "it is likely that a large majority of the code using 'breakless for-else' is written with the wrong semantics expected, and a substantial fraction of those will actually result in buggy behavior." On that basis, I'm about +0.5 on issuing a warning. I don't know if my opinion matters to anybody but me but it *is* up for grabs; my current stance is not set in concrete. I can tell you that the *style* of your last five posts or so have tended to tilt my opinion against you, while Nick and Georg (except that last ad hominem, of course) have been steadily scoring a point or two per post on content.
On Sat, Oct 10, 2009 at 9:36 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
We're talking about a single construct that the BDFL has deigned to deprecate as an unfortunate choice of keywords.
Wrong. I would not have the feature at all if I had to do it over. I would *not* choose another keyword. But I don't see the same level of danger in it that some here see. I am also against adding a syntax warning for this. It belongs in pylint etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sat, Oct 10, 2009 at 10:56 PM, Steven D'Aprano <steve@pearwood.info> wrote:
I realize that this will strike many here as heresy, but yelling "RTFM, n00b!" is not actually a productive way to resolve what is clearly a point of difficulty for a sizeable number of competent developers. Geremy Condra
Steven D'Aprano wrote:
I think it's a side effect of the way the check is implemented. It just looks at whether the first expression is a tuple constructor, and ignores the second one. While theoretically it's not ideal, it doesn't do any harm in practice, because it's so unlikely that anyone would intentionally write such an assert statement. There's no use for it whatsoever.
Hm, yes... well, at least you get an error rather than incorrect results! -- Greg
Steven D'Aprano schrieb:
This would be a problem indeed if programmers were computers. Last time I looked they were humans, and capable of not blindly following any instruction the computer gives them. [1] And even if they do, they obviously don't know how the assert statement works, and need to revisit the docs. Georg [1] Note, only speaking of programmers here ;) -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Nick Coghlan wrote:
If it is perfectly legal to put global statements anywhere in the function, and there is no intention to tighten the rule to disallow that (which would be ok with me), then I am against the warning. It suggests that one has done something wrong, when one has not. A global statement as module level has no effect, even though some newbies think it does (else why would they write it when they do?). Perhaps we should add a warning for *that*, especially given that it is more likely to indicate a real error of thinking rather and simply making the 'wrong' stylistic choice.
In this thread, the proposal is that we encourage people to only use the else clause on loops containing at least one break statement.
Whereas I want to be able to use while/else and for/else in didactic code without having to warn people against a spurious warning. Terry Jan Reedy
On Sun, Oct 11, 2009 at 12:09 AM, Terry Reedy <tjreedy@udel.edu> wrote:
Nobody is holding a gun to your head. Use it if you must- but when a construct is obviously confusing the heck out of people who are demonstrably good programmers, it's time to think seriously about warning people who are using it in ways that *do not make good sense*. All kinds of decisions in python have been made explicitly to discourage buggy, hard to read code, and the discussion here, complete with numerous mistakes and misunderstandings, constitutes abject proof that for-else is both of those things and hard to write correctly to boot. If we can't make it less dangerous, the least we can do is make it more obvious when you're screwing it up. Geremy Condra
Terry Reedy schrieb:
Why would you want to do that, except to confuse a new generation of Python programmers about its semantics? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Georg Brandl writes:
Terry Reedy schrieb:
Whereas I want to be able to use while/else and for/else in didactic code without having to warn people against a spurious warning.
Why would you want to do that, except to confuse a new generation of Python programmers about its semantics?
Sheesh. Now *you*'re doing it. He and Steven have explained their use case at length, and it's valid. Please keep to the point, which is the relative importance of the stylistic and didactic value of placing associated code inside the for-else block rather than outside it, versus the attractive nuisance that this construct is for many programmers.
Gerald Britton wrote:
If the code is written using a style that is misleading, I think it makes sense to issue a warning.
This would generate a warning, because the 'else' is unnecessary and potentially confusing here. The above code is exactly equivalent to: for x in y: if found_what_I_want(x): return True return False
Would also generate a warning. Again, the version without 'else': for a in b: if bogus(a): raise Bogus, "Found a bad one" nothing_bogus_found() In both cases, I find the version without 'else' to be easier to understand. +1 to making for/else and while/else without break generate a warning. +0 to making it an error. -1 to removing or renaming the 'else' of for and while loops. - Jacob
On Thu, Oct 8, 2009 at 9:44 AM, Gerald Britton <gerald.britton@gmail.com>wrote:
<snip> I grant you that compilers *shouldn't* be smarter than programmers, but I highly doubt that any program including an extraneous else clause is "working according to specifications", and that's the point of this proposal. Geremy Condra
geremy condra writes:
Actually, I can imagine someone developing a style where a redundant else was used to indicate "this is part of/cleanup of the for loop, and should go with it in any refactoring". Say, in a program that speaks SMTP: for line in line_factory: smtp_send(line + "\n") else: smtp_send(".\n")
Stephen J. Turnbull schrieb:
Usually, that effect can be created by newlines: for line in line_factory: smtp_send(line + "\n") smtp_send(".\n") more code goes here I'm not sure myself whether I like generating warnings for valid syntax, but in the often-misunderstood case of for/else, I think it would be reasonable. Georg
Gerald Britton writes:
That *may* be true for Python, but my face has been saved more times than I like to think by the warning on if (x = f(y)) in many C compilers. I think I would probably be saved at least once in my remaining years of Python programming by Nick's proposal, too.
On Fri, 9 Oct 2009 03:17:38 am Stephen J. Turnbull wrote:
I think I would probably be saved at least once in my remaining years of Python programming by Nick's proposal, too.
That makes no sense. If there's no difference between: for x in seq: pass else: whatever() and for x in seq: pass whatever() then what could the warning save you from? Embarrassment when people laugh at your coding style? Perhaps you're thinking of the following: for x in seq: if flag: whatever() else: # oops, I wanted this to be inside the loop something() But Nick's warning is insufficient, because it is defeated by the presence of a break *anywhere* in the loop: for x in seq: if today == tuesday: break if flag: whatever() else: # oops, I wanted this to be inside the loop something() The fact that it, occasionally, will catch a completely unrelated error (you outdented the else clause in if...else) is a fluke. -- Steven D'Aprano
Steven D'Aprano writes:
The warning can save me from the brain fart that has been repeated over and over in this thread: thinking that there *is* a difference. This thread is ample evidence that people can and do misread and miswrite that construct *even in a context where they are forewarned that they are discussing an easily misused construct*.
Perhaps you're thinking of the following:
Stop trying to read my mind. What you should be reading is some of the posts in this thread explaining that for x in seq: pass else: whatever() means if seq: for x in seq: pass else: whatever() or similar ... *after* Nick's explanation that the for/break/else idiom for search is the only semantically interesting form.
On Fri, 9 Oct 2009 12:15:10 am Masklinn wrote:
No entirely -- the compiler generates different byte-code.
Heavens no!!! What on earth gives you that idea? How can the compiler possibly know what you would like? And even if it could, it's the compiler's job to enforce the rules of the language, not to satisfy what people would like. Compiler warnings should never be spurious. Python warnings are used for meaningful warnings, such as deprecation warnings, not for stylistic ones like "by the way, you've used pass twice in a row, that's pointless". I don't want the compiler wasting my time with spurious warnings. If I want to run PyLint, I'll run PyLint. I don't want the compiler telling me I've defined a name and never used it, or declared a function that doesn't get called, or have code branches that can't possibly be taken, or anything else. I want it to compile, as fast as possible. The edit-compile-run cycle in Python is *very* lightweight, as it should be. When I want extra checking, I run PyLint, I don't want it every single time. -- Steven D'Aprano
Steven D'Aprano wrote:
I would advise against putting money on that prospect.
3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE
3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE If you leave out the newline after the loop in f2 then the final line number would change from a 5 to a 4, but the bytecode would otherwise remain identical. With the extra newline in there, the bytecode *is* identical. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Thu, 8 Oct 2009 08:24:46 pm Yuvgoog Greenle wrote:
In order to understand the "else" clause it's better that you think of it as an "if not break" after the loop.
That is completely wrong. for...else is a single block, and break jumps to the end of the block. See my previous reply for details. for x in seq: whatever else: suite is a single block. It's not the same as: for x in seq: whatever suite even if they happen to execute the same way. In the first case, the entire block is one single lump of code. In the second case, you have two independent lumps of code, a for-loop with no else, followed by an independent bunch of code following it. Lumps of code in this sense are a weaker grouping than functions, classes or modules, but they are a grouping. When you see for...else you KNOW that the else-suite belongs with the for without needing to read the suite in detail. You can't pick it up and move it somewhere else and expect it to work, they belong together, in that order. The presence of the else tells the reader that the entire for...else block is one single conceptual unit. This extra information is missing in the second case, where there is nothing suggesting that suite belongs with the for-loop: they look independent. To learn that they're not, you have to read the code in enough detail to understand it fully. -- Steven D'Aprano
Steven D'Aprano wrote:
I can see your point, but the confusion with the naive "if seq is contains at least 1 element do the loop, else do the else clause" outweighs it in my mind. That may have something to do with my personal preference for using vertical whitespace and comment lines to obtain logical groupings below the function level. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Thu, 8 Oct 2009 07:57:38 pm Stefan Rank wrote:
-1 on such a warning. This sort of check belongs in PyLint or equivalent, there is no need to complicate the compiler and make it part of the language. The compiler should be nicely lean and simple and not try to guess what the user's intention is. for...else with no break is legal code, if there's no break inside it, it is still legal code.
-1000 on making it a SyntaxError. SyntaxErrors are for things which are WRONG, not for things which are unnecessary or redundant or pointless. Should this be prohibited too? for x in seq: if 0: break else: print "whatever" What happens when the break is under a __debug__ test? The same source compiles to two different byte-codes, one of which has break, the other doesn't. Given this: code = compile("""for x in seq: if __debug__: break else: print "no break" """, '', 'exec') this is the byte-code you get running normally:
2 13 BREAK_LOOP 14 JUMP_ABSOLUTE 7 >> 17 POP_BLOCK 4 18 LOAD_CONST 0 ('no break') 21 PRINT_ITEM 22 PRINT_NEWLINE >> 23 LOAD_CONST 1 (None) 26 RETURN_VALUE and this is what you get running under python -O:
2 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 4 17 LOAD_CONST 0 ('no break') 20 PRINT_ITEM 21 PRINT_NEWLINE >> 22 LOAD_CONST 1 (None) 25 RETURN_VALUE No break in the second case. So now we have choices: * Code which runs fine normally becomes a SyntaxError when running with -O. * for...else does not require a break when running with -O, but does require it without -O. * We pile complication on top of complication and make an explicit exception for tests like if __debug__: break -- which means that we still have for loops with no break that run perfectly fine! * Stop optimizing if __debug__ tests. All four of these alternatives are unacceptable. -- Steven D'Aprano
Steven D'Aprano wrote:
No, see below.
If I understand Nick correctly, this is irrelevant. The warning/error generation would be handled by an earlier stage than the optimization that removes the break. - Jacob
Jacob Holm wrote:
I never promised that - someone else said that would happen, but it's actually implementation dependent. In the case of CPython, the compiler stage checks for recognised constants (such as 0, non-zero integers, True, False and __debug__) in if statement test expressions and doesn't even bother visiting any unreachable code. Accordingly, Steven is quite correct that having your only break statement inside "if __debug__:" would lead to the natural implementation approach for CPython issuing a SyntaxWarning when running under the optimiser. Make no mistake, the whole point of adding this warning would be to cause some currently legal code to generate a warning. It would be a policy decision, stating that "for/break/else" and "while/break/else" were the only accepted use cases for the else clause and that loop cleanup code should merely be written after the loop with either comments or vertical whitespace used for logical grouping, rather than having the else clause available for that purpose. The choice to be made is whether the potential for improvement in understanding of the nature and purpose of the else clause amongst Python programmers as a whole (even those that don't use static analysis tools like pylint or pychecker) would be worth the mild aggravation to folks like Steven and Gerald that currently use the else clause to logically group cleanup code with the relevant loop. (That said, I don't think the question should even be asked of python-dev without a patch on the tracker that actually adds the new warning. I personally have several other Python issues higher on my to-do list that I have yet to find the roundtuits to address, so I'm not likely to be producing such a patch anytime soon...) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, Oct 10, 2009 at 2:02 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'll have time to start working on a patch within 2 weeks. I've never touched the compiler but it sounds like an interesting subject. --yuv
On Thu, Oct 8, 2009 at 10:57 AM, Stefan Rank <list-ener@strank.info> wrote:
Your question is good and valid except that "return" has nothing to do with it. In order to understand the "else" clause it's better that you think of it as an "if not break" after the loop. Any way you look at it "return" behaves as expected - skips the "else" AND skips all the code that follows (much like "raise" would have). "break" is the only way to skip the "else" clause whilst still executing the code that follows the for/while loop. So the only way to the "else" clause is useful is when a "break" can skip it. If there is no "break", you might as well remove the "else" and deindent. --yuv
on 2009-10-08 11:24 Yuvgoog Greenle said the following:
You're right. In the return-but-no-break-example I had in mind:: def findit(bag): for elem in bag: if isgreat(elem): return elem else: return default the use of else, though correct and arguably readable, is still unnecessary. --strank
On 8 Oct 2009, at 11:59 , Stefan Rank wrote:
So is it in the following cases: def find_it(bag): for elem in bag: if is_great(elem): raise Whatever(elem) else: return default or def find_it(bag): flag = False for elem in bag: if is_great(elem): flag = True else: if flag: blow_up() the point of the SyntaxWarning proposal, as far as I read it, was to point out that the `else:` clause is only ever useful in the `for:break else:` case. So there's no reason to *not* generate a SyntaxWarning if there is a `return` in the loop if you generate one when there's a `raise`.
On Thu, 8 Oct 2009 08:59:21 pm Stefan Rank wrote:
We should raise UnnecessaryNotActuallyAnError then. There is lots of code which is correct and readable while still unnecessary. Should we ban these too? mylist.extend([]) flag = True and flag if (flag is True) is True: ... while False: ... if True: ... elif False: ... [item for item in seq if True] alist.sort(key=lambda x: x) and so forth. Just think, with a bit of imagination we can slow the compiler down by a factor of two, three or even ten, by prohibiting all sorts of perfectly legal code! -- Steven D'Aprano
Yuvgoog Greenle wrote:
Yup. As to the original question in this thread - definitely possible, and shouldn't be particularly difficult. The AST and the symbol table generation don't really keep track of loop nesting (as they don't care all that much), so you would defer picking this situation up to the actual compilation stage (Python/compile.c in the source tree). Conveniently, the compiler already searches for a LOOP fblock in order to generate the syntax error for using break outside a loop, so adding a "fb_break" field to the fblock struct would be a simple way to keep track of whether or not a loop contains a break statement (although I suspect you would have to reverse the current fblock search order to find the innermost loop instead of the outermost one). Then in compiler_for and compiler_while, the fb_break field on the relevant LOOP fblock could be checked whenever an OrElse clause was present in the AST. A SyntaxWarning would then be emitted whenever the else clause was present but the break flag was not set on the relevant LOOP fblock. That's just an implementation sketch obviously, and not actually tested in any way shape or form. Still, it should be fairly straightforward to add the warning. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
I'm still pretty much against the whole idea of changing this syntax or issuing warnings. I like writing: # for loop using else clause for x in y: do_something_with(x) if something_happened_with(x): break else: # only get here if nothing happened nothing_happened() #end of for loop since it visually ties the "nothing happened" to the "for" There's a logical context that one can see. An equivalent without an else clause: #for loop without else clause something_happened = False for x in y: do_something_with(x) if something_happened_with(x): something_happened = True break # end of for loop. We get here whether something happened or not if not something_happened: nothing_happened() #end of logical context lacks such visual clues (and is more verbose and introduces another variable) and has the potential for future problems, since another programmer may add something else between the end of the loop and the conditional call to nothing_happened. OTOH if said programmer put it in the else clause, he or she would hopefully think twice if it belonged there (which it might not). If the syntax were changed (snowball's chance in hades, I bet!) it would break existing programs; if a warning were issued instead, working programs would suddenly spout spurious messages. -1 On Thu, Oct 8, 2009 at 6:19 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- Gerald Britton
On 8 Oct 2009, at 15:08 , Gerald Britton wrote:
[snip]
if a warning were issued instead, working programs would suddenly spout spurious messages.
None of the proposals (as far as I can see) would emit a warning on this snippet: there is a `break` going with the `else:` clause, which is (as far as I can say from the discussion) the normal use case for `for: else:`. The SyntaxWarning proposed would only be emitted on a `for: else:` *without* a break as it's entirely equivalent to just deleting the `else:` clause and dedenting the code it contains. As far as outputting "spurious" messages in working programs, that's the point of a warning isn't it? Tell the programmer that something works, but might not do what one would like (or does nothing at all).
re warnings: The compiler is simply not smart enough to know if a program is doing what a programmer wants. If a given program is working quietly according to its specifications, we should do nothing to disturb the peace. re syntax: There are at least three ways to exit a for-loop early: break, return and raise (explicit or implicit). Would this code generate a warning? (I hope not) for x in y: if found_what_I_want(x): return True else: return False or this? for a in b: if bogus(a): raise Bogus, "Found a bad one" else: nothing_bogus_found() On Thu, Oct 8, 2009 at 9:15 AM, Masklinn <masklinn@masklinn.net> wrote:
-- Gerald Britton
On 8 Oct 2009, at 15:44 , Gerald Britton wrote: programmer to spot potential bugs and/or simplify their code. And having a `for: else:` statement without a `break` is one such case which, according to Nick Coghlan, can be recognized. So why not warn about it?
If the program is running quietly on its current runtime, it'll keep doing so as long as it isn't switched to a runtime implementing that warning won't it?
Both would, because in both cases the `else:` clause serves no purpose whatsoever. It's noise.
Masklinn writes:
Both would, because in both cases the `else:` clause serves no purpose whatsoever. It's noise.
Tut, tut. In Nick's proposal, both would raise because there is an "else:" but there is no break. I doubt the Python compiler would add the editorial comment about "noise," though.<wink>
On 8 Oct 2009, at 18:19 , Stephen J. Turnbull wrote: the `for:` clause and removing the `else:` one. Therefore the "else:" part of the `else:` clause is noise in the system, it's not signal and it's not absent.
In all my use cases, the else serves the purpose of keeping the code in a logical group. For me at least, it is a structural aid. I vote strongly against any change that would flag for/while loops with else clauses but no breaks, especially since there are at least two other ways for early exit. On Thu, Oct 8, 2009 at 12:19 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
-- Gerald Britton
On 8 Oct 2009, at 09:32, Gerald Britton wrote:
I am also against this. It's a PyLint problem, if anything. (Just for completeness, I suppose 'yield' is a third way; but like return/raise, it makes the else unnecessary) Jared
On Fri, 9 Oct 2009 06:09:19 am Jared Grubb wrote:
(Just for completeness, I suppose 'yield' is a third way; but like return/raise, it makes the else unnecessary)
That's incorrect. Code after a yield will (in general) be executed, when you call the iterator again. E.g.: def gen(y): for x in [1, 2, 3]: yield x if y == 0: break else: yield 0 yield -1 gen(1) will yield 1, 2, 3, 0, -1. -- Steven D'Aprano
On Fri, 9 Oct 2009 03:19:35 am Stephen J. Turnbull wrote:
But that's exactly what it is doing, even if it doesn't use the word "noise". It's making an editorial judgement that perfectly legal code that does exactly what it is meant to do is somehow "bad" and requires a warning. If I see a message SyntaxWarning('for...else with no break'), my immediate response is "Yes, so what?". The code runs, it runs correctly. People write code incrementally. A for...else block that ends up including a break may go through many edit-compile-run cycles before the break actually gets inserted. I shouldn't have to add a spurious "if False: break" to shut the compiler up during those incremental edits. -- Steven D'Aprano
On Thu, Oct 8, 2009 at 10:53 PM, Steven D'Aprano <steve@pearwood.info>wrote:
The fact that code is currently legal (or illegal) does not stop us from making judgments about whether it should or should not be legal- see the current discussion about relaxing the restriction on decorators if you want an example. The only difference here is that instead of attempting to outright forbid that behavior, the proposal is to issue a warning notifying the programmer that they have probably screwed up, like pretty much everybody providing code examples in the course of this discussion. If I see a message SyntaxWarning('for...else with no break'), my
immediate response is "Yes, so what?". The code runs, it runs correctly.
You're switching definitions of "what it is meant to do" from one based on the idea that the compiler is always right to one based on the idea that there's some higher definition of correctness.
So what? Suppress the warning if you want, otherwise, listen to it, it's telling you that your code probably isn't doing what you intended it to do. That's the sort of thing I'd generally like to know while in the middle of the dev cycle. Geremy Condra
On Fri, 9 Oct 2009 03:00:57 pm geremy condra wrote:
Of course. *We* do. The compiler shouldn't. It should shut up and compile, not comment on my coding style. That's what PyLint is for. [...]
I reject the position that the compiler should be guessing what I intended. The compiler should do what I say, exactly as I say it, and not editorialise with spurious warnings about things which aren't errors. Again, that's what PyLint is for. -- Steven D'Aprano
On Fri, Oct 9, 2009 at 6:09 AM, Steven D'Aprano <steve@pearwood.info> wrote:
So the compiler shouldn't guess you were wrong when you wrote a "break" not in a loop? Please do remember we prefer no silly limitations as much as we are against silly mistakes. And btw, concerning your "__debug__" example: that code won't raise a warning because the "__debug__: break" will pass the parsing stage and be optimized out in the code generation stage, so don't worry, it's taken care of. --yuv
Steven D'Aprano writes:
Tut, tut. You are confused. It is true that this perfectly legal code does exactly what *Guido* meant it to do. That is not a problem, as you correctly and oh so tediously insist. The problem is that it does *not* do what any number of Pythonistas in the street, as well as several posters to this very thread, meant it to do. That is a real problem, even if you do not suffer from it personally.
I shouldn't have to add a spurious "if False: break" to shut the compiler up during those incremental edits.
People who hire otherwise competent Python programmers who think they know what for...else does -- but are wrong -- shouldn't have to suffer buggy programs either. That's what warnings are for.
Masklinn wrote:
On 8 Oct 2009, at 15:44 , Gerald Britton wrote:
To you, but not to Gerald or me. To me, this is a formatting style issue that the *compiler* should keeps its hands off of. As I said at the begining of this thread, this issue is appropriate for separate and optional code/style checkers like pylint/pychecker. Suppose I have for i is s: do_a(): if c(i): break else: do_b do_c I decide *maybe* I do not want do_c in the function, so I comment out the last line and change 'break' to 'return'. You and the OP would force ME to also comment out or delete the 'else:' and dedent 'do_b' for no functional reason but merely to satisy YOUR esthetic style preference. To me, that attitude is pretty obnoxious. Terry Jan Reedy
On 8 Oct 2009, at 21:44 , Terry Reedy wrote:
A warning doesn't *force* anything, it merely… warns. If you want to ignore the warning, you're free to do so. Most of the time, there's even a switch to disable (or enable, if it's not enabled by default) the aforementioned warning. The point of a warning is very much that the code works and is legal, but *in most case* is not what people would want. If it's what you want, you're free to go ahead and keep the code as is, ignoring the warning (or even filtering it out).
Based upon Guido's post in this thread, I think the chances of getting such a warning are between slim and none. And, I'm very happy about that. In fact, I would be extremely unhappy if a program that has worked for 20 years (Guido's number, not mine) would suddenly start to spurt warnings just because I upgraded Python. Bad idea all round. OTOH, if you really would like such a warning, put it in pylint. Seems to me to be by far the best option. On Thu, Oct 8, 2009 at 4:31 PM, Masklinn <masklinn@masklinn.net> wrote:
-- Gerald Britton
Gerald, if this warning would have been generated you might have saved yourself the mistakes on the previous thread. Gerald wrote as an example of "for..else" being useful without a break:
If "# suite" has no break, "i" will always be 0. Either way after the loop "i" isn't going to have the number of things processed (unless "something" was empty). After being corrected Gerald wrote the following remedied example:
The last block "# we did something" is unreachable if "#do something" has no break. This isn't meant to go against Gerald here btw. I'm simply saying that if even the best Pythonistas can cause havoc with "for...else" then think about the noobs. Personally I would suggest not using the "for...break...else" idiom at all but the least we can do is give a warning when "for..else" is so obviously misused. You could allow "break" outside of loops and simply ignore it, but you don't allow it because it helps avoid bugs. Really, a warning is the LEAST python can do here. --yuv
Ach, I just whipped it up in a hurry. I didn't pull it from live code (or even test it). A better (working) version:
And before someone says _again_ that the else is not necessary here, that's not the reason I use it. I use it to structure the code so that it is obvious that the stuff in the else clause belongs to the for loop, not to code that follows it. The point is, there are valid reasons to use this construct. That's just one of many, many more. I would fight tooth-and-nail against a warning message, if I thought there was even the faintest chance of it being considered. As it is, there is essentially no chance that it will be considered for the compiler (pylint is another matter, I think that it _should_ be there). On Thu, Oct 8, 2009 at 6:52 PM, Yuvgoog Greenle <ubershmekel@gmail.com> wrote:
-- Gerald Britton
On Fri, Oct 9, 2009 at 1:35 AM, Gerald Britton <gerald.britton@gmail.com> wrote:
So you say using "else" to structure the code is a use case, to me it's just an obfuscation, we can agree to disagree on this. I would really like to hear your other "many" reasons for using the "for..else" construct where there are no "break" statements. --yuv
Gerald Britton schrieb:
Based upon Guido's post in this thread, I think the chances of getting such a warning are between slim and none.
I don't read the post that way. Changing for/else's semantics, or removing the else clause entirely won't happen, yes. But
Sorry, but that happens often enough, e.g. when new keywords are added.
Bad idea all round.
That's a very presumptuous statement. There is quite a heavy argument in favor, namely prevention of bugs for people who misunderstand how for/else works, see your initial examples. These bugs can be nasty, because they are triggered by a corner case. An example from a different language is mixing of && and || without parens, which GCC warns about. The syntax is valid C, but I assume the warning exists because the bugs it prevents weigh more than the irritation it causes in some developers. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On 9 Oct 2009, at 23:07 , Georg Brandl wrote:
Indeed, there are numerous examples of such warnings in compilers: warning about assignments in conditional contexts (which Python forbids entirely, the ultimate hand-holding), warning about mismatches between the format specifiers and the arguments of a `printf` or `scanf`, warning about two-digit year formats, warning about incomplete enumerations, warning about storing values but never reading them, warning about allocated objects which are might not get freed, warning about potential null-pointer dereferencings, warning about overlapping patterns, warning about incomplete patterns, warning about mixing tabs and spaces… Python even has the last one already. In all of those cases, the code is legal, and it might even have been written that way on purpose.
Well I guess you can read the post how you like. Based on the way Python works today, I'm sure you won't get a warning for people failing to read the documentation. In fact, Python issues few warnings at all, ever where many other compilers do. Without a significant change in philosophy, it never will. OTOH if Python did start warnings, I'd most like to see warnings for things like: 1. variables assigned but never read 2. dead code (like if False: do_something()) 3. meaningless code (like x = True or False or x == True or False) 4. try/except without any exceptions to catch 5. unused imports Still, I think the best place for these is pylint.
On Fri, Oct 9, 2009 at 7:04 PM, Gerald Britton <gerald.britton@gmail.com>wrote:
<snip> You'd do well to consider the fact that you CC'd this comment to someone with more than 2500 commits to core, and who happens to have authored our doc system to boot. He may be qualified to talk about what is or is not pythonic. Geremy Condra
Gerald Britton writes:
The point is giving warnings for constructs that are sufficiently confusing, even having read the documentation, or typoable (in Emacs, the difference between for thing in iterable: if thing.is_good(): break else: thing = default_thing() and for thing in iterable: if thing.is_good(): break else: thing = default_thing() is one DEL when typing the code). Whether it's worth doing it for this construct is marginal to me, but I find those who advocate it to be giving plausible arguments. And you really ought to stop posting and let Steven d'Aprano carry the flag.<wink> Given *your* record of repeatedly explaining the semantics incorrectly when you have clearly read the documentation, for-else qualifies on the criterion above. The others on your list don't, especially since most of them have nearly identical semantics to the preferred form, and at least one (#4) is not currently possible to implement.
Masklinn <maskl...@masklinn.net> wrote:
You could make the same case for issuing a warning every time someone uses a mutable object as a default value in a function definition. This has _certainly_ caused a lot more confusion than the loop-else construct. But recourse to naive realism isn't a valid argument here; you're in effect punishing those who have taken their time to familiarise themselves with the language to spare those who haven't from ever having to do so. The effort should be in _learning_ the language, not in continually bypassing the training wheels.
alex23 wrote:
We don't warn about mutable default arguments because in a lot of cases the compiler doesn't know if the default argument is mutable or not, and there are multiple well-established legitimate use cases that look identical to noobish mistakes. Warning about for-else usage without break is an entirely different scenario. The comparison would hold water if we were proposing to issue a warning for *any* use of for-else. We aren't - we're only proposing to issue it for cases where the lack of a break statement suggests that the programmer has misunderstood what the clause is for. Will that give the occasional false positive due to code that is in the process of being modified? Yeah, of course it will, otherwise I'd be proposing a SyntaxError. Are those occasional false positives worth the benefit of fewer buggy programs due to misunderstandings of the for/break/else construct? In my opinion, yes. For the record, there are three cases (not counting Py3k warnings) where we currently issue syntax warnings (but generate valid code anyway). 1. Using "import *" at function level (as it disables local variable optimisations):
2. Referencing a global variable before declaring it as such:
3. Assigning to a global variable before declaring it as such:
All 3 could easily be left to pylint/pychecker style tools, but were deemed worthy of being warned about by the compiler itself. Keep in mind that using a pylint/pychecker style tool at all is a comparatively advanced Python technique. For cases like this where misunderstandings are prevalent amongst both experienced and novice Pythoneers, getting the compiler itself to offer a helping hand still strikes me as a good idea. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, 10 Oct 2009 10:22:11 pm Nick Coghlan wrote:
`import *` inside a function is illegal in Python3. It's a warning in Python2.x only because of backwards compatibility. This was the primary use-case for the warning module in the first place: to warn when functionality is being deprecated and will be removed in the future. Even though `import *` at the module level is a frequent source of bugs and confusion even for experienced coders, and the usual advice is Just Don't Do It, Python does not generate a warning for that case.
2. Referencing a global variable before declaring it as such: 3. Assigning to a global variable before declaring it as such:
These are insignificant variations of the same act: using a global before declaring it. They may have two different error messages, but the underlying problem is the same.
All 3 could easily be left to pylint/pychecker style tools, but were deemed worthy of being warned about by the compiler itself.
The first could not, because it is functionality being removed. The second and third are warning about the same thing, and this is a genuine case of Python warning about a stylistic issue. -- Steven D'Aprano
On Fri, 9 Oct 2009 07:31:22 am Masklinn wrote:
It forces me to read it. It forces me to take the time to filter out in my head real warnings for real problems, and pretend warnings for non-problems, or to run Python with some switch to disable warnings -- which means I lose the ability to get real warnings I care about. Excessive warnings trains me to ignore them. "Oh, it's some stupid compiler warning, don't worry about it, it's complaining for no good reason". (Sometimes I think some coders haven't heard of the story of The Boy Who Cried Wolf.)
The point of a warning is very much that the code works and is legal, but *in most case* is not what people would want.
Can you give any examples in the Python standard library where code raises a warning on such a basis? I would be shocked if you can give any examples. -- Steven D'Aprano
Steven D'Aprano wrote:
See my other post - there are currently 3 in the 2.x branch. Two of them are pure style warnings strongly encouraging people to put their global statements at the beginning of the function, while the third is a warning issued due to the fact that using "import *" inside a function is a really bad idea (at the very least, it forces the compiler to disable all of its normal local variable optimisations, as it has no idea what local variable names are going to exist after that import statement executes). I just checked the Py3k branch, and that adds a few more: - A warning for assert statements with a non-empty tuple as the first argument (i.e. cases where the assert will never fail) - Another two extending the global statement warnings to cover nonlocal statements as well. All 6 are cases where the code is legal and the interpreter will do the right thing, but the core developers feel people should be encouraged to write their program differently. In this thread, the proposal is that we encourage people to only use the else clause on loops containing at least one break statement. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, 10 Oct 2009 10:32:55 pm Nick Coghlan wrote:
See my previous response.
That's fundamentally the same warning: don't access a global before you declare it global. Whether that access is an assignment or a lookup is trivial.
It's actually a warning that the functionality is in the process of being removed.
assert (1, 2), "error" <stdin>:1: SyntaxWarning: assertion is always true, perhaps remove
But only if the first argument is a tuple: parentheses? And ridiculously: parentheses? Why single out always-true tuples for a warning, when other always-true items are considered legitimate, even when there is no ambiguity about what the programmer meant? This sort of inconsistency makes little sense and should stand as a cautionary tale for people putting warnings in the compiler: sometimes the warning is *bad advice*:
- Another two extending the global statement warnings to cover nonlocal statements as well.
Given that Python warns about global, it makes sense to warn about nonlocal. Its all the same thing really, just the details change: whether the declaration is global or nonlocal, whether it is an assignment or a lookup, it's the same fundamental warning: don't use a non-local variable before declaring it as such. -- Steven D'Aprano
Steven D'Aprano wrote:
<from other thread>
However, this particular warning has nothing to do with Py3k, as it has been present at least since the AST branch was merged back in 2005. import * at function level is genuinely harmful because of the effect it has on the compiler on top of the effect it has on readability. It makes sense to warn about it. This was a case of generating a warning for legal syntax that the compiler could handle because the developers considered it dubious. It was just considered so dubious that the warning was upgraded to a SyntaxError for Py3k.
Actually, we don't generate a warning for this because there are significant legitimate use cases for import * at the module level that can't be handled any other way (the two that immediately come to mind are for convenience at the interactive prompt and for bringing in methods and classes from an optional acceleration module as happens in a number of standard library modules). We can hardly generate a warning for constructs that we use ourselves. A visual grouping technique that can easily be handled with vertical whitespace or comments isn't even close to being in the same league as the legitimate use cases for import * at module level.
That's why it says "perhaps remove" rather than "remove". That hint also indicates the developer error that it is actually trying to warn against: assert (longAndComplicatedAssertionCondition, "My assertion message") That's not an assert statement using parentheses purely for line continuation - it's an assertion of a 2-tuple that will never fail, and a naive user may not realise what is happening (thus getting very confused when their assert statement appears to pass, but subsequent code relying on that assertion fails anyway). You have to either use a backslash for the line continuation here, or else ensure the comma is outside the parentheses: assert longAndComplicatedAssertionCondition, \ "My assertion message" assert longAndComplicatedAssertionCondition, ( "My assertion message") In that context, the syntax warning turns what could be a very obscure and hard to debug problem into something fairly obvious (as the only parentheses present will be the ones that are causing the comma to be misinterpreted). The warning could probably be refined such that it only triggers for a 2-tuple as the sole argument to an assert statement, but how often is such a false triggering actually going to happen in real code rather than dummy examples? For the loop else clauses, the users who are going to get themselves into trouble are the ones who *think* they know what it means, but actually don't (i.e. not the sort of people that are likely to be regularly running pylint or pychecker over their code). A SyntaxWarning built directly into the compiler would hopefully give them the hint they need in order to realise that they don't actually know what is going on. ("break? What does break have to do with using else on a loop? Hmm, maybe I better look this up or ask someone about it..."). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Arnaud Delobelle writes:
Also, it would be odd to have a language feature that generates a warning whenever it is used :)
Not really. Deprecation warnings behave that way. AFAICS, a deprecation warning is just the extreme case of a syntax warning.
On Sun, 11 Oct 2009 05:36:07 am Stephen J. Turnbull wrote:
I don't see it that way at all. A deprecation warning makes sense, it's the primary use case for the warnings module. It says "Here is a feature which is changing, or being removed -- you MUST take steps to change your code, or be stuck with this version forever, because the language is changing." A syntax warning, as being suggested, is nothing like that. It says "Here is a legitimate feature, part of the language. It's not going away. You may have a good reason for doing what you want, but just in case you're ignorant or naive or stupid, I'm going to warn you that your perfectly legal code might be doing what something other than what you expect." This is, to my mind, particularly egregious because every single language feature might be doing something other than what you expect, if you are sufficiently ignorant, naive or stupid. Such warnings not only try to guess what users want, but they make an arbitrary, and inconsistent, decision as to how much ignorance they "protect" the user from. We don't, for example, protect the user from thinking that list.sort() returns the sorted list, or that arithmetic on floats is simple, or that locals() can be meaningfully modified, or that mutable defaults are recreated every time the function is called -- all errors FAR more common than misunderstanding for...else. If your aim is to protect users from their own mistakes, then there are far more serious mistakes that we just ignore. And if our attitude is "we're all adults here", then why on earth single out such an uncommon gotcha for special treatment? -- Steven D'Aprano
On Sun, 11 Oct 2009 02:30:51 am Nick Coghlan wrote:
Right. It's so dangerous that it's becoming illegal. You can't just turn language features illegal overnight, you have to go through a period of deprecation. This makes perfect sense, and is the primary use-case for warnings in Python.
Of course we can. But we shouldn't, it would be silly. The point is that there are many language features that are pointless, silly or troublesome, and we don't -- and shouldn't -- raise warnings for them. I don't see for...else to be exceptional enough to deserve a language warning. [...]
I don't believe the language should be hand-holding the naive user to that extent. What do we do, fill the compiler up with a thousand warnings for things which might confuse some naive user? What ever happened to "We're all adults here"? The solution to naive users is for the user to learn more about the language and be less naive, not for the language to hold his hand. The compiler isn't your babysitter, and Python isn't meant to be a teaching language for kiddies. The proliferation of warnings leads to complacently among naive users ("I don't need to understand this feature to use it, the compiler will warn me if I do something wrong") and, much worse, overload among non-naive users ("it's just another damn warning, ignore it, the compiler is just complaining about nothing"). In my opinion, we've already taken too many steps down this path, lets not go any further.
It's a bug -- it is output that is generated unnecessarily. Defending it on the basis that it's a rare bug is not a wise idea.
You can't protect ignorant, naive and downright stupid users against themselves. It's a never-ending problem. Perhaps we should raise a warning when the user says: for in in range(len(sequence)): SyntaxWarning: There's probably a better way to do this. or for list.insert? PerformanceWarning: This may be slow for large lists. You may think I'm exaggerating, but Python already does something similar, only worse. The built-in sum() prohibits summing strings because it may be O(N**2) -- even though in recent versions of CPython, it may not be, even though for small numbers of small strings it makes no practical difference, and even though it allows O(N**2) summation of lists! A naive user will be lulled into a false sense of security that sum(list_of_lists, []) is fine, and an advanced user who wants to use sum() as a teaching aid to show how repeated string concatenation is slow is frustrated. -- Steven D'Aprano
Steven D'Aprano writes:
Nobody (except you and Gerald) is talking about things that "might" confuse "some naive user", nor does anybody advocate proliferating warnings or even getting within a light-year of that slippery slope. We're talking about one construct that *demonstrably* confuses *people who have taken it upon themselves to explain correct usage in this thread*, including at least one who considers himself above needing such warnings. We're talking about a single construct that the BDFL has deigned to deprecate as an unfortunate choice of keywords. Please stop wandering off topic, and address the arguments that have been presented.
On Sun, 11 Oct 2009 03:36:45 pm Stephen J. Turnbull wrote:
With all respect to those people, just because somebody takes it upon themselves to explain correct usage, doesn't mean they know the correct usage. This isn't meant as a dig at anyone here, but sadly the least competent people tend to be the most sure of themselves: http://www.ncbi.nlm.nih.gov/pubmed/10626367 http://en.wikipedia.org/wiki/Illusory_superiority To quote Charles Darwin: "ignorance more frequently begets confidence than does knowledge". I don't exclude myself in this -- we all make mistakes, and those who are on comp.lang.python may have noticed me make a clanger just yesterday. Nobody here has demonstrated that for...else is a significant problem. I'd describe it as a gotcha, and Python, like every computer language, has a number of gotchas. There's no need have the compiler defensively warn people about them.
You're kidding, right? Or is anything but acquiescence to the idea of giving a warning "off-topic"? Just because somebody proposes a change, doesn't mean that opposition to that change is off-topic. -- Steven D'Aprano
Steven D'Aprano writes:
You're kidding, right?
No.
Or is anything but acquiescence to the idea of giving a warning "off-topic"?
That's not the alternative to "kidding".
Just because somebody proposes a change, doesn't mean that opposition to that change is off-topic.
"There you go again." I have no problem with opposition, and if you go back and read my posts, that's clear. It's the bogus attribution of motives and wildly exaggerated claims that need to stop, on both sides. On your side, the "proliferation of warnings" and the "warn on anything that might trip up a naive programmer who hasn't read the docs" nonsense. Neither is in question. There's *one* warning in question, the construct is legal and unambiguous as implemented. Good luck contesting those facts. So the topic is "how useful and common is the construct, correctly used?" vs. "how much code in actual use is likely to be written under the seductive spell of the meaning of 'else' in English?" (I'm open to other suggestions, within the framework of "only one warning" and "construct is unambiguous when understood.") At the moment, my assessment is "almost useless" and "it is likely that a large majority of the code using 'breakless for-else' is written with the wrong semantics expected, and a substantial fraction of those will actually result in buggy behavior." On that basis, I'm about +0.5 on issuing a warning. I don't know if my opinion matters to anybody but me but it *is* up for grabs; my current stance is not set in concrete. I can tell you that the *style* of your last five posts or so have tended to tilt my opinion against you, while Nick and Georg (except that last ad hominem, of course) have been steadily scoring a point or two per post on content.
On Sat, Oct 10, 2009 at 9:36 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
We're talking about a single construct that the BDFL has deigned to deprecate as an unfortunate choice of keywords.
Wrong. I would not have the feature at all if I had to do it over. I would *not* choose another keyword. But I don't see the same level of danger in it that some here see. I am also against adding a syntax warning for this. It belongs in pylint etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sat, Oct 10, 2009 at 10:56 PM, Steven D'Aprano <steve@pearwood.info> wrote:
I realize that this will strike many here as heresy, but yelling "RTFM, n00b!" is not actually a productive way to resolve what is clearly a point of difficulty for a sizeable number of competent developers. Geremy Condra
Steven D'Aprano wrote:
I think it's a side effect of the way the check is implemented. It just looks at whether the first expression is a tuple constructor, and ignores the second one. While theoretically it's not ideal, it doesn't do any harm in practice, because it's so unlikely that anyone would intentionally write such an assert statement. There's no use for it whatsoever.
Hm, yes... well, at least you get an error rather than incorrect results! -- Greg
Steven D'Aprano schrieb:
This would be a problem indeed if programmers were computers. Last time I looked they were humans, and capable of not blindly following any instruction the computer gives them. [1] And even if they do, they obviously don't know how the assert statement works, and need to revisit the docs. Georg [1] Note, only speaking of programmers here ;) -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Nick Coghlan wrote:
If it is perfectly legal to put global statements anywhere in the function, and there is no intention to tighten the rule to disallow that (which would be ok with me), then I am against the warning. It suggests that one has done something wrong, when one has not. A global statement as module level has no effect, even though some newbies think it does (else why would they write it when they do?). Perhaps we should add a warning for *that*, especially given that it is more likely to indicate a real error of thinking rather and simply making the 'wrong' stylistic choice.
In this thread, the proposal is that we encourage people to only use the else clause on loops containing at least one break statement.
Whereas I want to be able to use while/else and for/else in didactic code without having to warn people against a spurious warning. Terry Jan Reedy
On Sun, Oct 11, 2009 at 12:09 AM, Terry Reedy <tjreedy@udel.edu> wrote:
Nobody is holding a gun to your head. Use it if you must- but when a construct is obviously confusing the heck out of people who are demonstrably good programmers, it's time to think seriously about warning people who are using it in ways that *do not make good sense*. All kinds of decisions in python have been made explicitly to discourage buggy, hard to read code, and the discussion here, complete with numerous mistakes and misunderstandings, constitutes abject proof that for-else is both of those things and hard to write correctly to boot. If we can't make it less dangerous, the least we can do is make it more obvious when you're screwing it up. Geremy Condra
Terry Reedy schrieb:
Why would you want to do that, except to confuse a new generation of Python programmers about its semantics? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Georg Brandl writes:
Terry Reedy schrieb:
Whereas I want to be able to use while/else and for/else in didactic code without having to warn people against a spurious warning.
Why would you want to do that, except to confuse a new generation of Python programmers about its semantics?
Sheesh. Now *you*'re doing it. He and Steven have explained their use case at length, and it's valid. Please keep to the point, which is the relative importance of the stylistic and didactic value of placing associated code inside the for-else block rather than outside it, versus the attractive nuisance that this construct is for many programmers.
Gerald Britton wrote:
If the code is written using a style that is misleading, I think it makes sense to issue a warning.
This would generate a warning, because the 'else' is unnecessary and potentially confusing here. The above code is exactly equivalent to: for x in y: if found_what_I_want(x): return True return False
Would also generate a warning. Again, the version without 'else': for a in b: if bogus(a): raise Bogus, "Found a bad one" nothing_bogus_found() In both cases, I find the version without 'else' to be easier to understand. +1 to making for/else and while/else without break generate a warning. +0 to making it an error. -1 to removing or renaming the 'else' of for and while loops. - Jacob
On Thu, Oct 8, 2009 at 9:44 AM, Gerald Britton <gerald.britton@gmail.com>wrote:
<snip> I grant you that compilers *shouldn't* be smarter than programmers, but I highly doubt that any program including an extraneous else clause is "working according to specifications", and that's the point of this proposal. Geremy Condra
geremy condra writes:
Actually, I can imagine someone developing a style where a redundant else was used to indicate "this is part of/cleanup of the for loop, and should go with it in any refactoring". Say, in a program that speaks SMTP: for line in line_factory: smtp_send(line + "\n") else: smtp_send(".\n")
Stephen J. Turnbull schrieb:
Usually, that effect can be created by newlines: for line in line_factory: smtp_send(line + "\n") smtp_send(".\n") more code goes here I'm not sure myself whether I like generating warnings for valid syntax, but in the often-misunderstood case of for/else, I think it would be reasonable. Georg
Gerald Britton writes:
That *may* be true for Python, but my face has been saved more times than I like to think by the warning on if (x = f(y)) in many C compilers. I think I would probably be saved at least once in my remaining years of Python programming by Nick's proposal, too.
On Fri, 9 Oct 2009 03:17:38 am Stephen J. Turnbull wrote:
I think I would probably be saved at least once in my remaining years of Python programming by Nick's proposal, too.
That makes no sense. If there's no difference between: for x in seq: pass else: whatever() and for x in seq: pass whatever() then what could the warning save you from? Embarrassment when people laugh at your coding style? Perhaps you're thinking of the following: for x in seq: if flag: whatever() else: # oops, I wanted this to be inside the loop something() But Nick's warning is insufficient, because it is defeated by the presence of a break *anywhere* in the loop: for x in seq: if today == tuesday: break if flag: whatever() else: # oops, I wanted this to be inside the loop something() The fact that it, occasionally, will catch a completely unrelated error (you outdented the else clause in if...else) is a fluke. -- Steven D'Aprano
Steven D'Aprano writes:
The warning can save me from the brain fart that has been repeated over and over in this thread: thinking that there *is* a difference. This thread is ample evidence that people can and do misread and miswrite that construct *even in a context where they are forewarned that they are discussing an easily misused construct*.
Perhaps you're thinking of the following:
Stop trying to read my mind. What you should be reading is some of the posts in this thread explaining that for x in seq: pass else: whatever() means if seq: for x in seq: pass else: whatever() or similar ... *after* Nick's explanation that the for/break/else idiom for search is the only semantically interesting form.
On Fri, 9 Oct 2009 12:15:10 am Masklinn wrote:
No entirely -- the compiler generates different byte-code.
Heavens no!!! What on earth gives you that idea? How can the compiler possibly know what you would like? And even if it could, it's the compiler's job to enforce the rules of the language, not to satisfy what people would like. Compiler warnings should never be spurious. Python warnings are used for meaningful warnings, such as deprecation warnings, not for stylistic ones like "by the way, you've used pass twice in a row, that's pointless". I don't want the compiler wasting my time with spurious warnings. If I want to run PyLint, I'll run PyLint. I don't want the compiler telling me I've defined a name and never used it, or declared a function that doesn't get called, or have code branches that can't possibly be taken, or anything else. I want it to compile, as fast as possible. The edit-compile-run cycle in Python is *very* lightweight, as it should be. When I want extra checking, I run PyLint, I don't want it every single time. -- Steven D'Aprano
Steven D'Aprano wrote:
I would advise against putting money on that prospect.
3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE
3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE If you leave out the newline after the loop in f2 then the final line number would change from a 5 to a 4, but the bytecode would otherwise remain identical. With the extra newline in there, the bytecode *is* identical. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Thu, 8 Oct 2009 08:24:46 pm Yuvgoog Greenle wrote:
In order to understand the "else" clause it's better that you think of it as an "if not break" after the loop.
That is completely wrong. for...else is a single block, and break jumps to the end of the block. See my previous reply for details. for x in seq: whatever else: suite is a single block. It's not the same as: for x in seq: whatever suite even if they happen to execute the same way. In the first case, the entire block is one single lump of code. In the second case, you have two independent lumps of code, a for-loop with no else, followed by an independent bunch of code following it. Lumps of code in this sense are a weaker grouping than functions, classes or modules, but they are a grouping. When you see for...else you KNOW that the else-suite belongs with the for without needing to read the suite in detail. You can't pick it up and move it somewhere else and expect it to work, they belong together, in that order. The presence of the else tells the reader that the entire for...else block is one single conceptual unit. This extra information is missing in the second case, where there is nothing suggesting that suite belongs with the for-loop: they look independent. To learn that they're not, you have to read the code in enough detail to understand it fully. -- Steven D'Aprano
Steven D'Aprano wrote:
I can see your point, but the confusion with the naive "if seq is contains at least 1 element do the loop, else do the else clause" outweighs it in my mind. That may have something to do with my personal preference for using vertical whitespace and comment lines to obtain logical groupings below the function level. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Thu, 8 Oct 2009 07:57:38 pm Stefan Rank wrote:
-1 on such a warning. This sort of check belongs in PyLint or equivalent, there is no need to complicate the compiler and make it part of the language. The compiler should be nicely lean and simple and not try to guess what the user's intention is. for...else with no break is legal code, if there's no break inside it, it is still legal code.
-1000 on making it a SyntaxError. SyntaxErrors are for things which are WRONG, not for things which are unnecessary or redundant or pointless. Should this be prohibited too? for x in seq: if 0: break else: print "whatever" What happens when the break is under a __debug__ test? The same source compiles to two different byte-codes, one of which has break, the other doesn't. Given this: code = compile("""for x in seq: if __debug__: break else: print "no break" """, '', 'exec') this is the byte-code you get running normally:
2 13 BREAK_LOOP 14 JUMP_ABSOLUTE 7 >> 17 POP_BLOCK 4 18 LOAD_CONST 0 ('no break') 21 PRINT_ITEM 22 PRINT_NEWLINE >> 23 LOAD_CONST 1 (None) 26 RETURN_VALUE and this is what you get running under python -O:
2 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 4 17 LOAD_CONST 0 ('no break') 20 PRINT_ITEM 21 PRINT_NEWLINE >> 22 LOAD_CONST 1 (None) 25 RETURN_VALUE No break in the second case. So now we have choices: * Code which runs fine normally becomes a SyntaxError when running with -O. * for...else does not require a break when running with -O, but does require it without -O. * We pile complication on top of complication and make an explicit exception for tests like if __debug__: break -- which means that we still have for loops with no break that run perfectly fine! * Stop optimizing if __debug__ tests. All four of these alternatives are unacceptable. -- Steven D'Aprano
Steven D'Aprano wrote:
No, see below.
If I understand Nick correctly, this is irrelevant. The warning/error generation would be handled by an earlier stage than the optimization that removes the break. - Jacob
Jacob Holm wrote:
I never promised that - someone else said that would happen, but it's actually implementation dependent. In the case of CPython, the compiler stage checks for recognised constants (such as 0, non-zero integers, True, False and __debug__) in if statement test expressions and doesn't even bother visiting any unreachable code. Accordingly, Steven is quite correct that having your only break statement inside "if __debug__:" would lead to the natural implementation approach for CPython issuing a SyntaxWarning when running under the optimiser. Make no mistake, the whole point of adding this warning would be to cause some currently legal code to generate a warning. It would be a policy decision, stating that "for/break/else" and "while/break/else" were the only accepted use cases for the else clause and that loop cleanup code should merely be written after the loop with either comments or vertical whitespace used for logical grouping, rather than having the else clause available for that purpose. The choice to be made is whether the potential for improvement in understanding of the nature and purpose of the else clause amongst Python programmers as a whole (even those that don't use static analysis tools like pylint or pychecker) would be worth the mild aggravation to folks like Steven and Gerald that currently use the else clause to logically group cleanup code with the relevant loop. (That said, I don't think the question should even be asked of python-dev without a patch on the tracker that actually adds the new warning. I personally have several other Python issues higher on my to-do list that I have yet to find the roundtuits to address, so I'm not likely to be producing such a patch anytime soon...) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, Oct 10, 2009 at 2:02 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'll have time to start working on a patch within 2 weeks. I've never touched the compiler but it sounds like an interesting subject. --yuv
participants (16)
-
alex23 -
Arnaud Delobelle -
Georg Brandl -
Gerald Britton -
geremy condra -
Greg Ewing -
Guido van Rossum -
Jacob Holm -
Jared Grubb -
Masklinn -
Nick Coghlan -
Stefan Rank -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy -
Yuvgoog Greenle