Idea: Extend "for ... else ..." to allow "for ... if break ..." else
There's been a discussion of the "for ... else ..." construction in Python. Here's a suggestion as to how we might improve matters, and perhaps usefully extend the language. I hope it might benefit some, without harming anyone else. Aside: I've not read the whole of the previous discussion, and it may be that some or all of what I say has already been expressed there. So apologies for any missing credits to others. The errors, I'm responsible for. Here's what happens at present. for i in items: LOOP_BODY else: print('exit via StopIteration') I find the following clearer (where 'if break' is a new language 'compound keyword'). for i in items: LOOP_BODY if break: pass else: print('exit via StopIteration') This would now allow us to write for i in items: LOOP_BODY if break: print('exit via break') else: print('exit via StopIteration') Sometimes a loop has multiple break commands, which have different purposes. So a further extension might be for i in items: LOOP_BODY if break left: print('exit via break left') elif break right: print('exit via break right') elif break: print('exit via some other break') else: print('exit via StopIteration') To allow this to work, we extend the language to allow commands such as break left break right as well as the usual break Here the identifiers 'left' and 'right' look like variable names, but in fact they are labels. The compiler can, and I think should, produce a hard-coded jump from command break right to the code block under elif break right: This is the basic idea, keeping the syntax close to Python as it now is (but of course adding to the semantics). Some programmers, and I think I'm one, might want a clearer distinction between for i in items: LOOP_BODY if break: do_something and for i in items: LOOP_BODY if broke: do_something For this reason I tentatively suggest instead for i in items: LOOP_BODY case: break left: print('exit via break left') break right: print('exit via break right') break: print('exit via some other break') else: print('exit via StopIteration') Some people have asked a built-in means to detect, at the end of the iteration, whether theitems was empty to begin with. The syntax could easily provide this, for example via case: zero: print('no items to process') However, if we adopt this perhaps we'd also like to retain the simpler for ... if break ... else ... construct. I hope this helps. -- Jonathan
Jonathan Fine writes:
An if here is already valid syntax. I'm pretty sure that the parsers can handle it. But Python statement-introducing keywords are intentionally quite distinct for human consumption. The fact that else and elif are similar in some ways is mitigated by the fact that they have very similar semantics. It's also true that the "in" and "is" operators would be somewhat confusable, but I don't think that's as important as having quite distinct statement keywords.
'break' is of course already a "hard" keyword, globally reserved. But this looks very much like elif flag: to me.
Finally, much of what this syntax allows is possible like this: for i in items: LOOP_BODY_ABOVE_BREAKS if LEFT: # NOTE: additional indentation here is hidden in LOOP_BODY # in the OP. print('exit via break left') break; LOOP_BODY_AMID_BREAKS if RIGHT: print('exit via break right') break; LOOP_BODY_AMID_BREAKS if CENTER: print('exit via some other break') break; LOOP_BODY_BELOW_BREAKS else: print('exit via StopIteration') So the new feature is to distinguish some breaks from other breaks and collect the action suites for breaks that have exactly the same actions in a single place. But how often is this going to be useful? If it *is* useful, it occurs to me that (1) this looks a lot like the try ... except ... pattern, and (2) breaks are generally perceived as exceptional exits from a loop. Instead of "if break [LABEL]", "except [LABEL]" might work, although the semantic difference between labels and exceptions might get a ton of pushback. That said, I can't think of a time where I wanted more than one kind of a break from a loop, let alone where some kinds existed in multiple instances. So I'm -1 unless we see plausible use cases for the extra power, and a before-after comparison shows a perceptible readability improvement. Steve
.
...
Steve
In the related thread I suggested using `except`, but it was largely ignored. *If* you want tou propose clearer syntax for this, please extend the loop
syntax, not the ‘if’ syntax. So, ‘case ... zero’ makes more sense than ‘if [not] break’.
—Guido
So I understand, does this mean that any extended syntax for this should be *totally* new and not draw on existing constructs such as `try-except` or `if-else`? Or just that the additional clarity should come from extending the loop rather than the implicit `if`? Mathew -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
Thank you all, particularly Guido, for your contributions. Having some examples will help support the exploration of this idea. Here's a baby example - searching in a nested loop. Suppose we're looking for the word 'apple' in a collection of books. Once we've found it, we stop. for book in books: for page in book: if 'apple' in page: break if break: break However, suppose we say that we only look at the first 5000 or so words in each book. (We suppose a page is a list of words.) This leads to the following code. for book in books: word_count = 0 for page in book: word_count += len(page) if word in page: break if word_count >= 5000: break found if break found: break At this time, I'd like us to focus on examples of existing code, and semantics that might be helpful. I think once we have this, the discussion of syntax will be easier. By the way, the word_count example is as I typed it, but it has a typo. Did you spot it when you read it? (I only noticed it when re-reading my message.) Finally, thank you for your contributions. More examples please. -- Jonathan
On 29.07.20 13:33, Jonathan Fine wrote:
This can be realized already with `else: continue`: for book in books: for page in book: if 'apple' in page: break else: continue break However it looks more like this should be a function and just return when there is a match: for book in books: for page in book: if 'apple' in page: return True Or flatten the loop with itertools: for page in it.chain.from_iterable(books): if 'apple' in page: break This can also be combined with functions `any` or `next` to check if there's a match or to get the actual page.
This also could be a function that just returns on a match. Or you could use `itertools.islice` to limit the number of words. I don't see a reason for double break here.
I think the need for two (or more) distinct `break` reasons or the same `break` reason at multiple different locations in a loop is pretty rare. Are there any counter examples? Otherwise such cases can be handled already today and there's no need for additional syntax (apart from the "else" ambiguity).
Being able to break multiple loops and having "labelled" breaks would be achievable using `except`, i.e. adding `except` to the loop statements before `else` like this: for elem in iterable:
would be sugar for: try:
I (and others) have suggested this before and no one has said it's a *bad *option, it's just been ignored, despite seeming to be an intuitive way to accomplish `else` clarity, "labelled" breaks and breaking from multiple loops. Is there a reason that this suggestion is worse / no better than adding special break syntax? As for an example for labelled breaks how about something of the form: def insert_ordered_no_duplicates(duplicate_free_list, item):
which you can conceivably have a nested case where you don't want to let duplicate inserts even be attempted like this: def insert_many_ordered_strictly_no_duplicates(dup_free_list, items):
On Wed, 5 Aug 2020 at 13:40, Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:
-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On 05/08/2020 15:29, Mathew Elman wrote:
It's certainly a reasonable option. Indeed, I would consider using the second (currently legal) version in my own code if it seemed appropriate. Some pros and cons that occur to me (I may be biased, YMMV): Pros: (1) It can cleanly separate the handling of break-type exceptions and other exceptions, if needed. (2) It actually clarifies what the dreaded "else" means! (3) it allows you to group your "break" cases using exception subclasses (although this is probably OTT for most use cases). Cons: (4) It's more work. You have to decide what exceptions to use and (most likely) create them. (5) It adds the runtime overhead of setting up the "try" and possibly raising the exception. (6) Putting (implicitly) try...except around a possibly long for-suite feels bad somehow, even though it wouldn't catch other unwanted exceptions. (7) It does not catch the "zero iterations" case. If, contra your suggestion, special syntax were to be added, it could use "except" instead of "if ... elif". For example (this is just doodling, it's not a fully thought-out proposal): for x in range(10): ... if <something happens>: break "oops" # under the hood this is a compiled jump, not the raising of an exception if <something else happens>: break 42 except "oops": <handle something> except 42: <handle something else> except break: # catches all "break"s not already explicitly caught pass else: print("Did not break") Rob Cliffe
Mathew Elman writes:
This would certainly be consistent with the existing use of the except keyword, unlike the proposal to have it take both exceptions (in 'try' statements) and break labels (in 'for' and 'while' statements). However, we would probably not want to burden all loops with the exception-handling machinery, so the compiler would have to do some hacky backtracking (I doubt that the arbitrary lookahead needed to handle "maybe we got some except clauses coming?" during parsing would be acceptable) and fill that in *after* recognizing that there are except clauses in this for statement. Second, generally Python tries to avoid overloading keywords with multiple semantics. The potential for confusion and misunderstanding of "except" (which I've suggested myself and now dislike) is pretty large I think. It might be possible to save that level of indentation with this syntax: try for elem in iterable: ... if should_break(elem): raise SomeException except SomeException as e: handle_break_behaviour(e) else: print("Did not break") (and I suppose you could do the same for any control flow statement, although I'm not sure offhand that the various keywords are 100% disjoint -- that would need to be checked). But I don't think it's worth it. I don't see enough benefits from this mixing of try and for to make it worth the complexity.
I (and others) have suggested this before and no one has said it's a *bad *option,
It is, though, for the reasons above as well as the reasons Rob gives in his parallel followup. Steve
Thank you both for the feedback. Some pros and cons that occur to me (I may be biased, YMMV):
(5) It adds the runtime overhead of setting up the "try" and possibly
It is possible that 4 and 7 may be fixed by adding 2 new Exception subclasses, e.g. `BreakException`, which would "replace" break although using break would still be possible, and `NoIterationException` (though this may not be the cleanest approach). The question then becomes what to do with these when there is no `except`. At the sacrifice of automatically breaking out of multiple loops, they could be silently ignored unless caught explicitly? I do not know well enough how the python interpreter sets up `try-except` blocks to realistically respond to 5 and 6 but I can indeed see these being sticking points. On Thu, 6 Aug 2020 at 08:57, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Would it be so bad to simply wrap for loops with except clauses automatically that can be clobbered or not? e.g.
for x in iterable: ...
really does something like this
try:
for x in iterable:
Meaning that it doesn't need to wait to see if it does and backtrace but can just trust that there will be the default reraise? Apologies if this is a stupid question. Second, generally Python tries to avoid overloading keywords with
I am not sure I follow you here since the `except` is being used in the same way as in `try...except` so I wouldn't expect a high potential for confusion, although I suppose that is, ironically, the same line of thinking as with `for...else`.
I agree that this seems to add some complexity but I actually like this a lot. It would surely also address the issue of the parser knowing it should be looking for exceptions or not?
Thank you, I appreciate it at least being addressed! -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On 06/08/2020 08:57, Stephen J. Turnbull wrote:
Did you see Guido's post when I raised a similar object to detecting "zero iterations", where it would be unacceptable to slow down all for-loops, so they would have to be compiled differently? I wrote (in ignorance:-)): So: You're asking that the bytecode generated for the for-loop depends on something that happens (or not) after the end of the for-loop body (which could be arbitrarily long). I speak from ignorance, but I suspect that even with the new parser, which I am reliably informed can make the tea and implement world peace, that would be asking a lot. Guido replied: Rest assured this is not a problem. In any case it’s the compiler, not the parser, that generates the bytecode, from the AST. The compiler always has the full AST available before it is asked to generate any bytecode. The new parser just allows more flexible syntactic constructs, esp. “soft keywords”.
IMO this is a bit disingenuous: "as" can be used with "import" and with context managers with quite different semantics. "del" can be used to remove a variable binding, a sequence element, a sequence slice or a dictionary key. "not" can be used as Boolean negation or in the compound operator "not in". Whereas the new use of "except" that Matthew is proposing is very similar to its existing use (certainly conceptually, if not in the implementation details).
(To be clear: although I'm defending Matthew's proposal here, my preferred option is still some new syntax.)
Rob Cliffe writes:
Yes. My confusion (as you'll see elsewhere) was about the AST, not the parser.
IMO this is a bit disingenuous:
I'm not trying to mislead anybody, or try to imply there aren't cases where keywords have been repurposed.
"as" can be used with "import" and with context managers with quite different semantics.
I would disagree that the semantics are different. Context managers and imports have quite different semantics, but in both cases the "as" clause has name binding semantics, while the object bound is determined by the statement, not by the "as" clause.
"del" can be used to remove a variable binding, a sequence element, a sequence slice or a dictionary key.
The connection is more tenuous, but in each case an object loses a reference. I see your point of view, especially since the semantics of del on sequence elements and slices affects the "names" of other sequence elements, but I think the "reference destruction" semantics are "sufficiently" similar across the different uses of "del".
"not" can be used as Boolean negation or in the compound operator "not in".
Which is a negation. I don't see how anybody reading that could mistake the meaning.
As a restricted goto, that is true. In fact, it's so similar that we may as well use the original! Is one level of indentation really worth it? What I see as different is that Matthew's proposal is for a purpose that is explicitly local to the loop statement, where except is explicitly nonlocal. Another way to put it is in this thread "except" is proposed as marking a goto target, where in a try "except" is almost a "come from" (although not with full INTERCAL compatility). I also wonder about try ... excepts nested in a for loop with excepts. That's probably no harder to deal with than nested loops with breaks at different levels (but that can be a bit confusing).
(To be clear: although I'm defending Matthew's proposal here, my preferred option is still some new syntax.)
"try" is enough to implement any of the use cases in the relatively rare cases it's needed. On the plus side, a "try" and its except clauses require a bit of code to set up. I don't know whether that's a major consideration, but one advantage of a new implementation for the purpose of implementing "labelled breaks" would be to have a lighter implementation. Whether it's worth it is above my pay grade.
On Thu, Aug 6, 2020 at 01:00 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Maybe you’re thinking of a “one-pass” compiler, like the original Pascal compiler, that generates code during parsing. That’s not how modern compilers work anymore. :-) The parser builds an AST, and the code generator uses the whole AST as input. It would not be “hacky” for the compiler to generate different code for the first half of a compound statement depending on how it ends. (However, it *would* be hacky to vary based on whether a statement was followed by a certain other statement, as that would require going “up” in the AST.) —Guido -- --Guido (mobile)
Guido van Rossum writes:
Maybe you’re thinking of a “one-pass” compiler, like the original Pascal compiler, that generates code during parsing.
Not really. The "after" phrasing comes from the parsing part which does move through the source pretty much linearly in both the traditional and PEG parsers (as far as I understand the latter). What I really meant with respect to the code generation was what you describe as "going 'up'" with respect to the AST. Evidently I got that wrong. I have a somewhat nebulous understanding of which way is "up", I guess. :-) I need to go back to school on compiler tech.
On Wed, Jul 29, 2020 at 02:51 Mathew Elman <mathew.elman@ocado.com> wrote:
Actually, given that ‘else:’ already confuses people I very much doubt that any other new construct will be acceptable. It is totally five not to “solve” this problem at all — all use cases can be done by adding flag variables explicitly manipulated by the user’s code.
-- --Guido (mobile)
On Wed, 29 Jul 2020 at 14:42, Guido van Rossum <guido@python.org> wrote:
I understand that the "problem" with `else` is aesthetic, though one could debate the importance of such a problem. What I mean to ask is, for example, would having `for-except-else` (analogous to `try-except-else`) be an acceptable extension of "the loop syntax, not the ‘if’ syntax"?
-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On Wed, Jul 29, 2020 at 07:01 Mathew Elman <mathew.elman@ocado.com> wrote:
I honestly and strongly believe that we should do nothing here. Python thrives because it is relatively simple. Adding new syntax to deal with looping special cases makes it less simple, and encourages a bad coding style (nested loops, multiple breaks...).
-- --Guido (mobile)
On 2020-07-29 at 07:09:05 -0700, Guido van Rossum <guido@python.org> wrote:
I was about to go off on another Get Off My Lawn rant, so thank you for putting this so succintly and so politely. :-) If I end up with more than one flag controlling my search loop, then it's time to break (pun intended) the logic into smaller/simpler pieces rather than to look for a more complicated language construct.
Guido wrote: I honestly and strongly believe that we should do nothing here. Python
I agree with about 80% of this statement. In particular I believe strongly that we should do nothing here, without strong evidence that the change will bring at least significant benefit to some users, and no or very little harm to the rest. I also believe that meeting that criteria is only the first step. It is quite possible for a reasonable request for change to be reasonably refused. Again, I believe that one reason why Python thrives is that it is relatively simple for novices. Another reason is that it provides facilities such as __dunder__ methods and metaclasses, so that experts can do advanced things. List comprehensions perhaps lie somewhere in between. From a syntactic point of view, I think it important that we do what we can to avoid novices accidentally encountering an advanced feature. Here's an example. PEP 622 -- Structural Pattern Matching suggests introducing a language feature that, I think, most novices will find hard to understand properly. But it seems that experts will like it's power and simplicity, for doing some advanced things. PEP 622 introduces new keywords 'match' and 'case'. The new keywords make it easy for us to warn novices not to use it (and we don't put it in the beginners tutorial). Here's the example from the PEP. def make_point_3d(pt): match pt: case (x, y): return Point3d(x, y, 0) case (x, y, z): return Point3d(x, y, z) case Point2d(x, y): return Point3d(x, y, 0) case Point3d(_, _, _): return pt case _: raise TypeError("not a point we support") Some conditions necessary for "break to a label" to be accepted are strong use cases, and a syntax that keeps the construction out of the hands of the novices. These conditions are not sufficient. I intend to keep my eyes open, as I go about my daily activities, of strong use cases. If I don't see any, then most likely you won't hear anything more from me on this suggestion. -- Jonathan
Jonathan Fine writes:
An if here is already valid syntax. I'm pretty sure that the parsers can handle it. But Python statement-introducing keywords are intentionally quite distinct for human consumption. The fact that else and elif are similar in some ways is mitigated by the fact that they have very similar semantics. It's also true that the "in" and "is" operators would be somewhat confusable, but I don't think that's as important as having quite distinct statement keywords.
'break' is of course already a "hard" keyword, globally reserved. But this looks very much like elif flag: to me.
Finally, much of what this syntax allows is possible like this: for i in items: LOOP_BODY_ABOVE_BREAKS if LEFT: # NOTE: additional indentation here is hidden in LOOP_BODY # in the OP. print('exit via break left') break; LOOP_BODY_AMID_BREAKS if RIGHT: print('exit via break right') break; LOOP_BODY_AMID_BREAKS if CENTER: print('exit via some other break') break; LOOP_BODY_BELOW_BREAKS else: print('exit via StopIteration') So the new feature is to distinguish some breaks from other breaks and collect the action suites for breaks that have exactly the same actions in a single place. But how often is this going to be useful? If it *is* useful, it occurs to me that (1) this looks a lot like the try ... except ... pattern, and (2) breaks are generally perceived as exceptional exits from a loop. Instead of "if break [LABEL]", "except [LABEL]" might work, although the semantic difference between labels and exceptions might get a ton of pushback. That said, I can't think of a time where I wanted more than one kind of a break from a loop, let alone where some kinds existed in multiple instances. So I'm -1 unless we see plausible use cases for the extra power, and a before-after comparison shows a perceptible readability improvement. Steve
.
...
Steve
In the related thread I suggested using `except`, but it was largely ignored. *If* you want tou propose clearer syntax for this, please extend the loop
syntax, not the ‘if’ syntax. So, ‘case ... zero’ makes more sense than ‘if [not] break’.
—Guido
So I understand, does this mean that any extended syntax for this should be *totally* new and not draw on existing constructs such as `try-except` or `if-else`? Or just that the additional clarity should come from extending the loop rather than the implicit `if`? Mathew -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
Thank you all, particularly Guido, for your contributions. Having some examples will help support the exploration of this idea. Here's a baby example - searching in a nested loop. Suppose we're looking for the word 'apple' in a collection of books. Once we've found it, we stop. for book in books: for page in book: if 'apple' in page: break if break: break However, suppose we say that we only look at the first 5000 or so words in each book. (We suppose a page is a list of words.) This leads to the following code. for book in books: word_count = 0 for page in book: word_count += len(page) if word in page: break if word_count >= 5000: break found if break found: break At this time, I'd like us to focus on examples of existing code, and semantics that might be helpful. I think once we have this, the discussion of syntax will be easier. By the way, the word_count example is as I typed it, but it has a typo. Did you spot it when you read it? (I only noticed it when re-reading my message.) Finally, thank you for your contributions. More examples please. -- Jonathan
On 29.07.20 13:33, Jonathan Fine wrote:
This can be realized already with `else: continue`: for book in books: for page in book: if 'apple' in page: break else: continue break However it looks more like this should be a function and just return when there is a match: for book in books: for page in book: if 'apple' in page: return True Or flatten the loop with itertools: for page in it.chain.from_iterable(books): if 'apple' in page: break This can also be combined with functions `any` or `next` to check if there's a match or to get the actual page.
This also could be a function that just returns on a match. Or you could use `itertools.islice` to limit the number of words. I don't see a reason for double break here.
I think the need for two (or more) distinct `break` reasons or the same `break` reason at multiple different locations in a loop is pretty rare. Are there any counter examples? Otherwise such cases can be handled already today and there's no need for additional syntax (apart from the "else" ambiguity).
Being able to break multiple loops and having "labelled" breaks would be achievable using `except`, i.e. adding `except` to the loop statements before `else` like this: for elem in iterable:
would be sugar for: try:
I (and others) have suggested this before and no one has said it's a *bad *option, it's just been ignored, despite seeming to be an intuitive way to accomplish `else` clarity, "labelled" breaks and breaking from multiple loops. Is there a reason that this suggestion is worse / no better than adding special break syntax? As for an example for labelled breaks how about something of the form: def insert_ordered_no_duplicates(duplicate_free_list, item):
which you can conceivably have a nested case where you don't want to let duplicate inserts even be attempted like this: def insert_many_ordered_strictly_no_duplicates(dup_free_list, items):
On Wed, 5 Aug 2020 at 13:40, Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:
-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On 05/08/2020 15:29, Mathew Elman wrote:
It's certainly a reasonable option. Indeed, I would consider using the second (currently legal) version in my own code if it seemed appropriate. Some pros and cons that occur to me (I may be biased, YMMV): Pros: (1) It can cleanly separate the handling of break-type exceptions and other exceptions, if needed. (2) It actually clarifies what the dreaded "else" means! (3) it allows you to group your "break" cases using exception subclasses (although this is probably OTT for most use cases). Cons: (4) It's more work. You have to decide what exceptions to use and (most likely) create them. (5) It adds the runtime overhead of setting up the "try" and possibly raising the exception. (6) Putting (implicitly) try...except around a possibly long for-suite feels bad somehow, even though it wouldn't catch other unwanted exceptions. (7) It does not catch the "zero iterations" case. If, contra your suggestion, special syntax were to be added, it could use "except" instead of "if ... elif". For example (this is just doodling, it's not a fully thought-out proposal): for x in range(10): ... if <something happens>: break "oops" # under the hood this is a compiled jump, not the raising of an exception if <something else happens>: break 42 except "oops": <handle something> except 42: <handle something else> except break: # catches all "break"s not already explicitly caught pass else: print("Did not break") Rob Cliffe
Mathew Elman writes:
This would certainly be consistent with the existing use of the except keyword, unlike the proposal to have it take both exceptions (in 'try' statements) and break labels (in 'for' and 'while' statements). However, we would probably not want to burden all loops with the exception-handling machinery, so the compiler would have to do some hacky backtracking (I doubt that the arbitrary lookahead needed to handle "maybe we got some except clauses coming?" during parsing would be acceptable) and fill that in *after* recognizing that there are except clauses in this for statement. Second, generally Python tries to avoid overloading keywords with multiple semantics. The potential for confusion and misunderstanding of "except" (which I've suggested myself and now dislike) is pretty large I think. It might be possible to save that level of indentation with this syntax: try for elem in iterable: ... if should_break(elem): raise SomeException except SomeException as e: handle_break_behaviour(e) else: print("Did not break") (and I suppose you could do the same for any control flow statement, although I'm not sure offhand that the various keywords are 100% disjoint -- that would need to be checked). But I don't think it's worth it. I don't see enough benefits from this mixing of try and for to make it worth the complexity.
I (and others) have suggested this before and no one has said it's a *bad *option,
It is, though, for the reasons above as well as the reasons Rob gives in his parallel followup. Steve
Thank you both for the feedback. Some pros and cons that occur to me (I may be biased, YMMV):
(5) It adds the runtime overhead of setting up the "try" and possibly
It is possible that 4 and 7 may be fixed by adding 2 new Exception subclasses, e.g. `BreakException`, which would "replace" break although using break would still be possible, and `NoIterationException` (though this may not be the cleanest approach). The question then becomes what to do with these when there is no `except`. At the sacrifice of automatically breaking out of multiple loops, they could be silently ignored unless caught explicitly? I do not know well enough how the python interpreter sets up `try-except` blocks to realistically respond to 5 and 6 but I can indeed see these being sticking points. On Thu, 6 Aug 2020 at 08:57, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Would it be so bad to simply wrap for loops with except clauses automatically that can be clobbered or not? e.g.
for x in iterable: ...
really does something like this
try:
for x in iterable:
Meaning that it doesn't need to wait to see if it does and backtrace but can just trust that there will be the default reraise? Apologies if this is a stupid question. Second, generally Python tries to avoid overloading keywords with
I am not sure I follow you here since the `except` is being used in the same way as in `try...except` so I wouldn't expect a high potential for confusion, although I suppose that is, ironically, the same line of thinking as with `for...else`.
I agree that this seems to add some complexity but I actually like this a lot. It would surely also address the issue of the parser knowing it should be looking for exceptions or not?
Thank you, I appreciate it at least being addressed! -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On 06/08/2020 08:57, Stephen J. Turnbull wrote:
Did you see Guido's post when I raised a similar object to detecting "zero iterations", where it would be unacceptable to slow down all for-loops, so they would have to be compiled differently? I wrote (in ignorance:-)): So: You're asking that the bytecode generated for the for-loop depends on something that happens (or not) after the end of the for-loop body (which could be arbitrarily long). I speak from ignorance, but I suspect that even with the new parser, which I am reliably informed can make the tea and implement world peace, that would be asking a lot. Guido replied: Rest assured this is not a problem. In any case it’s the compiler, not the parser, that generates the bytecode, from the AST. The compiler always has the full AST available before it is asked to generate any bytecode. The new parser just allows more flexible syntactic constructs, esp. “soft keywords”.
IMO this is a bit disingenuous: "as" can be used with "import" and with context managers with quite different semantics. "del" can be used to remove a variable binding, a sequence element, a sequence slice or a dictionary key. "not" can be used as Boolean negation or in the compound operator "not in". Whereas the new use of "except" that Matthew is proposing is very similar to its existing use (certainly conceptually, if not in the implementation details).
(To be clear: although I'm defending Matthew's proposal here, my preferred option is still some new syntax.)
Rob Cliffe writes:
Yes. My confusion (as you'll see elsewhere) was about the AST, not the parser.
IMO this is a bit disingenuous:
I'm not trying to mislead anybody, or try to imply there aren't cases where keywords have been repurposed.
"as" can be used with "import" and with context managers with quite different semantics.
I would disagree that the semantics are different. Context managers and imports have quite different semantics, but in both cases the "as" clause has name binding semantics, while the object bound is determined by the statement, not by the "as" clause.
"del" can be used to remove a variable binding, a sequence element, a sequence slice or a dictionary key.
The connection is more tenuous, but in each case an object loses a reference. I see your point of view, especially since the semantics of del on sequence elements and slices affects the "names" of other sequence elements, but I think the "reference destruction" semantics are "sufficiently" similar across the different uses of "del".
"not" can be used as Boolean negation or in the compound operator "not in".
Which is a negation. I don't see how anybody reading that could mistake the meaning.
As a restricted goto, that is true. In fact, it's so similar that we may as well use the original! Is one level of indentation really worth it? What I see as different is that Matthew's proposal is for a purpose that is explicitly local to the loop statement, where except is explicitly nonlocal. Another way to put it is in this thread "except" is proposed as marking a goto target, where in a try "except" is almost a "come from" (although not with full INTERCAL compatility). I also wonder about try ... excepts nested in a for loop with excepts. That's probably no harder to deal with than nested loops with breaks at different levels (but that can be a bit confusing).
(To be clear: although I'm defending Matthew's proposal here, my preferred option is still some new syntax.)
"try" is enough to implement any of the use cases in the relatively rare cases it's needed. On the plus side, a "try" and its except clauses require a bit of code to set up. I don't know whether that's a major consideration, but one advantage of a new implementation for the purpose of implementing "labelled breaks" would be to have a lighter implementation. Whether it's worth it is above my pay grade.
On Thu, Aug 6, 2020 at 01:00 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Maybe you’re thinking of a “one-pass” compiler, like the original Pascal compiler, that generates code during parsing. That’s not how modern compilers work anymore. :-) The parser builds an AST, and the code generator uses the whole AST as input. It would not be “hacky” for the compiler to generate different code for the first half of a compound statement depending on how it ends. (However, it *would* be hacky to vary based on whether a statement was followed by a certain other statement, as that would require going “up” in the AST.) —Guido -- --Guido (mobile)
Guido van Rossum writes:
Maybe you’re thinking of a “one-pass” compiler, like the original Pascal compiler, that generates code during parsing.
Not really. The "after" phrasing comes from the parsing part which does move through the source pretty much linearly in both the traditional and PEG parsers (as far as I understand the latter). What I really meant with respect to the code generation was what you describe as "going 'up'" with respect to the AST. Evidently I got that wrong. I have a somewhat nebulous understanding of which way is "up", I guess. :-) I need to go back to school on compiler tech.
On Wed, Jul 29, 2020 at 02:51 Mathew Elman <mathew.elman@ocado.com> wrote:
Actually, given that ‘else:’ already confuses people I very much doubt that any other new construct will be acceptable. It is totally five not to “solve” this problem at all — all use cases can be done by adding flag variables explicitly manipulated by the user’s code.
-- --Guido (mobile)
On Wed, 29 Jul 2020 at 14:42, Guido van Rossum <guido@python.org> wrote:
I understand that the "problem" with `else` is aesthetic, though one could debate the importance of such a problem. What I mean to ask is, for example, would having `for-except-else` (analogous to `try-except-else`) be an acceptable extension of "the loop syntax, not the ‘if’ syntax"?
-- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the "Ocado Group" are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.
On Wed, Jul 29, 2020 at 07:01 Mathew Elman <mathew.elman@ocado.com> wrote:
I honestly and strongly believe that we should do nothing here. Python thrives because it is relatively simple. Adding new syntax to deal with looping special cases makes it less simple, and encourages a bad coding style (nested loops, multiple breaks...).
-- --Guido (mobile)
On 2020-07-29 at 07:09:05 -0700, Guido van Rossum <guido@python.org> wrote:
I was about to go off on another Get Off My Lawn rant, so thank you for putting this so succintly and so politely. :-) If I end up with more than one flag controlling my search loop, then it's time to break (pun intended) the logic into smaller/simpler pieces rather than to look for a more complicated language construct.
Guido wrote: I honestly and strongly believe that we should do nothing here. Python
I agree with about 80% of this statement. In particular I believe strongly that we should do nothing here, without strong evidence that the change will bring at least significant benefit to some users, and no or very little harm to the rest. I also believe that meeting that criteria is only the first step. It is quite possible for a reasonable request for change to be reasonably refused. Again, I believe that one reason why Python thrives is that it is relatively simple for novices. Another reason is that it provides facilities such as __dunder__ methods and metaclasses, so that experts can do advanced things. List comprehensions perhaps lie somewhere in between. From a syntactic point of view, I think it important that we do what we can to avoid novices accidentally encountering an advanced feature. Here's an example. PEP 622 -- Structural Pattern Matching suggests introducing a language feature that, I think, most novices will find hard to understand properly. But it seems that experts will like it's power and simplicity, for doing some advanced things. PEP 622 introduces new keywords 'match' and 'case'. The new keywords make it easy for us to warn novices not to use it (and we don't put it in the beginners tutorial). Here's the example from the PEP. def make_point_3d(pt): match pt: case (x, y): return Point3d(x, y, 0) case (x, y, z): return Point3d(x, y, z) case Point2d(x, y): return Point3d(x, y, 0) case Point3d(_, _, _): return pt case _: raise TypeError("not a point we support") Some conditions necessary for "break to a label" to be accepted are strong use cases, and a syntax that keeps the construction out of the hands of the novices. These conditions are not sufficient. I intend to keep my eyes open, as I go about my daily activities, of strong use cases. If I don't see any, then most likely you won't hear anything more from me on this suggestion. -- Jonathan
participants (7)
-
2QdxY4RzWzUUiLuE＠potatochowder.com -
Dominik Vilsmeier -
Guido van Rossum -
Jonathan Fine -
Mathew Elman -
Rob Cliffe -
Stephen J. Turnbull