26.07.19 23:46, MRAB пише:

On 2019-07-26 19:26, Serhiy Storchaka wrote: [snip]

...

I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".

      for VAR in EXPR:           BLOCK       except EXC:           HANDLER

should be equivalent to

      try:           _it = iter(EXPR)       except EXC:           HANDLER       else:           while True:               try:                   VAR = next(_it)               except StopIteration:                   break               except EXC:                   HANDLER                   break               BLOCK

[snip] 1. The 'for' loop can have an 'else' clause, and so can the 'try' statement. Is there any ambiguity over its meaning?

An 'else' clause is already have different meanings in different statements: 'if', 'try' and 'while'/'for'. This proposition does not add anything new to this. If an exception is not raised in 'for' or 'with' then its body is executed. So no need to add an 'else' clause in the meaning of 'try'. There will be no conflict between two 'else'.

2. In your example you have it catching the exception if EXPR or iter(EXPR) raises. Is that a good idea?

Definitely we should catch the exception if EXPR is raised in the 'with' statement, because different content managers can acquire resources either in `__enter__` or in constructor (like open()). There is less confidence about 'for', but I think that it is better to catch it, because iter() is called implicitly and there is no way to catch an exception in it. iter() can raise an exception: >>> f = open('python') >>> f.close() >>> iter(f) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: I/O operation on closed file. But all this is discussable.

26.07.19 23:57, Guido van Rossum пише:

These are interesting ideas. It looks like you intend the except clause of the for loop to *only* cover the iter() and next() calls that are implicit in the for loop. You're right that it's awkward to catch exceptions there.

Right. If you want to catch an exception in the loop body you have to wrap the loop body with "try ... except". If you want to catch an exception in the loop body *and* the 'for' clause you have to wrap the whole "for" statement with "try ... except". No new syntax is needed for this. But if you want to catch an exception *only* in the 'for' clause you can't (without getting rid of the "for" statement at all).

However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)

I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.

On 07/27/2019 01:19 AM, Serhiy Storchaka wrote:

26.07.19 23:57, Guido van Rossum пише:

...

However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)

And it's what I thought when I saw the psuedo-code.

I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.

Sorry, but that mistake will be made many, many times by the same people. It's taken me several years to sort out `for...else`, and even now I have to think "it's a search loop, adapt your algorithm". Another classic example is allowing single-"=" assignment in tests: if some_var = 7: ... Sure, folks /know/ what it means, but it's a common bug because it doesn't read as "if some_var is assigned 7" but as "if some_var is equal to 7". -- ~Ethan~

I think the focus shouldn't be on whether such syntax is possibly confusing or not because a) People will always remember that's *either* one way or the other, but are very unlikely to just assume one; hence they can always check what it does, and more importantly, b) It's actually pretty easy to remember which way it is, by just considering that a syntax feature exists for scenarios that can't be easily solved otherwise. For `for ... else` it's actually more tricky than for the proposed syntax because that doesn't let you distinguish between "`else` is only executed if the loop didn't `break`" and "`else` is only executed if the loop didn't execute at all". But for `for ... except` or `while ... except` it will be even more obvious because `try ... except`ing the body is just as easy as, well, `try ... except`ing the body; it's pretty clear that no new syntax would be required for that and hence it must concern the statement itself since that's not easy to work around. The only possible confusion I see is when people look at something like this: for ...: LENGTHY BODY except ...: HANDLER else: SOMETHING_ELSE and only look at the `except ... else` part and readily assume that the two are complementary. But that's easy to prevent by only allowing the opposite order, i.e. `for ... else ... except`, stressing that the two are unrelated in this case. In the end a feature should be driven by its usefulness to the community and whether it provides a solution for an otherwise (hard|awkward)-to-solve problem. I could imagine that due to the awkward workaround, especially regarding `with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`). Someone who encounters that feature for the first time, e.g. when reviewing some code, will probably check what it does and even if not, the assumption that it `except`s the whole body should be perplexing since (a) excepting whole blocks by default is not really best practice and (b) it's pretty easy to accomplish that with existing syntax.

Sorry, that does not convince me. You're assuming that everybody is a language designer. Many Python users actually have little language design sense, and you shouldn't need to have it in order to be able to use the language. People are productive by learning to recognize and copy patterns, but their pattern recognition isn't guided by what you can reason out from thinking about how the implementation works. An issue with except clauses in particular is that during normal execution they don't get executed, and hence they are often not tested carefully. (Witness the evergreen bug of having a typo in your exception logging code that takes down production.) In the case of the proposed except clause this will probably mean that people will add except clauses to for-loops because they've read somewhere "for-loops can now have an except clause" and never read the rest of the description, and then they'll add a non-functional except-clause to a for-loop, thinking they've solved a certain potential problem. So now future maintainers have *two* problems: the exception is not caught when it was meant to be caught, and there is a mysterious except-clause on the for-loop that nobody knows for sure why it was added. I don't think there's a solution that solves the specific issue without confusing most people. Users of for-loops should probably do one of three things: 1. Don't worry about exceptions in the iter() and next() calls. 2. Put a try/except around the entire for-loop and don't worry about whether the exception came from an iter() or next() call or from the loop body. 3. Rewrite things without a for-loop, like Serhiy showed in his first message. This is ugly but the need should be very rare. On Mon, Jul 29, 2019 at 12:26 PM Dominik Vilsmeier wrote:

I think the focus shouldn't be on whether such syntax is possibly confusing or not because

a) People will always remember that's *either* one way or the other, but are very unlikely to just assume one; hence they can always check what it does, and more importantly, b) It's actually pretty easy to remember which way it is, by just considering that a syntax feature exists for scenarios that can't be easily solved otherwise. For `for ... else` it's actually more tricky than for the proposed syntax because that doesn't let you distinguish between "`else` is only executed if the loop didn't `break`" and "`else` is only executed if the loop didn't execute at all". But for `for ... except` or `while ... except` it will be even more obvious because `try ... except`ing the body is just as easy as, well, `try ... except`ing the body; it's pretty clear that no new syntax would be required for that and hence it must concern the statement itself since that's not easy to work around.

The only possible confusion I see is when people look at something like this:

for ...: LENGTHY BODY except ...: HANDLER else: SOMETHING_ELSE

and only look at the `except ... else` part and readily assume that the two are complementary. But that's easy to prevent by only allowing the opposite order, i.e. `for ... else ... except`, stressing that the two are unrelated in this case.

In the end a feature should be driven by its usefulness to the community and whether it provides a solution for an otherwise (hard|awkward)-to-solve problem. I could imagine that due to the awkward workaround, especially regarding `with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`). Someone who encounters that feature for the first time, e.g. when reviewing some code, will probably check what it does and even if not, the assumption that it `except`s the whole body should be perplexing since (a) excepting whole blocks by default is not really best practice and (b) it's pretty easy to accomplish that with existing syntax. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JIHFJT... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Mon, Jul 29, 2019 at 2:36 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

On Jul 29, 2019, at 12:24, Dominik Vilsmeier wrote:

...

I could imagine that due to the awkward workaround, especially regarding

`with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`).

I think it’s a problem if this feature is accepted at a level more or less akin for for…else.

The problem that for…else solves is a bit clunkier to solve without it, but not that bad—novices can and do figure out how to write correct code without else. So, while it’s a shame so many people never learn to use it, it’s not a huge deal.

The problem that with…except solves is so much harder to solve without it that even experienced developers usually just punt and try…except the whole block around the with. That’s a lot worse. And this isn’t a rare thing; it comes up all the time in all kinds of code. So it really calls for a syntax that most people will learn to use, not something like for…else that some experts use but most people live without.

I am *guessing* the problem here is something like this: with open(filename) as f: data = f.read() raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before). Still I wouldn't call this "a lot worse". What makes you say that?

(But I don’t have a better syntax to offer. I actually like with…except, if it were just for my use, but on reflection I can see why others find it confusing. Just like for…else.)

In hindsight I probably added for..else prematurely; it would never be accepted now. But at this point there would be a cost at removing it as well and I don't think that's worth considering. But if the bar is about as high for with...except as it would be if for...else was proposed today, then I think it's clear we should not add with...except (nor for...except, whose use case is much weaker IMO). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Mon, Jul 29, 2019 at 3:51 PM Eric Fahlgren wrote:

On Mon, Jul 29, 2019 at 2:58 PM Guido van Rossum wrote:

...

I am *guessing* the problem here is something like this:

with open(filename) as f: data = f.read()

raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before).

Isn't the appropriate solution to that one as follows? It's pretty clear what you're trying to do, so the extra couple of lines aren't all that obtrusive.

try: f = open('x', 'r') except IOError as whatever: handle whatever else: with f: data = f.read()

It's quite verbose though. And it makes you think "oh, I don't need `with`, I can use `finally: f.close()`" -- until the IOError hits and you get `UnboundLocalError: f`... -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Jul 29, 2019, at 14:49, Guido van Rossum wrote:

Still I wouldn't call this "a lot worse". What makes you say that?

Not “worse” as in more common, or more dangerous (I don’t know if that’s true or not), but as in harder to work around. for…else being confusing and effectively “experts only” just means most people don’t get to save a line of code and an extra variable—not ideal, but not a huge problem. with…except being confusing and effectively “experts only” means that most people are unable to write correct exception-handling code. So if this kind of code really is a serious problem, then the proposal doesn’t solve it; we need a proposal everyone will be able to learn and use. (And if it’s not a serious problem, then the proposal isn’t needed in the first place.)

On 07/29/2019 06:35 PM, Steven D'Aprano wrote:

3. syntax that solves a problem, but may mislead people to think it solves a different problem until they learn better;

3.1 syntax that solves a problem, but is misleading to many who are still confused after they "have learnt better"

I feel your pain: I don't understand async code when I see it either.

Neither do I, but I have no need to. I do have a need to use for...else, and every time I do I have to think through the "this is a search loop" to get the right semantics (sorry for bringing that up again, but it's a great example). If this "with...except" leads to more robust code then I think many would like to use it, but not if it's confusing as that would lead to less robust code. -- ~Ethan~

On Tue, Jul 30, 2019 at 1:13 AM Paul Moore wrote:

On Tue, 30 Jul 2019 at 04:00, Ethan Furman wrote:

...

If this "with...except" leads to more robust code then I think many would like to use it, but not if it's confusing as that would lead to less robust code.

For me, "it's for robust code" is sufficient hint that I now remember what it does (much the same way that "it's for search loops" is the hint I need for for...else). And when I'm trying to write robust code, I *do* worry about exceptions in the with expression, and the fact that I can't easily catch them. So for me, this is definitely a real (albeit one I can usually ignore) problem.

I'm not sure I understand the desire to catch every possible exception right where it occurs. I've never felt this need somehow. *Apart from open()* can someone give me an example from real code where a context manager is likely to raise an exception that can meaningfully be handled? I've re-read Serhiy's original example, involving a context manager for connecting to a socket and a for-loop for reading the data from the socket and writing it to another one (all three operations that may fail), but I find it unconvincing. For reference I'm copying it here again: with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail This very much looks like toy networking code to me -- real networking code is always written in a completely different way, using different abstractions that raise different exceptions, and I don't think it would be written in this style even if `with` and `for` had optional `except` clauses. (Alternatively, you can write little wrappers that turn OSError into different exceptions so you can use a single try/except/except/except statement to handle them all.) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On 30/07/2019 23:37:38, Greg Ewing wrote:

Guido van Rossum wrote:

...

I'm not sure I understand the desire to catch every possible exception right where it occurs. I've never felt this need somehow.

This is my feeling too. I can't remember having a need for such needle-sharp targeting of exception catching, and if I ever did, it would be such a rare thing that I wouldn't mind writing things in a different way to achieve it.

With respect, perhaps you should consider it (for appropriate applications).  When troubleshooting (often an activity conducted under more pressure than coding) it helps to have as much information as possible at your fingertips.  Oh, I know you can always work through a trace (if one is available) to see exactly where an error occurred, but that takes time.

...

On Jul 30, 2019, at 11:38 AM, Guido van Rossum wrote:

..

...

with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail

This very much looks like toy networking code to me

Because Serhiy presented his idea in just about the simplest, clearest

On 31/07/2019 00:35:59, Eric V. Smith wrote: possible example, rather than something more complicated which a lot of people might not bother to read. (Don't get me wrong - I'm not definitely convinced this proposal should be implemented.  I just don't want its merits to be overlooked.)

Something I don't like about these kinds of proposals is that the except clause is far removed from the code that it covers, hurting readability. By the time you get to the end of a big for-loop or with-statement and see an "except", it's easy to forget that it isn't attached to an ordinary try-statement. I would be happier if there were some way to get the except clause at the top instead of the bottom, but it's hard to think of a nice way to do that. -- Greg

On 30/07/2019 02:35, Steven D'Aprano wrote:

On Mon, Jul 29, 2019 at 03:12:21PM +0100, Rhodri James wrote:

...

I'm afraid I agree with Guido. I don't imagine I will use this feature very often, and I can see myself making that mistake every time.

Well, if you don't use this feature very often, you aren't really the audience for the feature and your feedback should therefore be weighted lower *wink*

I'll admit I was thinking much the same ;-)

I don't have a strong option one way or another on this feature, but I think we should resist the trap of thinking that every feature must be immediately and intuitively obvious to every reader.

I would actually like this feature, my problem is that it is immediately and obviously *misleading*. It needs tagging somehow to stop you (well, me) immediately thinking "Oh, this about exceptions in the body of the with/for." You suggested elsewhere giving the exception a source field, but that might be impossibly fine grained. I've been trying to come up with something more like this: with something_exceptionable() as f: do_something_with(f) except with SomeException as e: handle_exception_in_setup_or_teardown(e) except SomeOtherException as e: handle_exception_in_body(e) # Because I like this syntactic sugar too -- Rhodri James *-* Kynesim Ltd