for ... except, with ... except
Python allows you to write code in tight and readable form. Consider the following example. with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail The problem is that different lines can raise an exception of the same type (for example OSError). We want to catch and handle exceptions raised when open a connection, when read a data and when write a data in different ways. Currently you need to expand so convenient Python statements "with" and "for" (see PEP 343) _mgr = connect() _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True try: stream = _value _it = iter(stream) while True: try: data = next(_it) except StopIteration: break write(data) except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None) and then add "try ... except" around corresponding explicit calls of `__enter__()` and `next()`. try: _mgr = connect() _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True except OSError: handle_connection_error() else: try: stream = _value try: _it = iter(stream) except OSError: handle_read_error() else: while True: try: data = next(_it) except StopIteration: break except OSError: handle_read_error() break try: write(data) except OSError: handle_write_error() except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None) Does not it look ugly? I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except". for VAR in EXPR: BLOCK except EXC: HANDLER should be equivalent to try: _it = iter(EXPR) except EXC: HANDLER else: while True: try: VAR = next(_it) except StopIteration: break except EXC: HANDLER break BLOCK and with EXPR as VAR: BLOCK except EXC: HANDLER try: _mgr = EXPR _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True except EXC: HANDLER else: try: VAR = _value BLOCK except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None) And correspondingly for asynchronous versions "async for" and "async with". So you will be able to add errors handling like in: with connect() as stream: for data in stream: try: write(data) except OSError: handle_write_error() except OSError: handle_read_error() except OSError: handle_connection_error()
On Jul 26, 2019, at 11:26, Serhiy Storchaka <storchaka@gmail.com> wrote:
I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".
for VAR in EXPR: BLOCK except EXC: HANDLER
I’m pretty sure for…except has been proposed in the past, and people didn’t think it was common enough or the workaround ugly enough. But with…except, that really is ugly to workaround. And very easy to get wrong, too. Usually, I think people either just scrap the nice context managers and use finally, or add two more layers of indentation, or, most commonly, just punt on the problem and don’t handle errors correctly. So, I think adding with to the idea makes it a lot more compelling.
On 2019-07-26 19:26, Serhiy Storchaka wrote: [snip]
I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".
for VAR in EXPR: BLOCK except EXC: HANDLER
should be equivalent to
try: _it = iter(EXPR) except EXC: HANDLER else: while True: try: VAR = next(_it) except StopIteration: break except EXC: HANDLER break BLOCK
[snip] 1. The 'for' loop can have an 'else' clause, and so can the 'try' statement. Is there any ambiguity over its meaning? 2. In your example you have it catching the exception if EXPR or iter(EXPR) raises. Is that a good idea?
These are interesting ideas. It looks like you intend the except clause of the for loop to *only* cover the iter() and next() calls that are implicit in the for loop. You're right that it's awkward to catch exceptions there. However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-) On Fri, Jul 26, 2019 at 11:26 AM Serhiy Storchaka <storchaka@gmail.com> wrote:
Python allows you to write code in tight and readable form. Consider the following example.
with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail
The problem is that different lines can raise an exception of the same type (for example OSError). We want to catch and handle exceptions raised when open a connection, when read a data and when write a data in different ways. Currently you need to expand so convenient Python statements "with" and "for" (see PEP 343)
_mgr = connect() _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True try: stream = _value _it = iter(stream) while True: try: data = next(_it) except StopIteration: break write(data) except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None)
and then add "try ... except" around corresponding explicit calls of `__enter__()` and `next()`.
try: _mgr = connect() _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True except OSError: handle_connection_error() else: try: stream = _value try: _it = iter(stream) except OSError: handle_read_error() else: while True: try: data = next(_it) except StopIteration: break except OSError: handle_read_error() break try: write(data) except OSError: handle_write_error() except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None)
Does not it look ugly?
I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".
for VAR in EXPR: BLOCK except EXC: HANDLER
should be equivalent to
try: _it = iter(EXPR) except EXC: HANDLER else: while True: try: VAR = next(_it) except StopIteration: break except EXC: HANDLER break BLOCK
and
with EXPR as VAR: BLOCK except EXC: HANDLER
try: _mgr = EXPR _enter = type(_mgr).__enter__ _exit = type(_mgr).__exit__ _value = _enter(_mgr) _exc = True except EXC: HANDLER else: try: VAR = _value BLOCK except: _exc = False if not _exit(_mgr, *sys.exc_info()): raise finally: if _exc: _exit(_mgr, None, None, None)
And correspondingly for asynchronous versions "async for" and "async with".
So you will be able to add errors handling like in:
with connect() as stream: for data in stream: try: write(data) except OSError: handle_write_error() except OSError: handle_read_error() except OSError: handle_connection_error() _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/M76F34... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
26.07.19 23:46, MRAB пише:
On 2019-07-26 19:26, Serhiy Storchaka wrote: [snip]
I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".
for VAR in EXPR: BLOCK except EXC: HANDLER
should be equivalent to
try: _it = iter(EXPR) except EXC: HANDLER else: while True: try: VAR = next(_it) except StopIteration: break except EXC: HANDLER break BLOCK
[snip] 1. The 'for' loop can have an 'else' clause, and so can the 'try' statement. Is there any ambiguity over its meaning?
An 'else' clause is already have different meanings in different statements: 'if', 'try' and 'while'/'for'. This proposition does not add anything new to this. If an exception is not raised in 'for' or 'with' then its body is executed. So no need to add an 'else' clause in the meaning of 'try'. There will be no conflict between two 'else'.
2. In your example you have it catching the exception if EXPR or iter(EXPR) raises. Is that a good idea?
Definitely we should catch the exception if EXPR is raised in the 'with' statement, because different content managers can acquire resources either in `__enter__` or in constructor (like open()). There is less confidence about 'for', but I think that it is better to catch it, because iter() is called implicitly and there is no way to catch an exception in it. iter() can raise an exception: >>> f = open('python') >>> f.close() >>> iter(f) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: I/O operation on closed file. But all this is discussable.
26.07.19 23:57, Guido van Rossum пише:
These are interesting ideas. It looks like you intend the except clause of the for loop to *only* cover the iter() and next() calls that are implicit in the for loop. You're right that it's awkward to catch exceptions there.
Right. If you want to catch an exception in the loop body you have to wrap the loop body with "try ... except". If you want to catch an exception in the loop body *and* the 'for' clause you have to wrap the whole "for" statement with "try ... except". No new syntax is needed for this. But if you want to catch an exception *only* in the 'for' clause you can't (without getting rid of the "for" statement at all).
However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)
I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.
On 27 Jul 2019, at 10:19, Serhiy Storchaka <storchaka@gmail.com> wrote:
26.07.19 23:57, Guido van Rossum пише:
These are interesting ideas. It looks like you intend the except clause of the for loop to *only* cover the iter() and next() calls that are implicit in the for loop. You're right that it's awkward to catch exceptions there.
Right. If you want to catch an exception in the loop body you have to wrap the loop body with "try ... except". If you want to catch an exception in the loop body *and* the 'for' clause you have to wrap the whole "for" statement with "try ... except". No new syntax is needed for this. But if you want to catch an exception *only* in the 'for' clause you can't (without getting rid of the "for" statement at all).
However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)
I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.
I don't think that's correct. I know many people who have learned what for-else does and promptly forgot it because it doesn't make sense to them. I'm fairly certain I know what it does after having this discussion many many times but I'm not 100% sure. If something reads one way and is in reality something else it's super hard to not read it as the plain meaning. In the for-else case it could have been "for.. on not break" (and "on empty", "on break", and such) and it would be very nice. I'm supportive of the proposal but not the syntax. / Anders
On 07/27/2019 01:19 AM, Serhiy Storchaka wrote:
26.07.19 23:57, Guido van Rossum пише:
However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)
And it's what I thought when I saw the psuedo-code.
I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.
Sorry, but that mistake will be made many, many times by the same people. It's taken me several years to sort out `for...else`, and even now I have to think "it's a search loop, adapt your algorithm". Another classic example is allowing single-"=" assignment in tests: if some_var = 7: ... Sure, folks /know/ what it means, but it's a common bug because it doesn't read as "if some_var is assigned 7" but as "if some_var is equal to 7". -- ~Ethan~
On 27/07/2019 09:19, Serhiy Storchaka wrote:
26.07.19 23:57, Guido van Rossum пише:
These are interesting ideas. It looks like you intend the except clause of the for loop to *only* cover the iter() and next() calls that are implicit in the for loop. You're right that it's awkward to catch exceptions there.
Right. If you want to catch an exception in the loop body you have to wrap the loop body with "try ... except". If you want to catch an exception in the loop body *and* the 'for' clause you have to wrap the whole "for" statement with "try ... except". No new syntax is needed for this. But if you want to catch an exception *only* in the 'for' clause you can't (without getting rid of the "for" statement at all).
However, I worry that when people see this syntax, they will think that the except clause is for handling exceptions in the loop body. (That's certainly what I assumed when I read just your subject line. :-)
I think people can make a mistake only when they see it for the first time. After you learn what this syntax means, you will not make this mistake the second time. The same applies to any other syntactic construction.
I'm afraid I agree with Guido. I don't imagine I will use this feature very often, and I can see myself making that mistake every time. -- Rhodri James *-* Kynesim Ltd
I think the focus shouldn't be on whether such syntax is possibly confusing or not because a) People will always remember that's *either* one way or the other, but are very unlikely to just assume one; hence they can always check what it does, and more importantly, b) It's actually pretty easy to remember which way it is, by just considering that a syntax feature exists for scenarios that can't be easily solved otherwise. For `for ... else` it's actually more tricky than for the proposed syntax because that doesn't let you distinguish between "`else` is only executed if the loop didn't `break`" and "`else` is only executed if the loop didn't execute at all". But for `for ... except` or `while ... except` it will be even more obvious because `try ... except`ing the body is just as easy as, well, `try ... except`ing the body; it's pretty clear that no new syntax would be required for that and hence it must concern the statement itself since that's not easy to work around. The only possible confusion I see is when people look at something like this: for ...: LENGTHY BODY except ...: HANDLER else: SOMETHING_ELSE and only look at the `except ... else` part and readily assume that the two are complementary. But that's easy to prevent by only allowing the opposite order, i.e. `for ... else ... except`, stressing that the two are unrelated in this case. In the end a feature should be driven by its usefulness to the community and whether it provides a solution for an otherwise (hard|awkward)-to-solve problem. I could imagine that due to the awkward workaround, especially regarding `with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`). Someone who encounters that feature for the first time, e.g. when reviewing some code, will probably check what it does and even if not, the assumption that it `except`s the whole body should be perplexing since (a) excepting whole blocks by default is not really best practice and (b) it's pretty easy to accomplish that with existing syntax.
On 07/29/2019 12:24 PM, Dominik Vilsmeier wrote:
I think the focus shouldn't be on whether such syntax is possibly confusing or not because
a) People will always remember that's *either* one way or the other
No, they won't.
b) It's actually pretty easy to remember which way it is, by just considering that a syntax feature exists for scenarios that can't be easily solved otherwise.
They also exist for scenarios that are easily resolved, but also easily gotten wrong: x += 1 instead of x = x + 1 I can easily see for this in some_iter: ... except ValueError(): .. as being syntactic sugar for for this in some_iter: try: ... except ValueError(): ... as it can easily save several levels of indenting. -- ~Ethan~
Sorry, that does not convince me. You're assuming that everybody is a language designer. Many Python users actually have little language design sense, and you shouldn't need to have it in order to be able to use the language. People are productive by learning to recognize and copy patterns, but their pattern recognition isn't guided by what you can reason out from thinking about how the implementation works. An issue with except clauses in particular is that during normal execution they don't get executed, and hence they are often not tested carefully. (Witness the evergreen bug of having a typo in your exception logging code that takes down production.) In the case of the proposed except clause this will probably mean that people will add except clauses to for-loops because they've read somewhere "for-loops can now have an except clause" and never read the rest of the description, and then they'll add a non-functional except-clause to a for-loop, thinking they've solved a certain potential problem. So now future maintainers have *two* problems: the exception is not caught when it was meant to be caught, and there is a mysterious except-clause on the for-loop that nobody knows for sure why it was added. I don't think there's a solution that solves the specific issue without confusing most people. Users of for-loops should probably do one of three things: 1. Don't worry about exceptions in the iter() and next() calls. 2. Put a try/except around the entire for-loop and don't worry about whether the exception came from an iter() or next() call or from the loop body. 3. Rewrite things without a for-loop, like Serhiy showed in his first message. This is ugly but the need should be very rare. On Mon, Jul 29, 2019 at 12:26 PM Dominik Vilsmeier <dominik.vilsmeier@gmx.de> wrote:
I think the focus shouldn't be on whether such syntax is possibly confusing or not because
a) People will always remember that's *either* one way or the other, but are very unlikely to just assume one; hence they can always check what it does, and more importantly, b) It's actually pretty easy to remember which way it is, by just considering that a syntax feature exists for scenarios that can't be easily solved otherwise. For `for ... else` it's actually more tricky than for the proposed syntax because that doesn't let you distinguish between "`else` is only executed if the loop didn't `break`" and "`else` is only executed if the loop didn't execute at all". But for `for ... except` or `while ... except` it will be even more obvious because `try ... except`ing the body is just as easy as, well, `try ... except`ing the body; it's pretty clear that no new syntax would be required for that and hence it must concern the statement itself since that's not easy to work around.
The only possible confusion I see is when people look at something like this:
for ...: LENGTHY BODY except ...: HANDLER else: SOMETHING_ELSE
and only look at the `except ... else` part and readily assume that the two are complementary. But that's easy to prevent by only allowing the opposite order, i.e. `for ... else ... except`, stressing that the two are unrelated in this case.
In the end a feature should be driven by its usefulness to the community and whether it provides a solution for an otherwise (hard|awkward)-to-solve problem. I could imagine that due to the awkward workaround, especially regarding `with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`). Someone who encounters that feature for the first time, e.g. when reviewing some code, will probably check what it does and even if not, the assumption that it `except`s the whole body should be perplexing since (a) excepting whole blocks by default is not really best practice and (b) it's pretty easy to accomplish that with existing syntax. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JIHFJT... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
On Jul 29, 2019, at 12:24, Dominik Vilsmeier <dominik.vilsmeier@gmx.de> wrote:
I could imagine that due to the awkward workaround, especially regarding `with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`).
I think it’s a problem if this feature is accepted at a level more or less akin for for…else. The problem that for…else solves is a bit clunkier to solve without it, but not that bad—novices can and do figure out how to write correct code without else. So, while it’s a shame so many people never learn to use it, it’s not a huge deal. The problem that with…except solves is so much harder to solve without it that even experienced developers usually just punt and try…except the whole block around the with. That’s a lot worse. And this isn’t a rare thing; it comes up all the time in all kinds of code. So it really calls for a syntax that most people will learn to use, not something like for…else that some experts use but most people live without. (But I don’t have a better syntax to offer. I actually like with…except, if it were just for my use, but on reflection I can see why others find it confusing. Just like for…else.)
On Mon, Jul 29, 2019 at 2:36 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:
On Jul 29, 2019, at 12:24, Dominik Vilsmeier <dominik.vilsmeier@gmx.de> wrote:
I could imagine that due to the awkward workaround, especially regarding
`with`, corresponding "self-made" code is either error-prone or people will not even try to work around it in the first place. This feature will probably be among the less prominent ones, but someone who needs it will be glad that it exists and they're also likely to be well aware of what it does (just like with `for ... else`).
I think it’s a problem if this feature is accepted at a level more or less akin for for…else.
The problem that for…else solves is a bit clunkier to solve without it, but not that bad—novices can and do figure out how to write correct code without else. So, while it’s a shame so many people never learn to use it, it’s not a huge deal.
The problem that with…except solves is so much harder to solve without it that even experienced developers usually just punt and try…except the whole block around the with. That’s a lot worse. And this isn’t a rare thing; it comes up all the time in all kinds of code. So it really calls for a syntax that most people will learn to use, not something like for…else that some experts use but most people live without.
I am *guessing* the problem here is something like this: with open(filename) as f: data = f.read() raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before). Still I wouldn't call this "a lot worse". What makes you say that?
(But I don’t have a better syntax to offer. I actually like with…except, if it were just for my use, but on reflection I can see why others find it confusing. Just like for…else.)
In hindsight I probably added for..else prematurely; it would never be accepted now. But at this point there would be a cost at removing it as well and I don't think that's worth considering. But if the bar is about as high for with...except as it would be if for...else was proposed today, then I think it's clear we should not add with...except (nor for...except, whose use case is much weaker IMO). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
On Mon, Jul 29, 2019 at 2:58 PM Guido van Rossum <guido@python.org> wrote:
I am *guessing* the problem here is something like this:
with open(filename) as f: data = f.read()
raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before).
Isn't the appropriate solution to that one as follows? It's pretty clear what you're trying to do, so the extra couple of lines aren't all that obtrusive. try: f = open('x', 'r') except IOError as whatever: handle whatever else: with f: data = f.read()
On Mon, Jul 29, 2019 at 3:51 PM Eric Fahlgren <ericfahlgren@gmail.com> wrote:
On Mon, Jul 29, 2019 at 2:58 PM Guido van Rossum <guido@python.org> wrote:
I am *guessing* the problem here is something like this:
with open(filename) as f: data = f.read()
raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before).
Isn't the appropriate solution to that one as follows? It's pretty clear what you're trying to do, so the extra couple of lines aren't all that obtrusive.
try: f = open('x', 'r') except IOError as whatever: handle whatever else: with f: data = f.read()
It's quite verbose though. And it makes you think "oh, I don't need `with`, I can use `finally: f.close()`" -- until the IOError hits and you get `UnboundLocalError: f`... -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
On Mon, Jul 29, 2019 at 3:51 PM Eric Fahlgren <ericfahlgren@gmail.com <mailto:ericfahlgren@gmail.com>> wrote:
On Mon, Jul 29, 2019 at 2:58 PM Guido van Rossum <guido@python.org <mailto:guido@python.org>> wrote:
I am *guessing* the problem here is something like this:
with open(filename) as f: data = f.read()
raises an exception if the open() call fails, but putting try... except IOError: ... around the whole thing also catches I/O errors in the read() call (or anything else you might put in the body). Other solutions are more verbose and run other risks. For other types of context managers it may be worse because the error you want to catch occurs in the __enter__() call that's implicit in the with-clause (for open(), __enter__() is just "return self" since all the work happens before).
Isn't the appropriate solution to that one as follows? It's pretty clear what you're trying to do, so the extra couple of lines aren't all that obtrusive.
try: f = open('x', 'r') except IOError as whatever: handle whatever else: with f: data = f.read()
It's quite verbose though. And it makes you think "oh, I don't need `with`, I can use `finally: f.close()`" -- until the IOError hits and you get `UnboundLocalError: f`... Hm, I'm not sure why you would be more likely to think that in this example than anywhere else where you use an open() context manager. (Observation: I think once you start using context managers, you appreciate their advantages and are unlikely to remove one from your code without good reason. This is certainly a rite of passage I went
On 30/07/2019 00:07:52, Guido van Rossum wrote: through with open().) I believe the/a motivation for Serhiy's proposal is to write really robust code, e.g. in a program that must not fail such as a network server serving multiple clients. ISTM this is a worthwhile goal, and at present the only way to achieve it is with somewhat verbose and clunky code.
On Mon, Jul 29, 2019 at 03:12:21PM +0100, Rhodri James wrote:
I'm afraid I agree with Guido. I don't imagine I will use this feature very often, and I can see myself making that mistake every time.
Well, if you don't use this feature very often, you aren't really the audience for the feature and your feedback should therefore be weighted lower *wink* I don't have a strong option one way or another on this feature, but I think we should resist the trap of thinking that every feature must be immediately and intuitively obvious to every reader. There's a hierachy of desirability: 1. syntax that obviously solves a problem; 2. syntax that solves a problem, but it isn't obvious 3. syntax that solves a problem, but may mislead people to think it solves a different problem until they learn better; 4. syntax which doesn't solve the problem; 5. syntax that makes the problem worse. Clearly we prefer 1 or 2 when possible, but 3 ought to be acceptible if the problem being solved is big enough. If this solves a big problem (*if* -- I'm not convinced one way or the other), and there is no better syntax, then we might decide that the cost to people like you is worth the benefit to people like Serhiy. Hypothetically, if Serhiy gets to write more robust, understandable code in fewer lines twice a week, and the cost is that you have to look the syntax up in the documentation once a year, or receive a negative code review from your workmates saying "your for...except block is wrong", so be it. I feel your pain: I don't understand async code when I see it either. But the benefit to those who use it outweighs the confusion it causes people like me. -- Steven
On Fri, Jul 26, 2019 at 09:26:00PM +0300, Serhiy Storchaka wrote:
Python allows you to write code in tight and readable form. Consider the following example.
with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail
The problem is that different lines can raise an exception of the same type (for example OSError).
I'm not sure if that is the best example, because as far as I know, errors in opening the file and errors in writing to the file will raise OSErrors with different error numbers, so you can distinguish them. I think. But your broader point is applicable: different parts of the block may raise the same exception, and we have no good way of telling them apart.
We want to catch and handle exceptions raised when open a connection, when read a data and when write a data in different ways.
Would it help if it were easier to tell the source of an exception? This is just a sketch, not a serious proposal, but something like this? try: with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail except OSError as err: if err.source == "connect": ... elif err.source == "stream.__enter__": ... elif err.source == "__next__": ... elif err.source == "write": ... else: raise
I propose to add "except" clause to "for" and "with" statement to catch exceptions in the code that can't be wrapped with "try ... except".
for VAR in EXPR: BLOCK except EXC: HANDLER
[...snip pure-python equivalent...] That implies that you will always want to treat an exception in EXPR.__iter__ and __next__ the same. What if you need to treat them differently? Your example:
So you will be able to add errors handling like in:
with connect() as stream: for data in stream: try: write(data) except OSError: handle_write_error() except OSError: handle_read_error() except OSError: handle_connection_error()
To be really robust, shouldn't you check the error codes, and only handle the codes you know how to handle? But if you're doing that, it seems to me that you only need one "except OSError" handler. Am I wrong? -- Steven
On Jul 29, 2019, at 14:49, Guido van Rossum <guido@python.org> wrote:
Still I wouldn't call this "a lot worse". What makes you say that?
Not “worse” as in more common, or more dangerous (I don’t know if that’s true or not), but as in harder to work around. for…else being confusing and effectively “experts only” just means most people don’t get to save a line of code and an extra variable—not ideal, but not a huge problem. with…except being confusing and effectively “experts only” means that most people are unable to write correct exception-handling code. So if this kind of code really is a serious problem, then the proposal doesn’t solve it; we need a proposal everyone will be able to learn and use. (And if it’s not a serious problem, then the proposal isn’t needed in the first place.)
On 07/29/2019 06:35 PM, Steven D'Aprano wrote:
3. syntax that solves a problem, but may mislead people to think it solves a different problem until they learn better;
3.1 syntax that solves a problem, but is misleading to many who are still confused after they "have learnt better"
I feel your pain: I don't understand async code when I see it either.
Neither do I, but I have no need to. I do have a need to use for...else, and every time I do I have to think through the "this is a search loop" to get the right semantics (sorry for bringing that up again, but it's a great example). If this "with...except" leads to more robust code then I think many would like to use it, but not if it's confusing as that would lead to less robust code. -- ~Ethan~
On Tue, 30 Jul 2019 at 04:00, Ethan Furman <ethan@stoneleaf.us> wrote:
If this "with...except" leads to more robust code then I think many would like to use it, but not if it's confusing as that would lead to less robust code.
For me, "it's for robust code" is sufficient hint that I now remember what it does (much the same way that "it's for search loops" is the hint I need for for...else). And when I'm trying to write robust code, I *do* worry about exceptions in the with expression, and the fact that I can't easily catch them. So for me, this is definitely a real (albeit one I can usually ignore) problem. The thing is that for most of the code I write, it's not *that* important to be highly robust. For command line interactive utilities, people can just fix unexpected errors and rerun the command. So putting a lot of effort into robustness is not that important. But then someone scripts your utility, and suddenly tracebacks are a much bigger issue. That's why the with statement is so useful - it makes robust exception handling low-cost to write, so my code moves a step closer to robust-by-default. This proposal takes that a step further, by lowering the cost of a tricky bit of boilerplate. So I don't think it's essential, but I *do* think it would be a useful addition to the language. On the other hand, for...except has much less appeal, as I don't think of for loops as blocks of code where I'd expect to be able to control what exceptions escape (whereas in my mind that's a primary feature of the with statement). Paul
On 30/07/2019 02:35, Steven D'Aprano wrote:
On Mon, Jul 29, 2019 at 03:12:21PM +0100, Rhodri James wrote:
I'm afraid I agree with Guido. I don't imagine I will use this feature very often, and I can see myself making that mistake every time.
Well, if you don't use this feature very often, you aren't really the audience for the feature and your feedback should therefore be weighted lower *wink*
I'll admit I was thinking much the same ;-)
I don't have a strong option one way or another on this feature, but I think we should resist the trap of thinking that every feature must be immediately and intuitively obvious to every reader.
I would actually like this feature, my problem is that it is immediately and obviously *misleading*. It needs tagging somehow to stop you (well, me) immediately thinking "Oh, this about exceptions in the body of the with/for." You suggested elsewhere giving the exception a source field, but that might be impossibly fine grained. I've been trying to come up with something more like this: with something_exceptionable() as f: do_something_with(f) except with SomeException as e: handle_exception_in_setup_or_teardown(e) except SomeOtherException as e: handle_exception_in_body(e) # Because I like this syntactic sugar too -- Rhodri James *-* Kynesim Ltd
On Tue, Jul 30, 2019 at 9:32 AM Rhodri James <rhodri@kynesim.co.uk> wrote:
I've been trying to come up with something more like this:
with something_exceptionable() as f: do_something_with(f) except with SomeException as e: handle_exception_in_setup_or_teardown(e) except SomeOtherException as e: handle_exception_in_body(e) # Because I like this syntactic sugar too
I had to read that three times before I noticed the "with" after the "except". Perhaps PEP 463 [1] could be resurrected on a limited basis, allowing something like this: with something_exceptionable() as f except SomeException -> handle_setup_exception(): do_something_with(f) and similar for the for statement. This can result in lengthy lines, but it's going to be a bit awkward no matter what, and I feel a long line is the simplest and most readable way to get past that awkwardness. Plus, line continuations are a thing, which can make it more readable. [1] https://www.python.org/dev/peps/pep-0463/
On Tue, Jul 30, 2019 at 1:13 AM Paul Moore <p.f.moore@gmail.com> wrote:
On Tue, 30 Jul 2019 at 04:00, Ethan Furman <ethan@stoneleaf.us> wrote:
If this "with...except" leads to more robust code then I think many would like to use it, but not if it's confusing as that would lead to less robust code.
For me, "it's for robust code" is sufficient hint that I now remember what it does (much the same way that "it's for search loops" is the hint I need for for...else). And when I'm trying to write robust code, I *do* worry about exceptions in the with expression, and the fact that I can't easily catch them. So for me, this is definitely a real (albeit one I can usually ignore) problem.
I'm not sure I understand the desire to catch every possible exception right where it occurs. I've never felt this need somehow. *Apart from open()* can someone give me an example from real code where a context manager is likely to raise an exception that can meaningfully be handled? I've re-read Serhiy's original example, involving a context manager for connecting to a socket and a for-loop for reading the data from the socket and writing it to another one (all three operations that may fail), but I find it unconvincing. For reference I'm copying it here again: with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail This very much looks like toy networking code to me -- real networking code is always written in a completely different way, using different abstractions that raise different exceptions, and I don't think it would be written in this style even if `with` and `for` had optional `except` clauses. (Alternatively, you can write little wrappers that turn OSError into different exceptions so you can use a single try/except/except/except statement to handle them all.) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Something I don't like about these kinds of proposals is that the except clause is far removed from the code that it covers, hurting readability. By the time you get to the end of a big for-loop or with-statement and see an "except", it's easy to forget that it isn't attached to an ordinary try-statement. I would be happier if there were some way to get the except clause at the top instead of the bottom, but it's hard to think of a nice way to do that. -- Greg
Guido van Rossum wrote:
I'm not sure I understand the desire to catch every possible exception right where it occurs. I've never felt this need somehow.
This is my feeling too. I can't remember having a need for such needle-sharp targeting of exception catching, and if I ever did, it would be such a rare thing that I wouldn't mind writing things in a different way to achieve it. -- Greg
On Jul 30, 2019, at 11:38 AM, Guido van Rossum <guido@python.org> wrote:
...
with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail
This very much looks like toy networking code to me -- real networking code is always written in a completely different way, using different abstractions that raise different exceptions, and I don't think it would be written in this style even if `with` and `for` had optional `except` clauses. (Alternatively, you can write little wrappers that turn OSError into different exceptions so you can use a single try/except/except/except statement to handle them all.)
This is what I do: wrappers with custom exceptions. I think it gives the most readable code. But I definitely do not need this pattern very often. Eric
On 30/07/2019 23:37:38, Greg Ewing wrote:
Guido van Rossum wrote:
I'm not sure I understand the desire to catch every possible exception right where it occurs. I've never felt this need somehow.
This is my feeling too. I can't remember having a need for such needle-sharp targeting of exception catching, and if I ever did, it would be such a rare thing that I wouldn't mind writing things in a different way to achieve it.
With respect, perhaps you should consider it (for appropriate applications). When troubleshooting (often an activity conducted under more pressure than coding) it helps to have as much information as possible at your fingertips. Oh, I know you can always work through a trace (if one is available) to see exactly where an error occurred, but that takes time.
On Jul 30, 2019, at 11:38 AM, Guido van Rossum <guido@python.org> wrote:
..
with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail
This very much looks like toy networking code to me
Because Serhiy presented his idea in just about the simplest, clearest
On 31/07/2019 00:35:59, Eric V. Smith wrote: possible example, rather than something more complicated which a lot of people might not bother to read. (Don't get me wrong - I'm not definitely convinced this proposal should be implemented. I just don't want its merits to be overlooked.)
On Tue, Jul 30, 2019 at 6:04 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Something I don't like about these kinds of proposals is that the except clause is far removed from the code that it covers, hurting readability. By the time you get to the end of a big for-loop or with-statement and see an "except", it's easy to forget that it isn't attached to an ordinary try-statement.
I would be happier if there were some way to get the except clause at the top instead of the bottom, but it's hard to think of a nice way to do that.
try with open('some_file') as f: except FileNotFoundError: do_something() else: do_something_else(f) was the best I could come up with, and I hate the `except` being indented (but I think a non-indented line right after a : is probably a non-starter). I do think even a clumsy solution is vaguely better than "just make your try: block encompass both the open() and then everything you do with the file after that".
On Wed, Jul 31, 2019 at 2:06 AM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:
On Jul 30, 2019, at 11:38 AM, Guido van Rossum <guido@python.org> wrote:
..
with connect() as stream: # connect() or __enter__() can fail. for data in stream: # __next__() can fail write(data) # write() can fail
This very much looks like toy networking code to me
Because Serhiy presented his idea in just about the simplest, clearest
On 31/07/2019 00:35:59, Eric V. Smith wrote: possible example, rather than something more complicated which a lot of people might not bother to read. (Don't get me wrong - I'm not definitely convinced this proposal should be implemented. I just don't want its merits to be overlooked.)
Serhiy presented the *technical* problem very clearly indeed. But comparing this toy example to the expanded version and saying "look how bad it is, this must be solved with new syntax" is attacking a strawman. We're now stuck with a proposed syntax change that is likely to cause confusion and broken code, and we have to judge how bad the original problem really is, and if there aren't other work-arounds for it. Searches for other syntactic solutions have been unsuccessful. The approach of defining separate exception types appears to be the best solution that doesn't require new syntax, so let's consider this solved. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
participants (16)
-
Anders Hovmöller -
Andrew Barnert -
Dominik Vilsmeier -
Eric Fahlgren -
Eric V. Smith -
Ethan Furman -
Geoffrey Spear -
Greg Ewing -
Guido van Rossum -
Jonathan Goble -
MRAB -
Paul Moore -
Rhodri James -
Rob Cliffe -
Serhiy Storchaka -
Steven D'Aprano