Hello, A common pattern for me is to write a with statement for resource cleanup, but also handle specific errors after that. Right now, this is a bit cumbersome: try: with open("somefile", "rb)" as f: ... except FileNotFoundError: # do something else, perhaps actually create the file or: try: with transaction.commit_on_success(): ... except ObjectDoesNotExist: # do something else, perhaps clean up some internal cache How about adding syntax sugar for the above, in the form of a with ... except clause? It would nicely reduce spurious indentation, as with the try / except / finally which, long ago(!), helped reduce indentation and typing by removing the need to nest a try / except inside a try / finally. Regards Antoine.
On 2013-03-08, at 11:13 , Antoine Pitrou wrote:
Isn't it essentially the same suggestion as Alan Johnson's last week? http://mail.python.org/pipermail/python-ideas/2013-March/019730.html
Le Fri, 8 Mar 2013 11:24:02 +0100, Masklinn <masklinn@masklinn.net> a écrit :
Hmm, I hadn't read that thread. "try with" looked sufficiently ugly that I wasn't interested :-) But anyway it seems that discussion was conflating a lot of things. I would only like a shortcut for a "try ... except" around a "with", without any other sophistication. Seems "with" is essentially a "try ... finally", there seems to be a syntactic precedent already. And, yes, I actually want to catch exceptions raised *inside* the "with" block, not just by the "with" statement itself. The database example above makes it clear: I want the "with" to issue a ROLLBACK on an exception inside the block, *and* I want to handle the exception in a specific way after the ROLLBACK. Regards Antoine.
On Friday 08 Mar 2013, Antoine Pitrou wrote:
Ugh, someone is going to suggest we have "try ... without" now... And it's so angocentric. How come the calls are to repeat *English grammar*? German grammar would probably be a lot clearer for a compiler/interpreter. Of course, that wouldn't be possible if we still had the 80-column limit... :-)
Mark Hackett wrote:
And it's so angocentric. How come the calls are to repeat *English grammar*? German grammar would probably be a lot clearer for a compiler/interpreter.
How about Latin? http://www.csse.monash.edu.au/~damian/papers/HTML/Perligata.html -- Greg
From: Greg Ewing <greg.ewing@canterbury.ac.nz>
Come on, what's the point of using inflection to get rid of word order if you're not going to also use it to get rid of punctuation and grammatical function words? For example, if you've got separate instrumental and accusative cases, you don't need "with". And with a distinction between the imperative mood and something else, like jussive, or mood modifiers that let you create something like conditional-imperative, you don't need "try". Thus, the entire issue that started this thread would never come up. In a language without case stacking, there are limits to how far you can take this, but there are plenty of languages that have case stacking—or, better, full polysynthesis. With both pervasive argument incorporation and unbounded compound agglutination, an entire function body can be written as a single word. No more indentation rules to break copy/paste on blog comments, no more limitations to the one-line lambda, … Latin is woefully insufficient. But Chukchi would work.
On Fri, Mar 8, 2013 at 8:13 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
The main problem with this kind of construct is that it makes the scope of the exception handler too broad - it's covering the entire body of the with statement, when you really only want to cover the creation of the file object: try: f = open("somefile", "rb") except FileNotFoundError: # Do something else, perhaps including creating the file else: with f: # This is not covered by the except clause... Generalising this to context manages with non-trivial __enter__ methods is actually one of the intended use cases for contextlib.ExitStack (see http://docs.python.org/dev/library/contextlib#catching-exceptions-from-enter...).
This use case is a bit more reasonable in terms of actually wanting the except clause to cover the whole body of the with statement, but trying to lose the extra indentation level suffers from an ambiguity problem. A full try statement looks like: try: ... except: ... else: ... finally: ... The defined semantics of a with statement already include three of those clauses (try, except, finally). Does the except clause still fire if the with statement suppresses the exception? With the nested form, the answer is clearly yes. With the flattened form, the answer is less obvious. Furthermore, if the with statement allows "except", does it also allow else and finally? If not, why not? It's these issues that make me feel this case is more like requests to merge for + if than it is the past merger of the two forms of try statemet.
The difference there was that the indentation truly was redundant - converting between the two forms literally meant dedenting the inner try/except/else and losing the extra "try:" line. For a long time, the AST didn't even have a merged try/except/finally construct (eventually they *were* merged so that source code could be roundtripped through the AST more reliably). The repeated "try:" was also substantially more irritating than "try:" following a "with:" header. It *is* annoying that composing with statements with explicit exception handling is somewhat clumsy, but I don't think this is the way to fix it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le Fri, 8 Mar 2013 21:43:52 +1000, Nick Coghlan <ncoghlan@gmail.com> a écrit :
No, it doesn't.
It doesn't, simply because I don't need it :-) (but, yes, that would be a reasonable request too)
Yep, I think we will eventually have to propose something to fix that, er, "wart" :-) Regards Antoine.
On 8 March 2013 10:13, Antoine Pitrou <solipsis@pitrou.net> wrote:
In some cases it might be reasonable to make a context manager that handles errors from the original context manager e.g.: import contextlib @contextlib.contextmanager def handle(errorcls, func, *args, **kwargs): try: yield except errorcls: func(*args, **kwargs) with handle(FileNotFoundError, print, 'error'), open('somefile', 'rb') as f: print('No error')
Another possibility is a context manager that handles both things, e.g.: @contextmanager def commit_or_clean(errorcls): try: with transaction.commit_on_success(): yield except errorcls: clean() Oscar
On Mar 8, 2013, at 7:16, Antoine Pitrou <solipsis@pitrou.net> wrote:
I think his point is that if you need any specific case often enough in your app, they will be related, and you can write a single context manager that wraps up your specific case so you can use it. In other words, messy with and try statements can be refactored into functions as needed, very easily. If that's not his point, I apologize--but it was definitely my point when I said the same thing last week in response to the similar try with suggestion.
On Sat, Mar 9, 2013 at 8:53 PM, Stefan Behnel <stefan_ml@behnel.de> wrote:
It's not just with statements that don't play nice with supplementary exception handling - for loops actually have the same problem. In both cases, you can fairly easily put an exception handler around just the expression, or around the entire statement, but you can't quite so easily define custom exception handling for the protocol methods invoked implicitly in the statement header (__enter__ in with statements, __iter__ and __next__ in for loops). While loops technically suffer from it as well, but it's rather rare for a bool() invocation to risk triggering an exception. contextlib.ExitStack at least brings context managers close to on par with iterables - you can use either stack.enter_context(cm) and iter(iterable) to lift just the __enter__ or __iter__ call out into a separate try block. Wrapping an exception handler around next() pretty much requires reverting to a while loop, though. Ultimately, though, this may be an inevitable price we pay for the abstraction - you *do* lose flexibility when you design for the typical case, and so you do eventually have to say "sorry, to handle that more complex case you need to drop back down to the lower level syntax" (try/except/else/finally for with statements, while loops for for loops). An important part of minimising language complexity is actually recognising when that limit has been reached and saying, no, sorry, we want to keep the higher level API simple, so that use case won't be supported, since it can already be handled with the lower level API in those cases where it is needed. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
From: Nick Coghlan <ncoghlan@gmail.com>
Sent: Saturday, March 9, 2013 6:34 AM
I agree completely with your main point. But:
It's worth noting off the bat that you don't actually need this too often, because most iterators can't be resumed after exception anyway. But when you do, it's possible. Of course you need a next-wrapper, but that's no different from needing a with-wrapper or an iter-wrapper. For example, let's say you wanted this fictitious construct: fooed_it = ((try: x.foo() except: continue) for x in it) You can just do this: def skip_exceptions(it): while True: try: yield next(it) except StopIteration: raise except: pass fooed_it = (x.foo() for x in skip_exceptions(it)) And needless to say, you can put in a realistic exception handler instead of just a skip-everything clause. I've actually got code similar to this. I've got a C-library enumerator-type function that can return both fatal and non-fatal errors. The first-level wrapper around this provides a next() that raises on non-fatal errors. Then I've got a wrapper around that which logs and continues for all exceptions but StopIteration and fatal errors.
Ultimately, though, this may be an inevitable price we pay for the
I think it's even better than that. You can use the higher-level abstractions even in more complex cases, as long as you build a somewhat complex wrapper. And this has the same tradeoffs as any refactoring—if you save more complexity in the top-level code than you spend in the helper code, it's worth doing. But the end result is the same: Making the abstractions more flexible makes them more complex, and there's a point at which the benefit (not requiring helpers in as many cases) loses to the cost.
On 2013-03-08, at 11:13 , Antoine Pitrou wrote:
Isn't it essentially the same suggestion as Alan Johnson's last week? http://mail.python.org/pipermail/python-ideas/2013-March/019730.html
Le Fri, 8 Mar 2013 11:24:02 +0100, Masklinn <masklinn@masklinn.net> a écrit :
Hmm, I hadn't read that thread. "try with" looked sufficiently ugly that I wasn't interested :-) But anyway it seems that discussion was conflating a lot of things. I would only like a shortcut for a "try ... except" around a "with", without any other sophistication. Seems "with" is essentially a "try ... finally", there seems to be a syntactic precedent already. And, yes, I actually want to catch exceptions raised *inside* the "with" block, not just by the "with" statement itself. The database example above makes it clear: I want the "with" to issue a ROLLBACK on an exception inside the block, *and* I want to handle the exception in a specific way after the ROLLBACK. Regards Antoine.
On Friday 08 Mar 2013, Antoine Pitrou wrote:
Ugh, someone is going to suggest we have "try ... without" now... And it's so angocentric. How come the calls are to repeat *English grammar*? German grammar would probably be a lot clearer for a compiler/interpreter. Of course, that wouldn't be possible if we still had the 80-column limit... :-)
Mark Hackett wrote:
And it's so angocentric. How come the calls are to repeat *English grammar*? German grammar would probably be a lot clearer for a compiler/interpreter.
How about Latin? http://www.csse.monash.edu.au/~damian/papers/HTML/Perligata.html -- Greg
From: Greg Ewing <greg.ewing@canterbury.ac.nz>
Come on, what's the point of using inflection to get rid of word order if you're not going to also use it to get rid of punctuation and grammatical function words? For example, if you've got separate instrumental and accusative cases, you don't need "with". And with a distinction between the imperative mood and something else, like jussive, or mood modifiers that let you create something like conditional-imperative, you don't need "try". Thus, the entire issue that started this thread would never come up. In a language without case stacking, there are limits to how far you can take this, but there are plenty of languages that have case stacking—or, better, full polysynthesis. With both pervasive argument incorporation and unbounded compound agglutination, an entire function body can be written as a single word. No more indentation rules to break copy/paste on blog comments, no more limitations to the one-line lambda, … Latin is woefully insufficient. But Chukchi would work.
On Fri, Mar 8, 2013 at 8:13 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
The main problem with this kind of construct is that it makes the scope of the exception handler too broad - it's covering the entire body of the with statement, when you really only want to cover the creation of the file object: try: f = open("somefile", "rb") except FileNotFoundError: # Do something else, perhaps including creating the file else: with f: # This is not covered by the except clause... Generalising this to context manages with non-trivial __enter__ methods is actually one of the intended use cases for contextlib.ExitStack (see http://docs.python.org/dev/library/contextlib#catching-exceptions-from-enter...).
This use case is a bit more reasonable in terms of actually wanting the except clause to cover the whole body of the with statement, but trying to lose the extra indentation level suffers from an ambiguity problem. A full try statement looks like: try: ... except: ... else: ... finally: ... The defined semantics of a with statement already include three of those clauses (try, except, finally). Does the except clause still fire if the with statement suppresses the exception? With the nested form, the answer is clearly yes. With the flattened form, the answer is less obvious. Furthermore, if the with statement allows "except", does it also allow else and finally? If not, why not? It's these issues that make me feel this case is more like requests to merge for + if than it is the past merger of the two forms of try statemet.
The difference there was that the indentation truly was redundant - converting between the two forms literally meant dedenting the inner try/except/else and losing the extra "try:" line. For a long time, the AST didn't even have a merged try/except/finally construct (eventually they *were* merged so that source code could be roundtripped through the AST more reliably). The repeated "try:" was also substantially more irritating than "try:" following a "with:" header. It *is* annoying that composing with statements with explicit exception handling is somewhat clumsy, but I don't think this is the way to fix it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le Fri, 8 Mar 2013 21:43:52 +1000, Nick Coghlan <ncoghlan@gmail.com> a écrit :
No, it doesn't.
It doesn't, simply because I don't need it :-) (but, yes, that would be a reasonable request too)
Yep, I think we will eventually have to propose something to fix that, er, "wart" :-) Regards Antoine.
On 8 March 2013 10:13, Antoine Pitrou <solipsis@pitrou.net> wrote:
In some cases it might be reasonable to make a context manager that handles errors from the original context manager e.g.: import contextlib @contextlib.contextmanager def handle(errorcls, func, *args, **kwargs): try: yield except errorcls: func(*args, **kwargs) with handle(FileNotFoundError, print, 'error'), open('somefile', 'rb') as f: print('No error')
Another possibility is a context manager that handles both things, e.g.: @contextmanager def commit_or_clean(errorcls): try: with transaction.commit_on_success(): yield except errorcls: clean() Oscar
On Mar 8, 2013, at 7:16, Antoine Pitrou <solipsis@pitrou.net> wrote:
I think his point is that if you need any specific case often enough in your app, they will be related, and you can write a single context manager that wraps up your specific case so you can use it. In other words, messy with and try statements can be refactored into functions as needed, very easily. If that's not his point, I apologize--but it was definitely my point when I said the same thing last week in response to the similar try with suggestion.
On Sat, Mar 9, 2013 at 8:53 PM, Stefan Behnel <stefan_ml@behnel.de> wrote:
It's not just with statements that don't play nice with supplementary exception handling - for loops actually have the same problem. In both cases, you can fairly easily put an exception handler around just the expression, or around the entire statement, but you can't quite so easily define custom exception handling for the protocol methods invoked implicitly in the statement header (__enter__ in with statements, __iter__ and __next__ in for loops). While loops technically suffer from it as well, but it's rather rare for a bool() invocation to risk triggering an exception. contextlib.ExitStack at least brings context managers close to on par with iterables - you can use either stack.enter_context(cm) and iter(iterable) to lift just the __enter__ or __iter__ call out into a separate try block. Wrapping an exception handler around next() pretty much requires reverting to a while loop, though. Ultimately, though, this may be an inevitable price we pay for the abstraction - you *do* lose flexibility when you design for the typical case, and so you do eventually have to say "sorry, to handle that more complex case you need to drop back down to the lower level syntax" (try/except/else/finally for with statements, while loops for for loops). An important part of minimising language complexity is actually recognising when that limit has been reached and saying, no, sorry, we want to keep the higher level API simple, so that use case won't be supported, since it can already be handled with the lower level API in those cases where it is needed. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
From: Nick Coghlan <ncoghlan@gmail.com>
Sent: Saturday, March 9, 2013 6:34 AM
I agree completely with your main point. But:
It's worth noting off the bat that you don't actually need this too often, because most iterators can't be resumed after exception anyway. But when you do, it's possible. Of course you need a next-wrapper, but that's no different from needing a with-wrapper or an iter-wrapper. For example, let's say you wanted this fictitious construct: fooed_it = ((try: x.foo() except: continue) for x in it) You can just do this: def skip_exceptions(it): while True: try: yield next(it) except StopIteration: raise except: pass fooed_it = (x.foo() for x in skip_exceptions(it)) And needless to say, you can put in a realistic exception handler instead of just a skip-everything clause. I've actually got code similar to this. I've got a C-library enumerator-type function that can return both fatal and non-fatal errors. The first-level wrapper around this provides a next() that raises on non-fatal errors. Then I've got a wrapper around that which logs and continues for all exceptions but StopIteration and fatal errors.
Ultimately, though, this may be an inevitable price we pay for the
I think it's even better than that. You can use the higher-level abstractions even in more complex cases, as long as you build a somewhat complex wrapper. And this has the same tradeoffs as any refactoring—if you save more complexity in the top-level code than you spend in the helper code, it's worth doing. But the end result is the same: Making the abstractions more flexible makes them more complex, and there's a point at which the benefit (not requiring helpers in as many cases) loses to the cost.
participants (10)
-
Andrew Barnert -
Antoine Pitrou -
Greg Ewing -
Mark Hackett -
Masklinn -
MRAB -
Nick Coghlan -
Oscar Benjamin -
Stefan Behnel -
Terry Reedy