Extending error handling on with statements.
I personally love using with statements when handling file like objects. This is all well and good until an exception is thrown from the with statement. This is ok if you expect the exception because you can use try and except but personally I feel that another condition to with would feel more 'pythonic' this means that you could fail the with statement with an exception jump to the clause, then jump back to the with statement trying the code in the clause e.g. rather than try: with open('nofile.txt','r') as inp: #nofile.txt does not exist and throws an exception except IOError: with open('another.txt','r') as inp: #carry on where you left off... You could simply have with open('nofile.txt','r') as inp: #exception here else: #give a new file to the with statement here and/or run some panic code where your program does something to fix the situation. It could be a foolish idea as I am only a intermediate user but I thought it might be worth voicing none the less as you don't learn from staying silent.
Jakob Bowyer wrote:
You should always expect an exception when doing file I/O.
You say "jump back to the with statement", and "give a new file to the with statement". It sounds like you are thinking of turning with into a looping construct, e.g. this BASIC-like pseudo-code: 10 somefile = 'nofile.txt') 20 with open(somefile, 'r') as inp: ... 70 else: # Try again with a new file. 80 somefile = 'a different file.txt' 90 goto 20 The obvious problem with this is obvious: if the *second* file also fails to open, you will loop forever as the handler jumps to the `else` clause, sets the same name, and returns to try the with statement again. This will be an annoying source of errors. I don't know if I like this idea: I can see that it can be useful to repeat a block if an error occurs, but I think that it needs to be more obvious that you are looping. You also seem to be assuming that the only error that will be caught will be "file not found" type errors. The beauty of a try-except block is that you can have different handlers depending on the error: somefile = 42 # Oops! try: with open(somefile, 'r') as inp: ... except TypeError: handle_filename_not_a_string() Your suggested `else` clause loses all information about what sort of error occurs, as well as where: outfile = 'output.txt' with open(outfile, 'w') as out: out.write(42) # Oops! else: # Try another file. outfile = 'another file.txt' Lastly, your suggested syntax would be confusing. In try blocks, the `else` clause runs when there is no error. In with blocks, it would run when there is an error. That's not helpful: things that look similar should be similar. Of course, you can fix this problem by changing the with statement to use `except` clauses: with open(fname) as f: ... except TypeError: ... except IOError as e: ... else: # no error ... but this adds much complexity to the with statement, and except for the magic goto, you can already do that at the cost of one line and one indent level: try: with open(fname) as f: ... except TypeError: ... except IOError as e: ... else: # no error ... Saving one indent level and a line doesn't seem important enough for new syntax, especially new syntax which essentially duplicates functionality that already exists. -- Steven
I don't like the idea of that magic goto. But: with ...: ... except ...: ... ... as a shortcut for: try: with ...: ... except ...: ... ... IMHO seems to be worth considering. Regards. *j
On Mon, 28 Mar 2011 01:18:54 +0200 Jan Kaliszewski <zuo@chopin.edu.pl> wrote:
I've played around with this kind of thing in the past, and always eventually decided there wasn't a nice way to handle all the various desirable uses. The with statement has two bits of code, and wraps a try/finally around one of them. You might legitimately want to handle exceptions in either bit of code differently. Which means there are three things you might reasonably want an except clause on a with statement to do: 1) Wrap the entire statement (what Jan proposed). 2) Wrap the block contained by the with. 3) Be part of the try/finally implied by the with. On the other hand, the concept of "magic goto" does generate another idea (and hence another post). <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Mar 28, 2011 at 7:33 AM, Jakob Bowyer <jkbbwr@gmail.com> wrote:
Don't fight the language, just write a new CM that does what you want: with open_any('r', 'nofile.txt', 'another.txt') as inp: # If we get here, one of the files was opened # We can use inp.name to find out which one (And open_any() is pretty easy to write as a generator with an initial loop containing a try/catch block, an else clause on the loop that throws an exception, and then a subsequent with statement that yields the open file) You *really* need to be careful when wrapping try blocks around with statements, as they're almost always too broad (typically, you only want to cover the CM creation, not the entire body of the with statement). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, 27 Mar 2011 22:33:52 +0100 Jakob Bowyer <jkbbwr@gmail.com> wrote:
The idea of exception handlers "jumping back" is actually good enough to have been implemented in one language (eiffel), but sufficiently different from what "except" does that I think it calls for new syntax. How about a "retry" clause for try statements? I think it runs into the same problems as an "except" clause when it comes to adding it to the with clause, so lets skip that for now. retry ...: as part of a try clause would work just like an except clause: if the exception was one of those listed after retry, then you'd enter the block following the retry, otherwise you skip it. If the retry block raises an exception or hits "return" or "yield", it behaves just like an except block. If the retry block executes it's last statement, it then branches back to the first statement of the "try" block. This would let you write something like: i = 0 try: with open("tmpname.%d" % i, 'r') as inp: .... retry IOError: if IOError.errno != ENOENT: raise i += 1 if i > 100: raise To search for a file. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Mar 28, 2011 at 5:15 AM, Mike Meyer <mwm@mired.org> wrote:
I think 'retry' would be clearer if it is used as a command on its own, like return, break or continue, but then only within an except block. Your code above could then go: ... except IOError: if IOError.errno != ENOENT: raise i += 1 if i <= 100: retry else: raise -- André Engels, andreengels@gmail.com
On Mon, Mar 28, 2011 at 1:15 PM, Mike Meyer <mwm@mired.org> wrote:
If you want a loop, write a loop. for fname in possible_fnames: try: f = open(fname) except IOError: continue break else: raise RuntimeError("Could not open any of {}".format(possible_fnames)) with f: # Do stuff Turning the above into a custom "open_any" context manager is trivial. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Jakob Bowyer wrote:
You should always expect an exception when doing file I/O.
You say "jump back to the with statement", and "give a new file to the with statement". It sounds like you are thinking of turning with into a looping construct, e.g. this BASIC-like pseudo-code: 10 somefile = 'nofile.txt') 20 with open(somefile, 'r') as inp: ... 70 else: # Try again with a new file. 80 somefile = 'a different file.txt' 90 goto 20 The obvious problem with this is obvious: if the *second* file also fails to open, you will loop forever as the handler jumps to the `else` clause, sets the same name, and returns to try the with statement again. This will be an annoying source of errors. I don't know if I like this idea: I can see that it can be useful to repeat a block if an error occurs, but I think that it needs to be more obvious that you are looping. You also seem to be assuming that the only error that will be caught will be "file not found" type errors. The beauty of a try-except block is that you can have different handlers depending on the error: somefile = 42 # Oops! try: with open(somefile, 'r') as inp: ... except TypeError: handle_filename_not_a_string() Your suggested `else` clause loses all information about what sort of error occurs, as well as where: outfile = 'output.txt' with open(outfile, 'w') as out: out.write(42) # Oops! else: # Try another file. outfile = 'another file.txt' Lastly, your suggested syntax would be confusing. In try blocks, the `else` clause runs when there is no error. In with blocks, it would run when there is an error. That's not helpful: things that look similar should be similar. Of course, you can fix this problem by changing the with statement to use `except` clauses: with open(fname) as f: ... except TypeError: ... except IOError as e: ... else: # no error ... but this adds much complexity to the with statement, and except for the magic goto, you can already do that at the cost of one line and one indent level: try: with open(fname) as f: ... except TypeError: ... except IOError as e: ... else: # no error ... Saving one indent level and a line doesn't seem important enough for new syntax, especially new syntax which essentially duplicates functionality that already exists. -- Steven
I don't like the idea of that magic goto. But: with ...: ... except ...: ... ... as a shortcut for: try: with ...: ... except ...: ... ... IMHO seems to be worth considering. Regards. *j
On Mon, 28 Mar 2011 01:18:54 +0200 Jan Kaliszewski <zuo@chopin.edu.pl> wrote:
I've played around with this kind of thing in the past, and always eventually decided there wasn't a nice way to handle all the various desirable uses. The with statement has two bits of code, and wraps a try/finally around one of them. You might legitimately want to handle exceptions in either bit of code differently. Which means there are three things you might reasonably want an except clause on a with statement to do: 1) Wrap the entire statement (what Jan proposed). 2) Wrap the block contained by the with. 3) Be part of the try/finally implied by the with. On the other hand, the concept of "magic goto" does generate another idea (and hence another post). <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Mar 28, 2011 at 7:33 AM, Jakob Bowyer <jkbbwr@gmail.com> wrote:
Don't fight the language, just write a new CM that does what you want: with open_any('r', 'nofile.txt', 'another.txt') as inp: # If we get here, one of the files was opened # We can use inp.name to find out which one (And open_any() is pretty easy to write as a generator with an initial loop containing a try/catch block, an else clause on the loop that throws an exception, and then a subsequent with statement that yields the open file) You *really* need to be careful when wrapping try blocks around with statements, as they're almost always too broad (typically, you only want to cover the CM creation, not the entire body of the with statement). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, 27 Mar 2011 22:33:52 +0100 Jakob Bowyer <jkbbwr@gmail.com> wrote:
The idea of exception handlers "jumping back" is actually good enough to have been implemented in one language (eiffel), but sufficiently different from what "except" does that I think it calls for new syntax. How about a "retry" clause for try statements? I think it runs into the same problems as an "except" clause when it comes to adding it to the with clause, so lets skip that for now. retry ...: as part of a try clause would work just like an except clause: if the exception was one of those listed after retry, then you'd enter the block following the retry, otherwise you skip it. If the retry block raises an exception or hits "return" or "yield", it behaves just like an except block. If the retry block executes it's last statement, it then branches back to the first statement of the "try" block. This would let you write something like: i = 0 try: with open("tmpname.%d" % i, 'r') as inp: .... retry IOError: if IOError.errno != ENOENT: raise i += 1 if i > 100: raise To search for a file. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Mon, Mar 28, 2011 at 5:15 AM, Mike Meyer <mwm@mired.org> wrote:
I think 'retry' would be clearer if it is used as a command on its own, like return, break or continue, but then only within an except block. Your code above could then go: ... except IOError: if IOError.errno != ENOENT: raise i += 1 if i <= 100: retry else: raise -- André Engels, andreengels@gmail.com
On Mon, Mar 28, 2011 at 1:15 PM, Mike Meyer <mwm@mired.org> wrote:
If you want a loop, write a loop. for fname in possible_fnames: try: f = open(fname) except IOError: continue break else: raise RuntimeError("Could not open any of {}".format(possible_fnames)) with f: # Do stuff Turning the above into a custom "open_any" context manager is trivial. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (8)
-
Andre Engels -
Greg Ewing -
Jakob Bowyer -
Jan Kaliszewski -
Mike Meyer -
Nick Coghlan -
Steven D'Aprano -
yoav glazner