syntax to continue into the next subsequent except block
I think it would be useful if there was a way to skip into the next except block, perhaps with continue as I think it's currently always illegal to use in an except block. I don't believe there's currently a way to do this. This is my reasoning, often there's multiple reasons for exceptions that raise the same exception, as an example an IOError might get raised for lots of different reasons. If you want to handle one or several of these reasons, you have to catch all exceptions of this type, but there's not really a way to "put back" the exception if it isn't the type you were after. For instance try: operation() except IOError as err: if err.errno == 2: do_something() else: continue #This would continue the except down to the next check, except Exception except Exception as err: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise The current alternatives to get this behaviour I don't believe are as nice, but maybe I'm missing something This works but clearly not as nice with nested try excepts, try: try: operation() except IOError as err: if err.errno == 2: do_something() else: raise except Exception as err: logger.error("Error performing operation: {}".format(err.message)) some_clean_up() raise This is clearly a not very good and un-dry solution: try: operation() except IOError as err: if err.errno == 2: do_something() else: logger.error("Error performing operation: {}".format(err.message)) some_clean_up() raise except Exception as err: logger.error("Error performing operation: {}".format(err.message)) some_clean_up() raise There's the option of using a context manager, but personally I don't think it's as explicit or as obvious as a try except block, but maybe others would disagree class SpecificCaseErrorHandler(object): def __enter__(self): pass def __exit__(self, exc_type, exc_value, tb): if exc_type is not None: if exc_type is IOError and exc_value.errno == 2: do_something() return True logger.error("Error performing operation: {}".format(err.message)) some_clean_up() with SpecificCaseErrorHandler(): operation()
On 9/13/2012 5:05 PM, Paul Wiseman wrote:
"continue" already has a meaning that would make this ambiguous: for i in range(10): try: raise IOError() except IOError as err: continue Also, I would inevitably write what you want as: try: operation() except Exception as err: if isinstance(err, IOError): if err.errno == 2: do_something() else: logger.error( "Error performing operation: {}".format(err.message)") some_clean_up() raise -- Scott Dial scott@scottdial.com
On 9/13/2012 5:05 PM, Paul Wiseman wrote:
The current alternatives to get this behaviour I don't believe are as nice, but maybe I'm missing something
As you already know, raise puts the exception back, in a sense try: try: operation() except IOError as err: if err.errno == 2: do_something() else: raise except Exception as err: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise or probably better try: operation() except Exception as err: if isinstance(err, IOError) and err.errno == 2: do_something() else: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise -- Terry Jan Reedy
Terry Reedy writes:
try: try:
Ugh-ugh.<0.5 wink>
Ugh.<0.5 wink> Not your fault, but these constructions are pretty ugly IMO, I have to go with the OP on that. ISTR there were discussions of "qualified except" clauses here maybe 6mo to 1yr ago? That is, they'd look something like try: operation() except IOError as err if err.errno == 2: do_something() except Exception: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise Again ISTR that this got spiked for some reason, but maybe it will be of use to the OP in formulating his next idea. Sorry for the lack of precise reference.
On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
They were one of the ideas discussed when Antoine was writing PEP 3151. As I recall, nobody could think of any good use cases that didn't involve errno checking, and PEP 3151 provides a far more elegant (and cross-platform) solution to most problems that require errno checking in versions prior to 3.3. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 14 September 2012 03:32, Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah I didn't know about that, maybe I chose a bad example with IOError. The reason that got me thinking is I had to handle specific S3ResponseErrors from boto. the S3ResponseError exception class has a code attribute (or errorcode, i forget exactly). Surely in this case and a lot of other modules different exceptions are grouped together that are likely to want to be handle differently? URLError and HTTPError I'm sure fall into this catagory
On 15Sep2012 09:15, Paul Wiseman <poalman@gmail.com> wrote: | On 14 September 2012 03:32, Nick Coghlan <ncoghlan@gmail.com> wrote: | > On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull | > <stephen@xemacs.org> wrote: | > > ISTR there were discussions of "qualified except" clauses here maybe | > > 6mo to 1yr ago? That is, they'd look something like | > > try: | > > operation() | > > except IOError as err if err.errno == 2: [...] | > > Again ISTR that this got spiked for some reason, but maybe it will be | > > of use to the OP in formulating his next idea. Sorry for the lack of | > > precise reference. | > | > They were one of the ideas discussed when Antoine was writing PEP | > 3151. As I recall, nobody could think of any good use cases that | > didn't involve errno checking, and PEP 3151 provides a far more | > elegant (and cross-platform) solution to most problems that require | > errno checking in versions prior to 3.3. | > | Ah I didn't know about that, maybe I chose a bad example with IOError. | | The reason that got me thinking is I had to handle specific | S3ResponseErrors from boto. | the S3ResponseError exception class has a code attribute (or errorcode, i | forget exactly). I have to say I find this supportive. I think the reason that there were no use cases that don't involve errno is that most exceptions don't provide fine grained failure information. IOError/OSError's errno is the main exception. Personally I think it is a shame that exceptions are generally so uninspectable: raise ValueError("arbitrary prose here") Who here thinks that is useful to a _program_ for inspecting issues? It's the equivalent of the useless bug report "foo is broken!" When we're raising an exception for something that should not occur, indicating a programming bug, the above is fine; for a system failure requiring a programmed and appropriate response, the above is pretty vague. OSError and IOError fall pretty clearly into the latter category, and so does S3ResponseError and so (should) a bunch of other exceptions raised by libraries. So in a better quality exception environment, the "if" qualifier would have many more use cases that Antoine encounted while writing his PEP? So I'm a big +1 for the: except foo as bar if wibble: syntax. We do it for list comprehensions, we (kicking and screaming) do it for the tertiary choice operator (eschewing C's ?: notation) and I think we should do it here. Cheers, -- Cameron Simpson <cs@zip.com.au> Outside of a dog, a book is a man's best friend. Inside of a dog, it's too dark to read. - Groucho Marx
On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote:
So you want to write code like: except ValueError as ex if 'foo is wrong' in ex.args[0]: ? This thread started with IOError and its errno attribute, and for those exact cases I find 'except .. if' approach quite useful. But now, in 3.3 with PEP 3151 we have a much more granular exceptions tree, so instead of writing except IOError as ex if ex.errno == errno.ENOENT: you will write: except FileNotFoundError as ex: And that's actually how this class of problems should be addressed: instead of adding attributes to exceptions and exception guards to language - just design your exception classes better. We have multiple inheritance after all, the perfect method of classifying objects/exceptions, why should we code information about the exception class to some attribute? If some library unifies all types of exceptions in one 'S3ResponseError' exception - that's the problem of the library design. Big -1. - Yury
The suggestion to add ad-hoc "if <condition>" clauses to random parts of the syntax doesn't appeal to me at all. If your code becomes too complex without this you're probably doing something else wrong. -- --Guido van Rossum (python.org/~guido)
Instead of adding syntax, we could make except use isinstance. This means that those people who are using exceptions with poor hierarchies can override the classes with their own. This makes for an ugly setup but a very readable result. If you put the class definitions in another file and import them, it may end up looking quite clean. Please forgive me if I am doing this wrong, as I've never done this sort of thing before. class SpecificOSErrorType(type): def __instancecheck__(cls, othercls): if isinstance(othercls, OSError): if othercls.errno == self.errno: return True return False class FileExistsOSError(OSError, metaclass=SpecificOSErrorType): errno = 17 a = OSError(17, None) b = OSError(10, None) print(a.errno) # >> 17 print(b.errno) # >> 10 print(isinstance(a, FileExistsOSError)) # >> True print(isinstance(b, FileExistsOSError)) # >> False try: raise FileExistsOSError except OSError as e: print(e.errno) # >> 17 import os try: os.mkdir("src") except FileExistsOSError: # Fails print("Could not make directory: File already exists") Advantages: - No syntax change - Clean result (as would be expected if the hierarchy was right from the start) - Works with any library, and is backwards compatible - Would enable forward-compatibility (if this was in 3.2, we could re-implement the 3.3 hierarchy) - [Half-point] More general use of this may encourage better exception hierarchies Disadvantages: - It doesn't work yet [*wink*] - It may be a bad idea to encourage people to override standard class hierarchies, even if we agree that they are badly designed in 3.2 - Requires use of metaclasses and stuff people shouldn't need to understand - Setup is extremely ugly and much longer than the replaced code
On Sun, Sep 16, 2012 at 1:51 PM, Joshua Landau <joshua.landau.ws@gmail.com> wrote:
There's actually an already-open bug regarding that: "Catching virtual subclasses in except clauses" http://bugs.python.org/issue12029 Cheers, Chris
Guido van Rossum wrote:
The suggestion to add ad-hoc "if <condition>" clauses to random parts of the syntax doesn't appeal to me at all.
I wouldn't call it a random part of the syntax. This is not like the proposals to add if-clauses to while loops, for loops, etc -- they would just be minor syntactic sugar. This proposal addresses something that is quite awkward to express using existing constructs. -- Greg
On Sun, Sep 16, 2012 at 5:11 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
It can address an important use case and still be a random syntax change. I'm sure there are possible refactorings of the error handling from the examples that make it a lot less awkward. I don't think I've ever had a use case in my own code where I found it particularly awkward that I couldn't jump from one except clause to the next; I do remember some cases where I could simply write try: <code that may fail> except <some exception>, err: if <on further inspection we don't want to handle it>: raise # re-raise err <special handling for some variant of the exception> This would possibly be combined with other except clauses but the would be no need for the 'raise' to transfer to one of these. -- --Guido van Rossum (python.org/~guido)
On 17 September 2012 05:20, Guido van Rossum <guido@python.org> wrote:
I conceded that my idea of jumping from one except clause to the next was a bad one, but with the except..if idea there would be no jumping; it would work the same as it currently does where an exception can only enter at most one except block in the same level of try..excepts.
On Mon, Sep 17, 2012 at 8:51 PM, Paul Wiseman <poalman@gmail.com> wrote:
The key thing to remember is that the bar for new syntax is *very high*. The reason it was kicked around as a possibility in the PEP 3151 discussions is that checking errno values is really common and (more importantly) often not done correctly (leading to overbroad trapping of errors). PEP 3151 ultimately chose to solve the problem a different way (i.e. modifying the standard exception hierarchy) that didn't require new syntax. In so doing, it also gutted the case for this being a common problem. It *used to be* a common problem, but most cases of errno checking are now better handled by catching more specific OSError subclasses. It's simply not worth increasing the complexity of the except clause definition to deal with the new rare cases that are already amenable to other solutions, in particular just catching the superclass and then interrogating for additional details. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 9/17/12, Guido van Rossum <guido@python.org> wrote:
I have had cases where I wanted to either (1) Say "oops, not really a match, keep looking at the other except clauses" The except ... if answers this, but I do think virtual subclasses would be a better solution. In my case, I would have created a subclass for file errors that could be fixed automatically. (2) If there is an exception [of such and such a type] do X, but sometimes *also* do Y. The continue could answer this, but I'm not convinced it would smell any less than the current workarounds. -jJ
On 17/09/12 10:11, Greg Ewing wrote:
I don't think it is quite awkward. Instead of the proposed: try: something() except ValueError as ex if condition(ex): spam() except ValueError: # if not condition, fall through to the next except block ham() this can be easily written as: try: something() except ValueError as ex: if condition(ex): spam() else: ham() which costs you an indent level, which is not a big deal. (If your code is so deeply nested that it is a big deal, it is already in desperate need of refactoring.) Instead of the original proposal to add a "continue" statement to skip to the next except block: try: something() except HTTPError as ex: if condition(ex): continue # skip to the next except clause spam() except URLError as ex: ham() we can do: try: something() except (HTTPError, URLError) as ex: # Technically, we don't even need to list HTTPError, since it # is a subclass it will be caught by URLError too. if type(ex) is URLError or condition(ex): ham() else: spam() Have I missed any suggested use-cases? I don't think any of the existing solutions are that awkward or unpleasant to require new syntax. They're actually quite straightforward. -- Steven
On 17 September 2012 05:49, Steven D'Aprano <steve@pearwood.info> wrote:
What I think made my original case a bit awkward was a general handler for a parent exception class, and a specific except clause for a subclass (it was IOError and Exception originally) with your ValueError case, say you wanted to call ham on every case of a parent exception to ValueError (I guess it could only be Exception for this example), but not ValueError where condition was true, so how would you write this without the except..if? try: something() except HTTPError as ex if condition(ex): spam() except URLError: # if not condition, fall through to the next except block ham() (I switched ValueError for HTTPError because it has more parent classes that just Exception, because having 'except Exception' probably isn't the greatest example) I don't think any of the existing solutions are that awkward or
On 17/09/12 20:43, Paul Wiseman wrote:
try: something() except URLError as ex: # will also catch HTTPError # if you want to be explicit, say "except HTTPError, URLError as ex" if isinstance(ex, HTTPError) and condition(ex): spam() else: ham()
Sure, no worries. -- Steven
Steven D'Aprano writes:
I don't think [the existing syntax] is quite awkward.
Of course it is. It doesn't fit the semantics, and that is why it is awkward. An exception should be caught by an 'except' clause, not by a conditional statement. As Antoine's refactoring demonstrates, these Exceptions that are differentiated by an internal errno are often conceptually a variety of different exceptions. I agree with the suggestion that really we should fix up other Exceptions that are clearly quite heterogeneous by subclassing more precise exceptions from them. But this isn't always possible, and of course may not be backward compatible. I don't know how to do it Pythonically, but it would be nice if there were some way to "subclass Exceptions on the fly". (other) Steve
On Tue, Sep 18, 2012 at 11:08 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I don't know how to do it Pythonically, but it would be nice if there were some way to "subclass Exceptions on the fly".
Most likely, this will mean fixing the bug that means the ABC machinery is currently being ignored by the exception machinery. Then you can do whatever you want to reshape exception hierarchies. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Sep 17, 2012 at 11:55 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
If done well, it should be fully backwards compatible. There is not reason that you cannot keep the errno (or however its named on the specific exception class) while still subclassing the exception. You could have issues with pickled exceptions if you add more details to the exceptions, and it is not forward compatible: newer code that gets the older form of exceptions is liable to break, without extreme care to update the exceptions.
One trick we've done at work in a few cases is to put a fairly thin wrapper around the low-level apis that wrap the exceptions into subclasses. Often, this can be implemented as a simple decorator that then can be used on a series of functions easily.
Chris Kaynor writes:
Of course that's backward compatible with old code receiving the new subclassed exceptions. Maybe I misused the term "backward compatible", but what I meant was that it's likely to be the case that the person doing the subclassing will unify groups of error codes he believes unlikely to need distinguishing in Python applications (vs. more low-level languages). Eg, I don't see more than one hundred new Exceptions in PEP 3151, but only a handful (about 15).
On 9/16/2012 8:11 PM, Greg Ewing wrote:
I see it as quite similar. The proposal is to lift conditions out of the body of a compound statement and put them in the header. There is always the option to catch everything in one except statement and conditionally process or re-raise as desired. Having all exceptions be instances of named subclasses of one baseclass makes this easier than when exceptions were strings. -- Terry Jan Reedy
On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml@gmail.com> wrote: | On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote: | > On 15Sep2012 09:15, Paul Wiseman <poalman@gmail.com> wrote: | > | The reason that got me thinking is I had to handle specific | > | S3ResponseErrors from boto. | > | the S3ResponseError exception class has a code attribute (or errorcode, i | > | forget exactly). | > | > I have to say I find this supportive. I think the reason that there were | > no use cases that don't involve errno is that most exceptions don't | > provide fine grained failure information. IOError/OSError's errno is | > the main exception. | > | > Personally I think it is a shame that exceptions are generally so | > uninspectable: | > | > raise ValueError("arbitrary prose here") | | So you want to write code like: | | except ValueError as ex if 'foo is wrong' in ex.args[0]: No! Not at all. I was moaning about the lack of inspectability of most of the exceptions, and ValueError is top of the list. Of course, it has to be since we raise it for almost any sanity check failure; there's not nice enumeration of the (unbounded) possible ways a value may be unsuitable. | This thread started with IOError and its errno attribute, and for those | exact cases I find 'except .. if' approach quite useful. But now, in | 3.3 with PEP 3151 we have a much more granular exceptions tree, so instead | of writing | | except IOError as ex if ex.errno == errno.ENOENT: | | you will write: | | except FileNotFoundError as ex: | | And that's actually how this class of problems should be addressed: | instead of adding attributes to exceptions and exception guards to language - | just design your exception classes better. I disagree here. Fairly strongly, actually. FOr the record I am much cooler on the except...if notion than I was yesterday, +0 or maybe +0.5. BUT... OSErrno and IOError are generally built on low level OS APIs, and returning errno is a highly correct thing to do. It passes our, _with_ the exception (so it doesn't get maked by another library call, as the global POSIX error is subject to), the actual OS-level failure that was reported. Likewise with the S3 exceptions and probably any other well designed exception response to a library call with an informative failure code. | We have multiple inheritance after all, the perfect method of classifying | objects/exceptions, why should we code information about the exception class | to some attribute? | If some library unifies all types of exceptions in one 'S3ResponseError' | exception - that's the problem of the library design. No, not necessarily. Having a ridiculous suite of a billion trite subclasses to enumerate the return codes from a lower level (or more "inner") library is just nuts. The PEP class tree is handy to _group_ an assortment of failure modes into small groups of the same flavour. But an exact one-to-one between exception subclasses and errno values? Ghastly. It _doubles_ the cognitive burden on the dev and the code reader, because the correspondence between the OS-level errno and the subclass names needs to kept in mind if the program cares about the OS level failure mode. Which it does if it is bothering to make a fine grained decision at all. It is all very well to offer an, um, rich suite of subclasses representing various library failure modes. But to toss the _actual_ library failure indicator value in favour of a arbitrary and possibly incomplete class name list? Bad, really bad. The exception _should_ carry with it the underlying library failure code if the library has such a thing. | Big -1. I'm not +1 any more, but still +. -- Cameron Simpson <cs@zip.com.au> Judging by my employee ID# my employer thinks I am a small filing cabinet, so I dont think they give a care about my opinions. - Michael Jones <michaelj@nafohq.hp.com>
On 16 September 2012 23:30, Cameron Simpson <cs@zip.com.au> wrote:
On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml@gmail.com> wrote: | On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote:
<snip>
<snip> OSErrno and IOError are generally built on low level OS APIs, and
"Informative" failure code? FileNotFoundError(...) contains *exactly* the same information as OSError(errno.ENOENT, ....). A number is not an informative error code, and never will be. Additionally, I don't quite follow your first paragraph ("It passes our, _with_ the exception (so it doesn't get maked by another library call, as the global POSIX error is subject to),", but from what I can tell it seems extremely irrelevant to the end-programmer. I don't care what the OS-level failure is in terms of a number, I care in terms of the actual problem and what happened. It's not like we're receiving native data from the OS itself. You have a point that error codes can be informative. They *can* be. When you have a ValueError it could come from almost anywhere. However, it is a rare case for you to need to distinguish between these, and when you do it is normally for a reason specific enough that two subclasses can easily account for it all? I wouldn't mind (except that I mind being wrong :P) you showing me where you do have a sort of structure where you need to differentiate between many of the same error class yet cannot split it up, but until you do I don't believe that there's an analog case where this could help. | We have multiple inheritance after all, the perfect method of classifying
As said above, how is Library.MathError(5) more arbitrary than Library.UncalculatableMathError()? If the number of errnos is large [n], then the cognitive burden is already large [n]. So if instead you have a large number [n] of error classes, how is the burden less [n == n]? It doesn't add any real effort on any side as you needed to allocate the numbers anyway, as you need to know the numbers. Yes, if you have an incomplete name list you will suffer. But so what? Just cover all your bases. If you are wrapping a program from a lower-level language, wrap *everything you need*. It's no different to any other aspect of wrapping libraries.
| Big -1.
I'm not +1 any more, but still +.
I'm negative, but not -1. The problem is: there are bad libraries. I think the stuff I mentioned already is a better solution though, and it seems it's not even my idea :).
On 9/16/12, Joshua Landau <joshua.landau.ws@gmail.com> wrote:
On 16 September 2012 23:30, Cameron Simpson <cs@zip.com.au> wrote:
If the library created them, it is distinguishing between them -- regardless of whether it distinguishes by name or by number.
As said above, how is Library.MathError(5) more arbitrary than Library.UncalculatableMathError()?
Numbers are more likely to get shifted by accident when someone adds a new value. But the point isn't that codes are more arbitrary -- it is that a name is more helpful when debugging.
And, more to the point, wrap *only* what you need. If you were providing the sole wrapper for a library, then you might have a fairly long translation list. But if you're just using the library, only create virtual subclasses for the conditions that you happen to care about, and name them based on why they matter. -jJ
On 17 September 2012 20:39, Jim Jewett <jimjjewett@gmail.com> wrote:
And I totally agree. My original post was meant to say "how is * Library.UncalculableMathError()* more arbitrary than *Library.MathError(5)*?" as a refute to a claim that they are. Thanks for catching that.
On 14 September 2012 02:52, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I like that "qualified except". Almost goes without saying it's a much better idea/solution that my idea of a continue (which has already pointed out to be flawed- I'm not sure why now I thought it was always a syntax error)
On 9/13/2012 5:05 PM, Paul Wiseman wrote:
"continue" already has a meaning that would make this ambiguous: for i in range(10): try: raise IOError() except IOError as err: continue Also, I would inevitably write what you want as: try: operation() except Exception as err: if isinstance(err, IOError): if err.errno == 2: do_something() else: logger.error( "Error performing operation: {}".format(err.message)") some_clean_up() raise -- Scott Dial scott@scottdial.com
On 9/13/2012 5:05 PM, Paul Wiseman wrote:
The current alternatives to get this behaviour I don't believe are as nice, but maybe I'm missing something
As you already know, raise puts the exception back, in a sense try: try: operation() except IOError as err: if err.errno == 2: do_something() else: raise except Exception as err: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise or probably better try: operation() except Exception as err: if isinstance(err, IOError) and err.errno == 2: do_something() else: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise -- Terry Jan Reedy
Terry Reedy writes:
try: try:
Ugh-ugh.<0.5 wink>
Ugh.<0.5 wink> Not your fault, but these constructions are pretty ugly IMO, I have to go with the OP on that. ISTR there were discussions of "qualified except" clauses here maybe 6mo to 1yr ago? That is, they'd look something like try: operation() except IOError as err if err.errno == 2: do_something() except Exception: logger.error("Error performing operation: {}".format(err.message)") some_clean_up() raise Again ISTR that this got spiked for some reason, but maybe it will be of use to the OP in formulating his next idea. Sorry for the lack of precise reference.
On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
They were one of the ideas discussed when Antoine was writing PEP 3151. As I recall, nobody could think of any good use cases that didn't involve errno checking, and PEP 3151 provides a far more elegant (and cross-platform) solution to most problems that require errno checking in versions prior to 3.3. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 14 September 2012 03:32, Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah I didn't know about that, maybe I chose a bad example with IOError. The reason that got me thinking is I had to handle specific S3ResponseErrors from boto. the S3ResponseError exception class has a code attribute (or errorcode, i forget exactly). Surely in this case and a lot of other modules different exceptions are grouped together that are likely to want to be handle differently? URLError and HTTPError I'm sure fall into this catagory
On 15Sep2012 09:15, Paul Wiseman <poalman@gmail.com> wrote: | On 14 September 2012 03:32, Nick Coghlan <ncoghlan@gmail.com> wrote: | > On Fri, Sep 14, 2012 at 11:52 AM, Stephen J. Turnbull | > <stephen@xemacs.org> wrote: | > > ISTR there were discussions of "qualified except" clauses here maybe | > > 6mo to 1yr ago? That is, they'd look something like | > > try: | > > operation() | > > except IOError as err if err.errno == 2: [...] | > > Again ISTR that this got spiked for some reason, but maybe it will be | > > of use to the OP in formulating his next idea. Sorry for the lack of | > > precise reference. | > | > They were one of the ideas discussed when Antoine was writing PEP | > 3151. As I recall, nobody could think of any good use cases that | > didn't involve errno checking, and PEP 3151 provides a far more | > elegant (and cross-platform) solution to most problems that require | > errno checking in versions prior to 3.3. | > | Ah I didn't know about that, maybe I chose a bad example with IOError. | | The reason that got me thinking is I had to handle specific | S3ResponseErrors from boto. | the S3ResponseError exception class has a code attribute (or errorcode, i | forget exactly). I have to say I find this supportive. I think the reason that there were no use cases that don't involve errno is that most exceptions don't provide fine grained failure information. IOError/OSError's errno is the main exception. Personally I think it is a shame that exceptions are generally so uninspectable: raise ValueError("arbitrary prose here") Who here thinks that is useful to a _program_ for inspecting issues? It's the equivalent of the useless bug report "foo is broken!" When we're raising an exception for something that should not occur, indicating a programming bug, the above is fine; for a system failure requiring a programmed and appropriate response, the above is pretty vague. OSError and IOError fall pretty clearly into the latter category, and so does S3ResponseError and so (should) a bunch of other exceptions raised by libraries. So in a better quality exception environment, the "if" qualifier would have many more use cases that Antoine encounted while writing his PEP? So I'm a big +1 for the: except foo as bar if wibble: syntax. We do it for list comprehensions, we (kicking and screaming) do it for the tertiary choice operator (eschewing C's ?: notation) and I think we should do it here. Cheers, -- Cameron Simpson <cs@zip.com.au> Outside of a dog, a book is a man's best friend. Inside of a dog, it's too dark to read. - Groucho Marx
On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote:
So you want to write code like: except ValueError as ex if 'foo is wrong' in ex.args[0]: ? This thread started with IOError and its errno attribute, and for those exact cases I find 'except .. if' approach quite useful. But now, in 3.3 with PEP 3151 we have a much more granular exceptions tree, so instead of writing except IOError as ex if ex.errno == errno.ENOENT: you will write: except FileNotFoundError as ex: And that's actually how this class of problems should be addressed: instead of adding attributes to exceptions and exception guards to language - just design your exception classes better. We have multiple inheritance after all, the perfect method of classifying objects/exceptions, why should we code information about the exception class to some attribute? If some library unifies all types of exceptions in one 'S3ResponseError' exception - that's the problem of the library design. Big -1. - Yury
The suggestion to add ad-hoc "if <condition>" clauses to random parts of the syntax doesn't appeal to me at all. If your code becomes too complex without this you're probably doing something else wrong. -- --Guido van Rossum (python.org/~guido)
Instead of adding syntax, we could make except use isinstance. This means that those people who are using exceptions with poor hierarchies can override the classes with their own. This makes for an ugly setup but a very readable result. If you put the class definitions in another file and import them, it may end up looking quite clean. Please forgive me if I am doing this wrong, as I've never done this sort of thing before. class SpecificOSErrorType(type): def __instancecheck__(cls, othercls): if isinstance(othercls, OSError): if othercls.errno == self.errno: return True return False class FileExistsOSError(OSError, metaclass=SpecificOSErrorType): errno = 17 a = OSError(17, None) b = OSError(10, None) print(a.errno) # >> 17 print(b.errno) # >> 10 print(isinstance(a, FileExistsOSError)) # >> True print(isinstance(b, FileExistsOSError)) # >> False try: raise FileExistsOSError except OSError as e: print(e.errno) # >> 17 import os try: os.mkdir("src") except FileExistsOSError: # Fails print("Could not make directory: File already exists") Advantages: - No syntax change - Clean result (as would be expected if the hierarchy was right from the start) - Works with any library, and is backwards compatible - Would enable forward-compatibility (if this was in 3.2, we could re-implement the 3.3 hierarchy) - [Half-point] More general use of this may encourage better exception hierarchies Disadvantages: - It doesn't work yet [*wink*] - It may be a bad idea to encourage people to override standard class hierarchies, even if we agree that they are badly designed in 3.2 - Requires use of metaclasses and stuff people shouldn't need to understand - Setup is extremely ugly and much longer than the replaced code
On Sun, Sep 16, 2012 at 1:51 PM, Joshua Landau <joshua.landau.ws@gmail.com> wrote:
There's actually an already-open bug regarding that: "Catching virtual subclasses in except clauses" http://bugs.python.org/issue12029 Cheers, Chris
Guido van Rossum wrote:
The suggestion to add ad-hoc "if <condition>" clauses to random parts of the syntax doesn't appeal to me at all.
I wouldn't call it a random part of the syntax. This is not like the proposals to add if-clauses to while loops, for loops, etc -- they would just be minor syntactic sugar. This proposal addresses something that is quite awkward to express using existing constructs. -- Greg
On Sun, Sep 16, 2012 at 5:11 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
It can address an important use case and still be a random syntax change. I'm sure there are possible refactorings of the error handling from the examples that make it a lot less awkward. I don't think I've ever had a use case in my own code where I found it particularly awkward that I couldn't jump from one except clause to the next; I do remember some cases where I could simply write try: <code that may fail> except <some exception>, err: if <on further inspection we don't want to handle it>: raise # re-raise err <special handling for some variant of the exception> This would possibly be combined with other except clauses but the would be no need for the 'raise' to transfer to one of these. -- --Guido van Rossum (python.org/~guido)
On 17 September 2012 05:20, Guido van Rossum <guido@python.org> wrote:
I conceded that my idea of jumping from one except clause to the next was a bad one, but with the except..if idea there would be no jumping; it would work the same as it currently does where an exception can only enter at most one except block in the same level of try..excepts.
On Mon, Sep 17, 2012 at 8:51 PM, Paul Wiseman <poalman@gmail.com> wrote:
The key thing to remember is that the bar for new syntax is *very high*. The reason it was kicked around as a possibility in the PEP 3151 discussions is that checking errno values is really common and (more importantly) often not done correctly (leading to overbroad trapping of errors). PEP 3151 ultimately chose to solve the problem a different way (i.e. modifying the standard exception hierarchy) that didn't require new syntax. In so doing, it also gutted the case for this being a common problem. It *used to be* a common problem, but most cases of errno checking are now better handled by catching more specific OSError subclasses. It's simply not worth increasing the complexity of the except clause definition to deal with the new rare cases that are already amenable to other solutions, in particular just catching the superclass and then interrogating for additional details. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 9/17/12, Guido van Rossum <guido@python.org> wrote:
I have had cases where I wanted to either (1) Say "oops, not really a match, keep looking at the other except clauses" The except ... if answers this, but I do think virtual subclasses would be a better solution. In my case, I would have created a subclass for file errors that could be fixed automatically. (2) If there is an exception [of such and such a type] do X, but sometimes *also* do Y. The continue could answer this, but I'm not convinced it would smell any less than the current workarounds. -jJ
On 17/09/12 10:11, Greg Ewing wrote:
I don't think it is quite awkward. Instead of the proposed: try: something() except ValueError as ex if condition(ex): spam() except ValueError: # if not condition, fall through to the next except block ham() this can be easily written as: try: something() except ValueError as ex: if condition(ex): spam() else: ham() which costs you an indent level, which is not a big deal. (If your code is so deeply nested that it is a big deal, it is already in desperate need of refactoring.) Instead of the original proposal to add a "continue" statement to skip to the next except block: try: something() except HTTPError as ex: if condition(ex): continue # skip to the next except clause spam() except URLError as ex: ham() we can do: try: something() except (HTTPError, URLError) as ex: # Technically, we don't even need to list HTTPError, since it # is a subclass it will be caught by URLError too. if type(ex) is URLError or condition(ex): ham() else: spam() Have I missed any suggested use-cases? I don't think any of the existing solutions are that awkward or unpleasant to require new syntax. They're actually quite straightforward. -- Steven
On 17 September 2012 05:49, Steven D'Aprano <steve@pearwood.info> wrote:
What I think made my original case a bit awkward was a general handler for a parent exception class, and a specific except clause for a subclass (it was IOError and Exception originally) with your ValueError case, say you wanted to call ham on every case of a parent exception to ValueError (I guess it could only be Exception for this example), but not ValueError where condition was true, so how would you write this without the except..if? try: something() except HTTPError as ex if condition(ex): spam() except URLError: # if not condition, fall through to the next except block ham() (I switched ValueError for HTTPError because it has more parent classes that just Exception, because having 'except Exception' probably isn't the greatest example) I don't think any of the existing solutions are that awkward or
On 17/09/12 20:43, Paul Wiseman wrote:
try: something() except URLError as ex: # will also catch HTTPError # if you want to be explicit, say "except HTTPError, URLError as ex" if isinstance(ex, HTTPError) and condition(ex): spam() else: ham()
Sure, no worries. -- Steven
Steven D'Aprano writes:
I don't think [the existing syntax] is quite awkward.
Of course it is. It doesn't fit the semantics, and that is why it is awkward. An exception should be caught by an 'except' clause, not by a conditional statement. As Antoine's refactoring demonstrates, these Exceptions that are differentiated by an internal errno are often conceptually a variety of different exceptions. I agree with the suggestion that really we should fix up other Exceptions that are clearly quite heterogeneous by subclassing more precise exceptions from them. But this isn't always possible, and of course may not be backward compatible. I don't know how to do it Pythonically, but it would be nice if there were some way to "subclass Exceptions on the fly". (other) Steve
On Tue, Sep 18, 2012 at 11:08 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I don't know how to do it Pythonically, but it would be nice if there were some way to "subclass Exceptions on the fly".
Most likely, this will mean fixing the bug that means the ABC machinery is currently being ignored by the exception machinery. Then you can do whatever you want to reshape exception hierarchies. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Sep 17, 2012 at 11:55 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
If done well, it should be fully backwards compatible. There is not reason that you cannot keep the errno (or however its named on the specific exception class) while still subclassing the exception. You could have issues with pickled exceptions if you add more details to the exceptions, and it is not forward compatible: newer code that gets the older form of exceptions is liable to break, without extreme care to update the exceptions.
One trick we've done at work in a few cases is to put a fairly thin wrapper around the low-level apis that wrap the exceptions into subclasses. Often, this can be implemented as a simple decorator that then can be used on a series of functions easily.
Chris Kaynor writes:
Of course that's backward compatible with old code receiving the new subclassed exceptions. Maybe I misused the term "backward compatible", but what I meant was that it's likely to be the case that the person doing the subclassing will unify groups of error codes he believes unlikely to need distinguishing in Python applications (vs. more low-level languages). Eg, I don't see more than one hundred new Exceptions in PEP 3151, but only a handful (about 15).
On 9/16/2012 8:11 PM, Greg Ewing wrote:
I see it as quite similar. The proposal is to lift conditions out of the body of a compound statement and put them in the header. There is always the option to catch everything in one except statement and conditionally process or re-raise as desired. Having all exceptions be instances of named subclasses of one baseclass makes this easier than when exceptions were strings. -- Terry Jan Reedy
On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml@gmail.com> wrote: | On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote: | > On 15Sep2012 09:15, Paul Wiseman <poalman@gmail.com> wrote: | > | The reason that got me thinking is I had to handle specific | > | S3ResponseErrors from boto. | > | the S3ResponseError exception class has a code attribute (or errorcode, i | > | forget exactly). | > | > I have to say I find this supportive. I think the reason that there were | > no use cases that don't involve errno is that most exceptions don't | > provide fine grained failure information. IOError/OSError's errno is | > the main exception. | > | > Personally I think it is a shame that exceptions are generally so | > uninspectable: | > | > raise ValueError("arbitrary prose here") | | So you want to write code like: | | except ValueError as ex if 'foo is wrong' in ex.args[0]: No! Not at all. I was moaning about the lack of inspectability of most of the exceptions, and ValueError is top of the list. Of course, it has to be since we raise it for almost any sanity check failure; there's not nice enumeration of the (unbounded) possible ways a value may be unsuitable. | This thread started with IOError and its errno attribute, and for those | exact cases I find 'except .. if' approach quite useful. But now, in | 3.3 with PEP 3151 we have a much more granular exceptions tree, so instead | of writing | | except IOError as ex if ex.errno == errno.ENOENT: | | you will write: | | except FileNotFoundError as ex: | | And that's actually how this class of problems should be addressed: | instead of adding attributes to exceptions and exception guards to language - | just design your exception classes better. I disagree here. Fairly strongly, actually. FOr the record I am much cooler on the except...if notion than I was yesterday, +0 or maybe +0.5. BUT... OSErrno and IOError are generally built on low level OS APIs, and returning errno is a highly correct thing to do. It passes our, _with_ the exception (so it doesn't get maked by another library call, as the global POSIX error is subject to), the actual OS-level failure that was reported. Likewise with the S3 exceptions and probably any other well designed exception response to a library call with an informative failure code. | We have multiple inheritance after all, the perfect method of classifying | objects/exceptions, why should we code information about the exception class | to some attribute? | If some library unifies all types of exceptions in one 'S3ResponseError' | exception - that's the problem of the library design. No, not necessarily. Having a ridiculous suite of a billion trite subclasses to enumerate the return codes from a lower level (or more "inner") library is just nuts. The PEP class tree is handy to _group_ an assortment of failure modes into small groups of the same flavour. But an exact one-to-one between exception subclasses and errno values? Ghastly. It _doubles_ the cognitive burden on the dev and the code reader, because the correspondence between the OS-level errno and the subclass names needs to kept in mind if the program cares about the OS level failure mode. Which it does if it is bothering to make a fine grained decision at all. It is all very well to offer an, um, rich suite of subclasses representing various library failure modes. But to toss the _actual_ library failure indicator value in favour of a arbitrary and possibly incomplete class name list? Bad, really bad. The exception _should_ carry with it the underlying library failure code if the library has such a thing. | Big -1. I'm not +1 any more, but still +. -- Cameron Simpson <cs@zip.com.au> Judging by my employee ID# my employer thinks I am a small filing cabinet, so I dont think they give a care about my opinions. - Michael Jones <michaelj@nafohq.hp.com>
On 16 September 2012 23:30, Cameron Simpson <cs@zip.com.au> wrote:
On 16Sep2012 13:16, Yury Selivanov <yselivanov.ml@gmail.com> wrote: | On 2012-09-15, at 10:20 PM, Cameron Simpson <cs@zip.com.au> wrote:
<snip>
<snip> OSErrno and IOError are generally built on low level OS APIs, and
"Informative" failure code? FileNotFoundError(...) contains *exactly* the same information as OSError(errno.ENOENT, ....). A number is not an informative error code, and never will be. Additionally, I don't quite follow your first paragraph ("It passes our, _with_ the exception (so it doesn't get maked by another library call, as the global POSIX error is subject to),", but from what I can tell it seems extremely irrelevant to the end-programmer. I don't care what the OS-level failure is in terms of a number, I care in terms of the actual problem and what happened. It's not like we're receiving native data from the OS itself. You have a point that error codes can be informative. They *can* be. When you have a ValueError it could come from almost anywhere. However, it is a rare case for you to need to distinguish between these, and when you do it is normally for a reason specific enough that two subclasses can easily account for it all? I wouldn't mind (except that I mind being wrong :P) you showing me where you do have a sort of structure where you need to differentiate between many of the same error class yet cannot split it up, but until you do I don't believe that there's an analog case where this could help. | We have multiple inheritance after all, the perfect method of classifying
As said above, how is Library.MathError(5) more arbitrary than Library.UncalculatableMathError()? If the number of errnos is large [n], then the cognitive burden is already large [n]. So if instead you have a large number [n] of error classes, how is the burden less [n == n]? It doesn't add any real effort on any side as you needed to allocate the numbers anyway, as you need to know the numbers. Yes, if you have an incomplete name list you will suffer. But so what? Just cover all your bases. If you are wrapping a program from a lower-level language, wrap *everything you need*. It's no different to any other aspect of wrapping libraries.
| Big -1.
I'm not +1 any more, but still +.
I'm negative, but not -1. The problem is: there are bad libraries. I think the stuff I mentioned already is a better solution though, and it seems it's not even my idea :).
On 9/16/12, Joshua Landau <joshua.landau.ws@gmail.com> wrote:
On 16 September 2012 23:30, Cameron Simpson <cs@zip.com.au> wrote:
If the library created them, it is distinguishing between them -- regardless of whether it distinguishes by name or by number.
As said above, how is Library.MathError(5) more arbitrary than Library.UncalculatableMathError()?
Numbers are more likely to get shifted by accident when someone adds a new value. But the point isn't that codes are more arbitrary -- it is that a name is more helpful when debugging.
And, more to the point, wrap *only* what you need. If you were providing the sole wrapper for a library, then you might have a fairly long translation list. But if you're just using the library, only create virtual subclasses for the conditions that you happen to care about, and name them based on why they matter. -jJ
On 17 September 2012 20:39, Jim Jewett <jimjjewett@gmail.com> wrote:
And I totally agree. My original post was meant to say "how is * Library.UncalculableMathError()* more arbitrary than *Library.MathError(5)*?" as a refute to a claim that they are. Thanks for catching that.
On 14 September 2012 02:52, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I like that "qualified except". Almost goes without saying it's a much better idea/solution that my idea of a continue (which has already pointed out to be flawed- I'm not sure why now I thought it was always a syntax error)
participants (16)
-
Cameron Simpson -
Chris Kaynor -
Chris Rebert -
Greg Ewing -
Gregory P. Smith -
Guido van Rossum -
Jim Jewett -
Joshua Landau -
Nick Coghlan -
Paul Wiseman -
Rob Cliffe -
Scott Dial -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy -
Yury Selivanov