Consistent programming error handling idiom
I like that in Python all errors are exceptions. This enables elegant code and provides a default error idiom for all Python code. An important distinction between exceptions arises when handling an exception in a top-level exception handler that doesn't have the context to properly handle it. In these situations some exceptions can be ignored (and hopefully logged) and some exceptions should the terminate/reset the program (not necessarily literally). I'd call exceptions that should terminate the program "programming errors” or “bugs.” It's practically impossible to gracefully recover from a programming error when encountered (you can potentially hot reload a bug fix but I digress). Examples of current exceptions that definitely represent programming errors include AssertionError, NameError and SyntaxError. In the case of AssertionError, it's very important the program terminates or the state is reset lest you run the risk of corrupting persistent data. I'd argue that other exceptions like AttributeError, ValueError, and TypeError can also represent programming errors depending on where they are caught. If one of these exceptions is caught near the top of the call stack where nothing useful can be done it's very likely a programming error. If it's caught locally where it can be predictably handled, no hard reset is necessary, e.g.: try: foo = val.bar except AttributeError: foo = 0 Contrast with the following code that never makes sense (and is why I said that NameError definitely signifies a programming error): try: foo = bar except NameError: foo = 0 Toy examples aside, this problem arises in real programs, like an extensible HTTP server: try: response = client_request_handler(global_state, connection_state, request) except Exception: response = create_500_response() # TODO: should the server be reset? # is global_state invalid? is connection_state invalid? In this case I think it’s polite to always send a “500” error (the HTTP status code for “internal server error”) but the question remains as to whether or not the server should reset its global state or close the connection. Something smells here and I don’t think Python currently has a solution to this problem. I don’t think it should be ambiguous by the time an exception is caught whether or not the program has a bug in it or whether it has simply run into an external error. I don’t know what the solution to this should be. I’ve seen proposals like a new exception hierarchy for “bad code” but I don’t think a good solution necessarily requires changes to the language. I think the solution could be as simple as a universally accepted PEP (like PEP 8) that discusses some of these issues, and related ones, and presents correct / pythonic ways of dealing with them. Maybe the answer is recommending that top-level exception handlers should only be used with extreme care and, unless you know what you’re doing, it’s best to let your program die (or affected state reset) and bias towards more fine-grained exception handling. Thoughts? Thanks for reading. Rian
On Fri, Apr 8, 2016 at 12:17 PM, Rian Hunter <rian@thelig.ht> wrote:
This is exactly the idiom used to cope with builtins that may or may not exist. If you want to support Python 2 as well as 3, you might use something like this: try: input = raw_input except NameError: raw_input = input After this, you're guaranteed that both names exist and refer to the non-evaluating input function. So it doesn't *definitely* signify a bug; like every other exception, you can declare that it's a known and expected situation by try/excepting. An *uncaught* NameError is legitimately a bug - but then, so would most uncaught exceptions. StopIteration gets raised and caught all the time when you iterate over things, but if one leaks out, it's a bug somewhere.
This is what I'd call a boundary location. You have "outer" code and "inner" code. Any uncaught exception in the inner code should get logged rather than aborting the outer code. I'd spell it like this: try: response = ... except BaseException as e: logging.exception(...) response = create_500_response() Notably, the exception should be *logged* in some way that the author of the inner code can find it. (The outer and inner code needn't be the same 'thing', and needn't have the same author, although they might.) At this kind of boundary, you basically catch-and-log *all* exceptions, handling them the same way. Doesn't matter whether it's ValueError, SyntaxError, NameError, RuntimeError, GeneratorStop, or SystemExit - they all get logged, the client gets a 500, and you go back and look for another request. As to resetting stuff: I wouldn't bother; your functions should already not mess with global state. The only potential mess you should consider dealing with is a database rollback; and actually, my personal recommendation is to do that with a context manager inside the inner code, rather than a reset in the exception handler in the outer code. I don't think there's anything to be codified here; all you have is the basic principle that uncaught exceptions are bugs, modified by boundary locations.
This should already be the recommendation. The only time you should ever catch an exception is when you can actually do something useful with it; at a boundary location, you log all exceptions and return to some sort of main loop, and everywhere else, you catch stuff because you can usefully cope with it. This is exactly how structured exception handling should normally be used; most programs have no boundaries in them, so you simply catch what you can handle and let the rest hit the console. ChrisA
On Fri, Apr 8, 2016 at 12:17 PM, Rian Hunter <rian@thelig.ht> wrote:
This is exactly the idiom used to cope with builtins that may or may not exist. If you want to support Python 2 as well as 3, you might use something like this: try: input = raw_input except NameError: raw_input = input After this, you're guaranteed that both names exist and refer to the non-evaluating input function. So it doesn't *definitely* signify a bug; like every other exception, you can declare that it's a known and expected situation by try/excepting. An *uncaught* NameError is legitimately a bug - but then, so would most uncaught exceptions. StopIteration gets raised and caught all the time when you iterate over things, but if one leaks out, it's a bug somewhere.
This is what I'd call a boundary location. You have "outer" code and "inner" code. Any uncaught exception in the inner code should get logged rather than aborting the outer code. I'd spell it like this: try: response = ... except BaseException as e: logging.exception(...) response = create_500_response() Notably, the exception should be *logged* in some way that the author of the inner code can find it. (The outer and inner code needn't be the same 'thing', and needn't have the same author, although they might.) At this kind of boundary, you basically catch-and-log *all* exceptions, handling them the same way. Doesn't matter whether it's ValueError, SyntaxError, NameError, RuntimeError, GeneratorStop, or SystemExit - they all get logged, the client gets a 500, and you go back and look for another request. As to resetting stuff: I wouldn't bother; your functions should already not mess with global state. The only potential mess you should consider dealing with is a database rollback; and actually, my personal recommendation is to do that with a context manager inside the inner code, rather than a reset in the exception handler in the outer code. I don't think there's anything to be codified here; all you have is the basic principle that uncaught exceptions are bugs, modified by boundary locations.
This should already be the recommendation. The only time you should ever catch an exception is when you can actually do something useful with it; at a boundary location, you log all exceptions and return to some sort of main loop, and everywhere else, you catch stuff because you can usefully cope with it. This is exactly how structured exception handling should normally be used; most programs have no boundaries in them, so you simply catch what you can handle and let the rest hit the console. ChrisA
participants (2)
-
Chris Angelico -
Rian Hunter