[Python-ideas] Consistent programming error handling idiom

[Chris Angelico]

On Fri, Apr 8, 2016 at 12:17 PM, Rian Hunter <rian at thelig.ht> wrote:
> Contrast with the following code that never makes sense (and is why I
> said that NameError definitely signifies a programming error):
>
>     try:
>         foo = bar
>     except NameError:
>         foo = 0

This is exactly the idiom used to cope with builtins that may or may
not exist. If you want to support Python 2 as well as 3, you might use
something like this:

try:
    input = raw_input
except NameError:
    raw_input = input

After this, you're guaranteed that both names exist and refer to the
non-evaluating input function. So it doesn't *definitely* signify a
bug; like every other exception, you can declare that it's a known and
expected situation by try/excepting. An *uncaught* NameError is
legitimately a bug - but then, so would most uncaught exceptions.
StopIteration gets raised and caught all the time when you iterate
over things, but if one leaks out, it's a bug somewhere.

> Toy examples aside, this problem arises in real programs, like an
> extensible HTTP server:
>
>     try:
>         response = client_request_handler(global_state, connection_state,
> request)
>     except Exception:
>         response = create_500_response()
>         # TODO: should the server be reset?
>         #       is global_state invalid? is connection_state invalid?

This is what I'd call a boundary location. You have "outer" code and
"inner" code. Any uncaught exception in the inner code should get
logged rather than aborting the outer code. I'd spell it like this:

try:
    response = ...
except BaseException as e:
    logging.exception(...)
    response = create_500_response()

Notably, the exception should be *logged* in some way that the author
of the inner code can find it. (The outer and inner code needn't be
the same 'thing', and needn't have the same author, although they
might.)

At this kind of boundary, you basically catch-and-log *all*
exceptions, handling them the same way. Doesn't matter whether it's
ValueError, SyntaxError, NameError, RuntimeError, GeneratorStop, or
SystemExit - they all get logged, the client gets a 500, and you go
back and look for another request.

As to resetting stuff: I wouldn't bother; your functions should
already not mess with global state. The only potential mess you should
consider dealing with is a database rollback; and actually, my
personal recommendation is to do that with a context manager inside
the inner code, rather than a reset in the exception handler in the
outer code.

I don't think there's anything to be codified here; all you have is
the basic principle that uncaught exceptions are bugs, modified by
boundary locations.

> Maybe the
> answer is recommending that top-level exception handlers should only
> be used with extreme care and, unless you know what you’re doing, it’s
> best to let your program die (or affected state reset) and bias
> towards more fine-grained exception handling.

This should already be the recommendation. The only time you should
ever catch an exception is when you can actually do something useful
with it; at a boundary location, you log all exceptions and return to
some sort of main loop, and everywhere else, you catch stuff because
you can usefully cope with it. This is exactly how structured
exception handling should normally be used; most programs have no
boundaries in them, so you simply catch what you can handle and let
the rest hit the console.

ChrisA