[Python-ideas] Consistent programming error handling idiom

[Rian Hunter]

I like that in Python all errors are exceptions. This enables elegant
code and provides a default error idiom for all Python code.

An important distinction between exceptions arises when handling an
exception in a top-level exception handler that doesn't have the
context to properly handle it. In these situations some exceptions can
be ignored (and hopefully logged) and some exceptions should the
terminate/reset the program (not necessarily literally).

I'd call exceptions that should terminate the program "programming
errors” or “bugs.” It's practically impossible to gracefully recover
from a programming error when encountered (you can potentially hot
reload a bug fix but I digress). Examples of current exceptions that
definitely represent programming errors include AssertionError,
NameError and SyntaxError. In the case of AssertionError, it's very
important the program terminates or the state is reset lest you
run the risk of corrupting persistent data.

I'd argue that other exceptions like AttributeError, ValueError, and
TypeError can also represent programming errors depending on where
they are caught. If one of these exceptions is caught near the top of
the call stack where nothing useful can be done it's very likely a
programming error. If it's caught locally where it can be predictably
handled, no hard reset is necessary, e.g.:

     try:
         foo = val.bar
     except AttributeError:
         foo = 0

Contrast with the following code that never makes sense (and is why I
said that NameError definitely signifies a programming error):

     try:
         foo = bar
     except NameError:
         foo = 0

Toy examples aside, this problem arises in real programs, like an
extensible HTTP server:

     try:
         response = client_request_handler(global_state, connection_state, request)
     except Exception:
         response = create_500_response()
         # TODO: should the server be reset?
         #       is global_state invalid? is connection_state invalid?

In this case I think it’s polite to always send a “500” error (the
HTTP status code for “internal server error”) but the question remains
as to whether or not the server should reset its global state or close
the connection.

Something smells here and I don’t think Python currently has a
solution to this problem. I don’t think it should be ambiguous by the
time an exception is caught whether or not the program has a bug in it
or whether it has simply run into an external error.

I don’t know what the solution to this should be. I’ve seen proposals
like a new exception hierarchy for “bad code” but I don’t think a good
solution necessarily requires changes to the language.

I think the solution could be as simple as a universally accepted PEP
(like PEP 8) that discusses some of these issues, and related ones,
and presents correct / pythonic ways of dealing with them. Maybe the
answer is recommending that top-level exception handlers should only
be used with extreme care and, unless you know what you’re doing, it’s
best to let your program die (or affected state reset) and bias
towards more fine-grained exception handling.

Thoughts? Thanks for reading.

Rian