[Python-Dev] PEP 348: Exception Reorganization for Python 3.0

[Raymond Hettinger]

[Raymond Hettinger wrote]
> >The PEP moves StopIteration out from under Exception so that it
cannot
> >be caught by a bare except or an explicit "except Exception".
> >
> >IMO, this is a mistake.  In either form, a programmer is stating that
> >they want to catch and handle just about anything.  There is a
> >reasonable argument that SystemExit special and should float to the
top,
> >but that is not the case with StopIteration.

[Phillip J. Eby]
> While I agree with most of your -1's on gratuitous changes, this
> particular
> problem isn't gratuitous.  A StopIteration that reaches a regular
> exception
> handler is a programming error; allowing StopIteration and other
> control-flow exceptions to be caught other than explicitly *masks*
> programming errors.

Thanks for clearly articulating the rationale behind moving control-flow
exceptions out from under Exception.  The idea is not entirely without
merit but I believe it is both misguided and has serious negative
consequences.

Two things are both true.  Writers of bare excepts sometimes catch more
than they intended and mask errors in their programs.  It is also true
that there are valid use cases for wanting to trap and log all
recoverable errors in long running programs (i.e. not crashing your
whole air traffic control system if submodule fails to trap a control
flow exception).

I favor the current setup for several reasons:

1.  Writing a bare except is its own warning to a programmer.  It is a
Python basic to be careful with it and to focus attention on whether it
is really intended.  PyChecker flags it and it stands out during code
review.  IOW, it is a documented, well-understood hazard that should
surprise no one.

2.  There is a lesson to be taken from a story in the ACM risks forum
where a massive phone outage was traced to a single line of C code that
ran a "break" to get out of a nested if-statement.  The interesting part
is that this was known to be mission critical code yet the error
survived multiple, independent code reviews.  The problem was that the
code created an optical illusion.  We risk the same thing when an
"except Exception" doesn't catch ControlFlowExceptions.  The
recovery/logging handler will look like it ought to catch everything,
but it won't.  That is a disaster for fault-tolerant coding and for
keeping your sales demo from exploding in front of customers.

3.  As noted above, there ARE valid use cases for bare excepts.  Also
consider that Python rarely documents or even can document all the
recoverable exceptions that can be raised by a method call.  A broad
based handler is sometimes the programmer's only defense.

4.  As noted in another email, user defined control flow exceptions are
a key use case.  I believe that was the typical use for string
exceptions.  The idea is that that user defined control flow exceptions
are one of the key means for exiting multiple layers of loops or
function calls.  If you have a bunch of these, it is reasonable to
expect that "except Exception" will catch them.

Summary:  It is a noble thought to save someone from shooting themselves
in the foot with a bare except.  However, bare excepts are clearly a
we-are-all-adults construct.  It has valid current use cases and its
current meaning is likely the intended meaning.  Making the change will
break some existing code and produce dubious benefits.  The change is at
odds with fundamental use cases for user defined control flow
exceptions.  The change introduces the serious risk of a hard-to-spot
optical illusion error where an "except Exception" doesn't catch
exceptions that were intended to be caught.

Nice try, but don't do anything this radical without validating that it
solves significant problems without introducing worse, unintended
effects.  Don't break existing code unless there is a darned good
reason.  Check with the Zope and Twisted people to see if this would
improve their lives or make things worse.  There are user constituencies
that are not being well represented in these discussions.


Raymond