[Python-ideas] Proposal: Abolition of bare except clauses

[Steven D'Aprano]

On Sat, Apr 11, 2015 at 10:39:04PM +1000, Chris Angelico wrote:
> On Sat, Apr 11, 2015 at 10:25 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> > On Apr 11, 2015, at 04:29, Chris Angelico <rosuav at gmail.com> wrote:
> >
> > A bare "except:" clause catches any exception. As of Python 3.0, this
> > can be spelled more explicitly as "except BaseException:", and AFAIK
> > the only difference between the two is that someone might rebind
> > BaseException.
> >
> > String exceptions were abolished during the 2.x line, without causing
> > major upheaval.
> >
> >
> > I couldn't remember when this happened, so I look at the docs... But 2.7
> > (https://docs.python.org/2.7/reference/executionmodel.html#exceptions) still
> > say "Exceptions can also be identified by strings, in which case the except
> > clause is selected by object identity." Do the docs need to be fixed?
> 
> 1) Yes, sounds like it.
> 2) By identity? Not by equality??
> 3) I have absolutely no idea about string exceptions - I do my best to
> avoid them.

Good plan :-)

String exceptions were noisily deprecated in 2.5 and removed in 2.6, but 
they were obsolete for a long time before that. I don't think they were 
really common since the Python 1.5 days. And yes, they matched by 
identity! That was a common source of bugs back in 1.5 days.

Python 2.7 does allow you to raise from arbitrary old-style classes:

[steve at ando ~]$ python2.7 -c "class YeOldeClass: pass
> raise YeOldeClass"
Traceback (most recent call last):
  File "<string>", line 2, in <module>
__main__.YeOldeClass: <__main__.YeOldeClass instance at 0xb7f2d1cc>

but new style classes must be derived from BaseException in both 2.7 and 
3.x, so people migrating from 2.x to 3.x will have to change their 
exceptions to inherit directly or indirectly from BaseException. This 
will not change that.

* In 3.x only code, bare except is equivalent to `except BaseException`;

* in 2.x only code, they are not the same;

* but if you are migrating to 3.x, or have 2+3 hybrid code, you cannot 
  use non-BaseException exceptions, which means that in hybrid code
  a bare except is *effectively* equivalent to `except BaseException`.

So I don't think this proposed change would add any significant burden 
to migration. You still have to make sure all your exceptions are actual 
exceptions.

I think that a bare except is an attractive nuisance. Nearly always it 
should be catching a specific exception, or at most Exception. Very 
rarely you want to catch BaseException. I think that Python would be 
better off without bare except clauses, having to explicitly write 
Exception or (more rarely) BaseException is a good thing.

I can see four objections:

(1) In the interactive interpreter, sometimes you don't care about being 
careful. Being able to just catch everything with a bare except is fine 
for interactive use. (I do so myself.)

Response: I don't think much of this argument from laziness. If you're 
using the II a lot, you should have a startup file. Just define E = 
Exception in your startup file and then you can say "except E:" which is 
almost as convenient. Or use tab completion (enabled by default from 3.4 
onwards). We shouldn't allow convenience in the interactive interpreter 
be an excuse for keeping a feature which gets misused as often as bare 
except does.

(2) Should 2to3 automatically translate "except:" to "except Exception:" 
or "except BaseException:"? The second is the literal translation, but 
the first is what the programmer probably intended. Probably, but not 
always.

Response: I think the 2to3 fixer will have to use BaseException, even 
though normally we recommend to catch Exception.

If the fixer uses Exception, some code that was working will break (it 
now catches too little), but some code that was broken (it caught too 
much) will be less broken. If the fixer uses BaseException, working code 
will stay working, and broken code will stay broken. 2to3 isn't 
responsible for fixing your bugs, and it certainly shouldn't introduce 
bugs if it can avoid it. Hence, it must transform bare except to 
catching BaseException. (Maybe it can ask the programmer?)

(3) Bare excepts have their uses and they aren't all bad. We should keep 
them.

Response: Yeah, well, that's like just your opinion!

*wink*

I guess if you mostly work with good-quality mature code written by 
experienced Python developers who never use bare except clauses except 
when they really are needed, you may not think this is a problem to be 
solved. Bare excepts aren't *all* bad, and if they are abused 
occasionally, oh well, any feature can be abused.

If, on the other hand, you mostly work with variable-quality code 
written by inexperienced, lazy, or just plain poor programmers who think 
that tossing a bare except around everything makes the code bug-free, 
then you might have a different opinion, and you might curse bare except 
as an attractive nuisance and want to see it die.

(4) At this late stage in Python's development we shouldn't remove even 
a dubious feature. Every little incompatibility between 2.7 and 3.x just 
slows down migration.

Response: I'm sympathetic to this argument, but I don't want to see 
"slows down migration" be used to block removal of mistakes. Even C 
removes features eventually (although on a much slower timescale than 
Python's relatively fast development cycle). 

But perhaps we don't have to remove them *right now*. We could deprecate 
them, and leave them deprecated indefinitely, for eventual removal in 
Python 5000 or something.


> > Meanwhile, in the C API
> > (https://docs.python.org/2.7/c-api/exceptions.html#string-exceptions), it
> > says "Changed in version 2.6: All exceptions to be raised or caught must be
> > derived from BaseException." I think that one is correct, but until I sit
> > down at a bigger screen where I can browse the source or run an interpreter,
> > I'm not sure.

I think that means that the C API is now fixed so that all exceptions 
raised from builtins are now real exceptions, not random classes and 
certainly not strings. But it doesn't affect the Python API.


> > Pros:
> > * Remove the attractive nuisance of "hmm, my code's throwing an error
> > that I don't recognize, I'll use try/except" and just catching
> > everything.
> >
> >
> > But if you just teach people "spell that as except BaseException:" (or, more
> > likely, "... as except Exception:"), it's still the exact same attractive
> > nuisance. And, because "except:" is almost always a red flag in novice code,
> > but "except Exception:" is occasionally reasonable, you're making it a tiny
> > bit harder to spot the mistake when helping novices, without actually fixing
> > it.
> >
> > (Of course requiring an "as" clause would solve all of this, but nobody
> > wants that...)
> 
> So you don't teach people to spell it as "except Exception". You teach
> them to look up the actual exception type they want to catch. That
> already happens, but I've seen quite a bit of code from my own
> students where the shorthand of "except:" is so tempting that it gets
> in. If that came straight back with SyntaxError, they'd be forced to
> put _something_ in, and would be more likely to put in a useful
> exception type.

That's the point. We cannot force people to catch the correct exception. 
We might not even know the correct exceptions to catch! But bare excepts 
are an attractive nuisance because (1) they're attractive because 
they're so easy (less typing is always good!) and (2) sometimes they're 
a bug magnet. We can't do anything about (2), but if people have to use 
an exception type, they may be less gung-ho about hiding bugs instead of 
fixing them ("except Exception is too much typing, I guess I'll just 
have to fix my code instead..." *wink*).


-- 
Steve