try... except with unknown error types

[Steven D'Aprano]

Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>>> But there's no way to know what that minimum is.  Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>>
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever
>> the right approach.
> 
> I'm not sure what to do instead.  The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.)  The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later.  But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown.  A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

I was waiting for you to raise network errors :)

Network errors are a particularly annoying case, because although rare and
undocumented, they are legitimate errors that should be caught. I feel your
pain.


> The Erlang approach is tempting.  Don't catch the exception at all--just
> let the process crash, and restart it.  But that's a more heavyweight
> operation in Python.

The Erland approach sounds good, but as I've never used it, I don't know how
well it works in practice.


>> After all, unless you're writing
>> software for a nuclear reactor, or an aeroplane's autopilot, chances are
>> that *bugs don't really matter*. That is to say, if you release software
>> with a hidden bug, the consequences generally aren't very important.
> 
> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down.  Also it's in an
> embedded box on a customer site.  It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.

Customers are always pissed off when something goes wrong, but ask them to
pay an extra $300 for a battery backup unit to the hardware RAID controller
*they* insisted on against your advice, and they say no. But I'm not
bitter...

Customer gets pissed off. What's the consequences? Do they sue you? Leave?
Are they locked into a five year contract and have to pay you even if they
do leave? Do you have to fix the incident at no charge, or give them two
hours free support? Are you competing with armour-plated mature software
that never goes down, or is the field yours alone? How price sensitive are
your customers? Will they pay an extra $100,000 for an extra 9 in the
expected uptime?

Without knowing the consequences to *you* of failure, I can't tell you where
you should be spending your time: trying to predict errors ahead of time,
or fixing them once they've been seen. This is a business problem, not a
software problem.

Don't misunderstand me, I'm not suggesting that you shouldn't try to avoid
things going wrong. But it's a hard problem. Solving it is why they pay you
the big bucks *cough*:

http://www.joelonsoftware.com/items/2007/12/06.html

and is the difference between version 0.4 of an application, and version
3.4. If you're trying to go straight to version 3.4 without putting the
software through real-world testing, then your testing better be mean and
tough. *Really* tough.

http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html


-- 
Steven