Exception handling wart in Python

Leo Lipelis aeoo at myspamrealbox.com
Fri Nov 2 15:16:02 CET 2001

On Fri, 02 Nov 2001 02:45:38 -0500, Paul Rubin wrote:

> "Leo Lipelis" <aeoo at myspamrealbox.com> writes:
>> In 99.99% of Python code, the exceptions that are thrown are well known
>> in advance.  Even dynamically executed strings are really not so
>> dynamic, because, for example, they are often scripts that could be
>> exception-checked prior to execution.
> You said "track all the exception flows".  In technical subjects like
> programming, "all" means 100%, not 99.99%.

That was a mistake.  I will be more careful next time.  However, I believe
I did say I would settle for a good guess also.

> It's possible to write spaghetti code with exceptions, that's for sure.
> It's best to avoid doing that.  Your modules should generally not have
> the possibility of raising a lot of different exceptions. Rather, they
> should catch exceptions from modules they call, and deal with them or

That's what I want to do.  But without a good tool, it's very hard.  It
means I have to sift through endless documentation, and worse, I have to
trace the execution flow by hand in order to read all the right docs.  Or
alternatively, I have to put a lot of faith in a docstring, thinking that
the author of the docstring already traced all the exception flows for me
and is informing me of all the worthwhile exceptions I should know about.

> re-raise them in a uniform way.  Also, use the standard exception
> classes for things like system errors, attempting to raise the right
> kind of exception for the condition you encounter. Designing the
> exception system in a program is just like designing the class
> structure--it takes a certain amount of thought and tweaking but
> generally you can come up with something clean and well suited to your
> program.

Right.  I know how to use exceptions and I agree with the above and your
other advice on exception use.

>> In reality, there is no reason why you shouldn't know it.
> Well, as a matter of coding convention, that's possibly true, but in
> Python it can't really be enforced, as we've seen.

Well, I think it's more accurate to say that not all exceptions can be
enforced, but most normal exceptions can be.

Besides, I am not looking to duplicate Java by enforcing exceptions.  Even
Java itself doesn't enforce *all* exceptions.  Java has the unenforced
RuntimeException. Although you could *only* throw RuntimeException's if you
wanted, it would be bad programming practice.  Thus, Java is not a fascist
dictator.  Java does it's own thing, and it's fine.  Python has a different
flavour.  I think Python should provide a standard tool that gives warnings
about unhandled exceptions (maybe not all, but as many as possible).
Invoking this tool would be up to the programmer.  For small scripts it
wouldn't even make sense to use it.  But for a large code base, it would be
a great tool.

>> Then, you can deal with various exceptions differently.  For some, you
>> can just log them and keep going.  For others, you may request the user
>> to try again.  For yet others you can attempt automatic recovery.  All
>> this is impossible if you don't know what exceptions you're dealing
>> with.
> I don't know, maybe I'm missing something.  It's always seemed to me
> that having to declare every single exception that a function can raise
> (like Java requires) goes against the principle of modularity. Callers
> should not have to care about the internal intricacies of things they
> call.

I tend to agree with you here.  That's why I wouldn't want to copy Java
precisely.  Maybe I shouldn't say "exception handling like in Java", but
rather, "exception handling reminiscent of Java?"  Forcing everything is
what Java does, and it pays the price for doing that by losing some
expressive power and picking up some extra verboseness.  But a standard way
to get hints about unhandled exceptions in Python would go a long way
toward making exceptions more heavy-duty imo.  It should, at the very
least, be much better than just relying on the docstrings.

It's like driving and seeing.  Driving skill is not that useful for someone
who can't see the road.  Driving skill alone is not all it takes to drive.
This is the same with exceptions.  Just because Python allows raising and
catching exceptions, doesn't mean it's providing all the functionality
necessary for proper exception use.  I think it's really missing a good set
of eyes in the form of a tool that could trace the exception flow (in most

>> But, no matter how you write it, you *still have to know* what
>> exceptions you need to handle.  You can't magically decide it at run
>> time, nor do you need to, nor should you.
> This doesn't seem like a worse situation to me than if you have a
> function that returns a tuple, and you're supposed to know what the
> elements of the tuple signify, and if it returns something other than
> what you expected, your program has a bug.

I think it's much worse!  The tuple doesn't propagate until you catch it!
You always have to handle the tuple at each function call, and then, if you
also return a tuple, you do so willingly and there is a better chance of
you documenting what you are returning.  Tuples are very explicit and they
do not propagate on their own.  Exceptions are very sneaky and they DO
propagate until they're caught.  And what's worse, if they're not caught,
they terminate your program, guaranteed.  If you don't know what the
elements in tuple are, your program could potentially terminate too, but
it's not guaranteed.

I think exceptions are very beneficial, and in fact, it's good that they
can terminate your program and so on.  That's what we want.  But it's
doubly good when you know what exceptions are potentially flying by, so
that you can handle them.  Because exceptions are not in the same category
as returning tuples they deserve special attention.

>> The dynamic nature of Python is absolutely not a reason for such a
>> lucklaster handling of exceptions in Python.  Let's get this argument
>> out of the way, because I don't think it's even worth discussing
>> further, unless someone can demonstrate that in 99.99% of cases you
>> really can't possibly know what exceptions you should handle anyway.
> I don't think static analysis of the program can tell you what
> exceptions are possible, just because if you have
>   def foo(frob):
>     x = frob.mumble()
> you don't know what class frob belongs to, so you don't know what
> exceptions it can raise.  Global dataflow analysis of the whole program
> can help only some of the time, or else optimizing compilers would work
> a lot differently than they do.

Ok looking at above it's not possible.  But how about here:

---begin script---
    def foo(frob):
        x = frob.mumble()

    foo("String") # let's assume that string class has mumble method
    foo(gdbm.open("bar", "rf"))
---end script---

At some point you do know the type.  You just have to infer it.  I agree
that makes it more difficult than in Java.

> It's different in java, where frob has to be declared in the function
> header.  If you add some syntax (or a convention) declaring what frob
> is, e.g.
>   def foo(frob):
>     assert isinstance(frob, FrobClass)
>     x = frob.mumble()
> then your chances are a lot better.  But you really have to know the
> type of everything in the program that can raise an exception.


>>  Again, just because it's possible to use the language in a twisted
>> way, doesn't mean that the other 99.99% of exceptions should be
>> penalized by not having a tool that automatically warns you about
>> unhandled exceptions.
> I agree that a tool like that can be useful in some situations. But

I think most of the time it would be useful.  It wouldn't be a guarantee
like in Java, but it would go a long way to a point of "almost guaranteed
unless you did something weird with dynamic execution".

> knowing what something will do in the usual case (e.g. 99.99% of the
> time) is a lot different than being able to make rigorous inferences
> about it which require you to know what it will do 100% of the time.
> Java tries to provide the latter and I thought at first that that was
> what you wanted.

How is providing 99% of the rigour dissimilar from 100%?  From the point of
view of the coder, who is using that tool, it would be practically the
same.  Let's keep in mind, that only in the weird cases you really couldn't
guarantee correct exception flow tracing.  Usually these would be the cases
of bad programming practice.  How much more rigorous would this be than
relying on the mercy of the docs?

>> I say that without such a tool, exceptions in Python are pointless at
>> worst, and are very awkward to use at best.
> That seems like an extreme overstatement.  There's no tool like that for
> now, and exceptions still currently work pretty well, both in Python and
> in millions of lines of Lisp code, though improvements are possible in
> both languages.  Having such a tool is one of the possible improvements.

Is there any other way to improve them such that an exception flow tracing
tool would become useless?

More information about the Python-list mailing list