Exception handling wart in Python

[Paul Rubin]

"Leo Lipelis" <aeoo at myspamrealbox.com> writes:
> >> Considering that in Python exceptions are matched by identity instead
> >> of equivalence, it's possible to track all the exception flows, right?
> > 
> > Of course not.  Since you can execute arbitrary strings, you can create
> > new exception classes at runtime and throw them.
> 
> This is a non-argument.  It's like saying, you can access anything with a
> pointer in C, thus you shouldn't use any tools that can warn you about
> possible segfaults.  It's nonsense.  Just because you can dynamically
> execute dynamically created strings with unknown exceptions doesn't mean
> it's good to do so.  

But it means you can't have the exception discipline Python that
you can have in Java.  The Python compiler can't generate code based
on the idea that it knows every possible return path from the
function.  Apparently that wasn't what you cared about--I thought
maybe it was.

> In 99.99% of Python code, the exceptions that are thrown are well
> known in advance.  Even dynamically executed strings are really not
> so dynamic, because, for example, they are often scripts that could
> be exception-checked prior to execution.

You said "track all the exception flows".  In technical subjects like
programming, "all" means 100%, not 99.99%.

> > I don't understand the problem.  The whole idea of an exception is it's
> > something you don't have to know how to deal with.  You catch the
> 
> This is simply wrong.  That's not the idea.  The whole idea of exceptions
> is that you will know 99.99% of the time which exceptions you should
> handle, so that you can intelligently decide where to handle them, or
> whether to handle them at all, and how to handle them.  This decision is
> impossible if you don't know what is being thrown at a certain point.  

I mean the idea of RAISING an exception is that it's a condition you
don't know how to deal with.  The caller then has to either deal with
it if it knows how.  If it doesn't know how, it should raise another
exception.  If it's the top level and gets an unexpected exception,
that's an error condition.

It's possible to write spaghetti code with exceptions, that's for
sure.  It's best to avoid doing that.  Your modules should generally
not have the possibility of raising a lot of different exceptions.
Rather, they should catch exceptions from modules they call, and deal
with them or re-raise them in a uniform way.  Also, use the standard
exception classes for things like system errors, attempting to raise
the right kind of exception for the condition you encounter.
Designing the exception system in a program is just like designing the
class structure--it takes a certain amount of thought and tweaking
but generally you can come up with something clean and well suited
to your program.

> In reality, there is no reason why you shouldn't know it.

Well, as a matter of coding convention, that's possibly true, but in
Python it can't really be enforced, as we've seen.

> Then, you can deal with various exceptions differently.  For some, you can
> just log them and keep going.  For others, you may request the user to try
> again.  For yet others you can attempt automatic recovery.  All this is
> impossible if you don't know what exceptions you're dealing with.

I don't know, maybe I'm missing something.  It's always seemed to me
that having to declare every single exception that a function can
raise (like Java requires) goes against the principle of modularity.
Callers should not have to care about the internal intricacies of
things they call.  

> > exceptions you can handle, and leave the rest for higher levels of the
> > program.  You can always catch all the exceptions, and then if you get
> > something you can't deal with, re-raise it.
> 
> This is also nonsense.  Sure you can catch all exceptions and examine them.
> But in order to code anything intelligent, you again have to know ahead of
> time what these exceptions might be.  At the very *LEAST* you need a list
> of *all possible* exceptions in an entire Python code base, including
> Python standard module library!  But how much better would it be if you
> could narrow the scope to the most likely exceptions.

> except: # foo is not bound here... because I am catching ALL exceptions
>     if (not self.__canHandleException(foo)): # what is foo?
>         raise foo # and how can you re-raise it here?  what is foo?

Getting the exception object done in a kludgy way in Python, which I
agree should be improved.  You'd say

  except:
    foo = sys.exc_info[1]
    if not self.__canHandleException(foo):
       raise foo

but normally you'd write something more like your other example:

> except BlahType:
>     self.__handleBlahError()
> except BuzType:
>     self.__handleBuzError()
> except OtherType:
>     self.__handleOtherError()
> 
> But, no matter how you write it, you *still have to know* what exceptions
> you need to handle.  You can't magically decide it at run time, nor do you
> need to, nor should you.

This doesn't seem like a worse situation to me than if you have a
function that returns a tuple, and you're supposed to know what the
elements of the tuple signify, and if it returns something other
than what you expected, your program has a bug. 

> The dynamic nature of Python is absolutely not a reason for such a
> lucklaster handling of exceptions in Python.  Let's get this argument out
> of the way, because I don't think it's even worth discussing further,
> unless someone can demonstrate that in 99.99% of cases you really can't
> possibly know what exceptions you should handle anyway.

I don't think static analysis of the program can tell you what
exceptions are possible, just because if you have

  def foo(frob):
    x = frob.mumble()

you don't know what class frob belongs to, so you don't know what
exceptions it can raise.  Global dataflow analysis of the whole
program can help only some of the time, or else optimizing compilers
would work a lot differently than they do.

It's different in java, where frob has to be declared in the function
header.  If you add some syntax (or a convention) declaring what frob
is, e.g.

  def foo(frob):
    assert isinstance(frob, FrobClass)
    x = frob.mumble()

then your chances are a lot better.  But you really have to know the
type of everything in the program that can raise an exception.

>  Again, just because it's possible to use the language in a twisted
> way, doesn't mean that the other 99.99% of exceptions should be
> penalized by not having a tool that automatically warns you about
> unhandled exceptions.

I agree that a tool like that can be useful in some situations.
But knowing what something will do in the usual case (e.g. 99.99%
of the time) is a lot different than being able to make rigorous
inferences about it which require you to know what it will do 100%
of the time.  Java tries to provide the latter and I thought at
first that that was what you wanted.

> I say that without such a tool, exceptions in Python are pointless
> at worst, and are very awkward to use at best.

That seems like an extreme overstatement.  There's no tool like that
for now, and exceptions still currently work pretty well, both in
Python and in millions of lines of Lisp code, though improvements are
possible in both languages.  Having such a tool is one of the possible
improvements.