[Python-ideas] Type hints for functions with side-effects and for functions raising exceptions

[Ben Rudiak-Gould]

On Tue, Feb 19, 2019 at 3:19 PM Steven D'Aprano <steve at pearwood.info> wrote:
> And I note that in Java, where the idea of checked exceptions
> originated, it turned out to be full of problems and a very bad idea.

What should be the default behavior of a language when the programmer
doesn't explicitly handle an error? Options include:

1. Type error (Java, ML/Haskell)
2. Ignore it (C)
3. Pass it as-is to the caller
4. "Genericize" it, e.g. wrap it in a RuntimeError, then pass to the caller

The problem with the Java approach is that people don't want to think
about how to properly handle every error, and just wrap their code in
catch (...) {} instead. I think it works much better in ML/Haskell,
though perhaps only because the average skill level of the programmers
is higher.

The problem with the C approach is that people don't want to think
about how to properly handle every error, and just call every function
in a void context.

The problem with passing exceptions as-is to the caller is that
they're very often implementation details. If you're lucky, they will
propagate to a generic catch-all somewhere which will generate a
traceback that a human may be able to use to fix the problem. If
you're unlucky, the caller wrote `return d[k].frobnicate()` inside a
try block and frobnicate's internal KeyError gets misinterpreted as a
lookup failure in d.

That problem, of an inadvertently leaked implementation detail
masquerading as a proper alternate return value, used to be a huge
issue with StopIteration, causing bugs that were very hard to track
down, until PEP 479 fixed it by translating StopIteration into
RuntimeError when it crossed an abstraction boundary.

I think converting exceptions to RuntimeErrors (keeping all original
information, but bypassing catch blocks intended for specific
exceptions) is the best option. (Well, second best after ML/Haskell.)
But to make it work you probably need to support some sort of
exception specification.

I'm rambling. I suppose my points are:

* Error handing is inherently hard, and all approaches do badly
because it's hard and programmers hate it.

* Stronger typing is only bad if you just want your code to run and
are tired of fixing it to be type-correct. The people who voluntarily
add type annotations to Python programs probably aren't those kinds of
people; they're probably much more likely than the average programmer
to want checked exceptions.

* Declaring that a function only raises Foo doesn't have to mean "it
raises Foo, and also passes exceptions from subfunctions to the caller
unchanged, no matter what they are." It could also mean "it raises
Foo, and converts other exceptions into RuntimeError." This would
actually be useful because it would mean that you could safely put
longer expressions in try blocks, instead of defensively just putting
the one method call whose KeyError you want to handle in the try:
block, and moving everything else to the else: block, as I tend to do
all the time because I'm paranoid and I was bitten several times by
that StopIteration problem.

-- Ben