[Python-ideas] Improving Catching Exceptions

[Steven D'Aprano]

On Fri, Jun 23, 2017 at 09:29:23AM +1000, Cameron Simpson wrote:
> On 23Jun2017 06:55, Steven D'Aprano <steve at pearwood.info> wrote:
> >On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
> >>We usually teach our newbies to catch exceptions as narrowly as
> >>possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This
> >>works out quite well for now but the number of examples continue to grow
> >>where it's not enough.
> >
> >(1) Under what circumstances is it not enough?
> 
> I believe that he means that it isn't precise enough. In particular, 
> "nested exceptions" to me, from his use cases, means exceptions thrown from 
> within functions called at the top level. I want this control too sometimes.

But why teach it to newbies? Sven explicitly mentions teaching 
beginners. If we are talking about advanced features for experts, that's 
one thing, but it's another if we're talking about Python 101 taught to 
beginners and newbies.

Do we really need to be teaching beginners how to deal with circular 
imports beyond "don't do it"?



> Consider:
> 
>    try:
>        foo(bah[5])
>    except IndexError as e:
>        ... infer that there is no bah[5] ...
> 
> Of course, it is possible that bah[5] existed and that foo() raised an 
> IndexError of its own. One might intend some sane handling of a missing 
> bah[5] but instead silently conceal the IndexError from foo() by 
> mishandling it as a missing bah[5].

Indeed -- if both foo and bah[5] can raise IndexError when the coder 
believes that only bah[5] can, then the above code is simply buggy.

On the other hand, if the author is correct that foo cannot raise 
IndexError, then the code as given is fine.


> Naturally one can rearrange this code to call foo() outside that 
> try/except, but that degree of control often leads to quite fiddly looking 
> code with the core flow obscured by many tiny try/excepts.

Sadly, that is often the nature of real code, as opposed to "toy" or 
textbook code that demonstrates an algorithm as cleanly as possible. 
It's been said that for every line of code in the textbook, the 
function needs ten lines in production.


> One can easily want, instead, some kind of "shallow except", which would 
> catch exceptions only if they were directly raised from the surface code; 
> such a construct would catch the IndexError from a missing bah[5] in the 
> example above, but _not_ catch an IndexError raised from deeper code such 
> within the foo() function.

I think the concept of a "shallow exception" is ill-defined, and to the 
degree that it is defined, it is *dangerous*: a bug magnet waiting to 
strike.

What do you mean by "directly raised from the surface code"? Why is 
bah[5] "surface code" but foo(x) is not? But call a function (or 
method).

But worse, it seems that the idea of "shallow" or "deep" depends on 
*implementation details* of where the exception comes from.

For example, changing from a recursive function to a while loop might 
change the exception from "50 function calls deep" to "1 function deep".

What makes bah[5] "shallow"? For all you know, it calls a chain of a 
dozen __getitem__ methods, due to inheritance or proxying, before the 
exception is actually raised. Or it might call just a single __getitem__ 
method, but the method's implementation puts the error checking into a 
helper method:

    def __getitem__(self, n):
        self._validate(n)  # may raise IndexError
        ...

How many function calls are shallow, versus deep?

This concept of a shallow exception is, it seems to me, a bug magnet. It 
is superficially attractive, but then you realise that:

try:
    spam[5]
except shallow IndexError:
    ...

will behave differently depending on how spam is implemented, even if 
the interface (raises IndexError) is identical.

It seems to me that this concept is trying to let us substitute some 
sort of undefined but mechanical measurement of "shallowness" for 
actually understanding what our code does. I don't think this can work.

It would be awesome if there was some way for our language to Do What We
Mean instead of What We Say. And then we can grow a money tree, and have 
a magic plum-pudding that stays the same size no matter how many slices 
we eat, and electricity so cheap the power company pays you to use it...

*wink*


> The nested exception issue actually bites me regularly, almost always with 
> properties.
[...]
> However, more commonly I end up hiding coding errors with @property, 
> particularly nasty when the coding error is deep in some nested call. Here 
> is a nondeep example based on the above:
> 
>    @property
>    def target(self):
>        if len(self.targgets) == 1:
>            return self.targets[0]
>        raise AttributeError('only exists when this has exactly one target')

The obvious solution to this is to learn to spell correctly :-)

Actually, a linter probably would have picked up that typo. But I do 
see that the issue if more than just typos.

[...]
>            try:
>              eval('raise e2 from e', globals(), locals())
>            except:
>              # FIXME: why does this raise a SyntaxError?

Because "raise" is a statement, not an expression. You need exec().


-- 
Steve