[Python-ideas] Improving Catching Exceptions

[Cameron Simpson]

On 23Jun2017 06:55, Steven D'Aprano <steve at pearwood.info> wrote:
>On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
>> We usually teach our newbies to catch exceptions as narrowly as
>> possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This
>> works out quite well for now but the number of examples continue to grow
>> where it's not enough.
>
>(1) Under what circumstances is it not enough?

I believe that he means that it isn't precise enough. In particular, "nested 
exceptions" to me, from his use cases, means exceptions thrown from within 
functions called at the top level. I want this control too sometimes.

Consider:

    try:
        foo(bah[5])
    except IndexError as e:
        ... infer that there is no bah[5] ...

Of course, it is possible that bah[5] existed and that foo() raised an 
IndexError of its own. One might intend some sane handling of a missing bah[5] 
but instead silently conceal the IndexError from foo() by mishandling it as a 
missing bah[5].

Naturally one can rearrange this code to call foo() outside that try/except, 
but that degree of control often leads to quite fiddly looking code with the 
core flow obscured by many tiny try/excepts.

One can easily want, instead, some kind of "shallow except", which would catch 
exceptions only if they were directly raised from the surface code; such a 
construct would catch the IndexError from a missing bah[5] in the example 
above, but _not_ catch an IndexError raised from deeper code such within the 
foo() function.

Something equivalent to:

    try:
        foo(bah[5])
    except IndexError as e:
        if e.__traceback__ not directly from the try..except lines:
            raise
        ... infer that there is no bah[5] ...

There doesn't seem to be a concise way to write that. It might not even be 
feasible at all, as one doesn't have a way to identify the line(s) within the 
try/except in a form that one can recognise in a traceback.

I can imagine wanting to write something like this:

    try:
        foo(bah[5])
    except shallow IndexError as e:
        ... deduce that there is no bah[5] ...

Note that one can then deduce the missing bah[5] instead of inferring it.

Obviously the actual syntax above is a nonstarter, but something that succinct 
and direct would be very handy.

The nested exception issue actually bites me regularly, almost always with 
properties. The property system appears designed to allow one to make 
"conditional" properties, which appear to exist only in some circumstances. I 
wrote one of them just the other day, along the lines of:

    @property
    def target(self):
        if len(self.targets) == 1:
            return self.targets[0]
        raise AttributeError('only exists when this has exactly one target')

However, more commonly I end up hiding coding errors with @property, 
particularly nasty when the coding error is deep in some nested call. Here is a 
nondeep example based on the above:

    @property
    def target(self):
        if len(self.targgets) == 1:
            return self.targets[0]
        raise AttributeError('only exists when this has exactly one target')

Here I have misspelt ".targets" as ".targgets". And quietly the .target 
property is simply missing, and a caller may then infer, incorrectly, things 
about the number of targets. What I, as the coder, actually wanted was for the 
errant .targgets reference to trigger something different from Attribute error, 
something akin to a NameError. (Obviously it _is_ a missing attribute and that 
is what AttributeError is for, but within a property that is ... unhelpful.)

This is so common that I actually keep around a special hack:

    def prop(func):
      ''' The builtin @property decorator lets internal AttributeErrors escape.
          While that can support properties that appear to exist conditionally,
          in practice this is almost never what I want, and it masks deeper errors.
          Hence this wrapper for @property that transmutes internal AttributeErrors
          into RuntimeErrors.
      '''
      def wrapper(*a, **kw):
        try:
          return func(*a, **kw)
        except AttributeError as e:
          e2 = RuntimeError("inner function %s raised %s" % (func, e))
          if sys.version_info[0] >= 3:
            try:
              eval('raise e2 from e', globals(), locals())
            except:
              # FIXME: why does this raise a SyntaxError?
              raise e
          else:
            raise e2
      return property(wrapper)

and often define properties like this:

    from cs.py.func import prop
    .......
    @prop
    def target(self):
        if len(self.targgets) == 1:
            return self.targets[0]
        raise AttributeError('only exists when this has exactly one target')

Same shape, better semantics from a debugging point of view.

This is just one example where "nested" exceptions can be a misleading 
behavioural symptom.

>> Chris showed how to deal with 3). Catching nested exception is not what
>> people want many times.
>
>Isn't it? Why not? Can you explain further?

I hope this real world example shows why the scenario is real, and that my 
discussion shows why for me at least it would be handy to _easily_ catch the 
"shallow" exception only.

Cheers,
Cameron Simpson <cs at zip.com.au>