[Python-ideas] Improving Catching Exceptions

[Cameron Simpson]

On 24Jun2017 05:02, Steven D'Aprano <steve at pearwood.info> wrote:
>On Fri, Jun 23, 2017 at 09:29:23AM +1000, Cameron Simpson wrote:
>> On 23Jun2017 06:55, Steven D'Aprano <steve at pearwood.info> wrote:
>> >On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
>> >>We usually teach our newbies to catch exceptions as narrowly as
>> >>possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This
>> >>works out quite well for now but the number of examples continue to grow
>> >>where it's not enough.
>> >
>> >(1) Under what circumstances is it not enough?
>>
>> I believe that he means that it isn't precise enough. In particular,
>> "nested exceptions" to me, from his use cases, means exceptions thrown from
>> within functions called at the top level. I want this control too sometimes.
>
>But why teach it to newbies? Sven explicitly mentions teaching
>beginners. If we are talking about advanced features for experts, that's
>one thing, but it's another if we're talking about Python 101 taught to
>beginners and newbies.

It depends. Explaining that exceptions from called code can be mishandled by a 
naive try/except is something newbies need to learn to avoid common pitfalls 
with exceptions, and a real world situation that must be kept in mind when 
acting on caught exceptions.

>Do we really need to be teaching beginners how to deal with circular
>imports beyond "don't do it"?

Sven's example is with import. The situation is more general.

[... snip basic example of simple code where IndexError can arise from
  multiple causes
...]
>> Naturally one can rearrange this code to call foo() outside that
>> try/except, but that degree of control often leads to quite fiddly looking
>> code with the core flow obscured by many tiny try/excepts.
>
>Sadly, that is often the nature of real code, as opposed to "toy" or
>textbook code that demonstrates an algorithm as cleanly as possible.
>It's been said that for every line of code in the textbook, the
>function needs ten lines in production.

But not always so. And the reason for many language constructs and idioms is 
explicitly to combat what would otherwise need 10 lines of code (or 100 to do 
it "right" with corner cases), obscuring the core task and reducing 
readability/maintainability.

So "Sadly, that is often the nature of real code" is not itself an argument 
against this idea.

>> One can easily want, instead, some kind of "shallow except", which would
>> catch exceptions only if they were directly raised from the surface code;
>> such a construct would catch the IndexError from a missing bah[5] in the
>> example above, but _not_ catch an IndexError raised from deeper code such
>> within the foo() function.
>
>I think the concept of a "shallow exception" is ill-defined, and to the
>degree that it is defined, it is *dangerous*: a bug magnet waiting to
>strike.
>
>What do you mean by "directly raised from the surface code"? Why is
>bah[5] "surface code" but foo(x) is not? But call a function (or
>method).
[...]

I've replied to Paul Moore and suggested this definition as implementable and 
often useful:

  A shallow catch would effectively need to mean "the exception's
  uppermost traceback frame refers to one of the program lines
  in the try/except suite".  Which would work well for lists and
  other builtin types. And might be insufficient for a duck-type
  with python-coded dunder methods.

The target here is not perform magic but to have a useful tool to identify 
exceptions that arise fairly directly from the adjacent clode and not what it 
calls. Without writing cumbersome and fragile boilerplate to dig into an 
exception's traceback.

Cheers,
Cameron Simpson <cs at zip.com.au>