[Python-ideas] Arguments to exceptions

[Terry Reedy]

On 7/4/2017 10:03 AM, Steven D'Aprano wrote:
> On Mon, Jul 03, 2017 at 01:59:02AM -0700, Ken Kundert wrote:
>> I think in trying to illustrate the existing behavior I made things more
>> confusing than they needed to be.  Let me try again.
> 
> I understood you the first time :-)
> 
> I agree that scraping the name from the NameError exception is a fragile
> hack. What I'm questioning is *how often that needs be done*.
> 
> As I see it, there are broadly two use-cases for wanting the name from
> NameError (or AttributeError, or the index from IndexError, or the key
> from KeyError -- for brevity, I will use "name" and "NameError" stand in
> for *all* these cases).
> 
> 1. You are a developer reading an unexpected NameError exception, and
> now you need to debug the code and fix the bug.
> 
> In this case, just reading the error message is sufficient. There's no
> need to programmatically extract the name.
> 
> 2. You have a `try...except` block and you've just caught NameError and
> want to handle it programmatically.
> 
> In that second case, needing to extract the name from the exception is a
> code smell. A *strong* code smell -- it suggests that you're doing too
> much in the try... block. You should already know which name lookup
> failed, and so extracting the name from the exception is redundant:
> 
> try:
>      unicode
> except NameError:
>      # Python 2/3 compatibility
>      unicode = str
> 
> What other name could it be?
> 
> I believe that if you are dealing with a NameError where you want to
> programmatically deal with a missing name, but you don't know what that
> name is, you're already in deep, deep trouble and the fact that you have
> to scrape the error message for the name is the least of your problems:
> 
> try:
>      # Feature detection for Python 2/3 compatibility.
>      unicode
>      ascii
>      reduce
>      callable
> except NameError as err:
>      name = extract_name(err)  # somehow...
>      if name == 'unicode':
>          unicode = str
>      elif name == 'ascii':
>          ascii = ...
>      elif name == 'reduce':
>          from functools import reduce
>      elif name == 'callable':
>          def callable(obj): ...
> 
> I trust that the problem with this is obvious. The only way to safely
> write this code is to test for each name individually, in which case
> we're back to point 2 above where you know what name failed and you
> don't need to extract it at all.
> 
> It is my hand-wavy estimate that these two use-cases cover about 95% of
> uses for the name. We might quibble over percentages, but I think we
> should agree that whatever the number is, it is a great majority.
> 
> Any other use-cases, like your example of translating error messages, or
> suggesting "Did you mean...?" alternatives, are fairly niche uses.
> 
> So my position is that given the use-cases for programmatically
> extracting the name from NameError fall into a quite small niche, this
> feature is a "Nice To Have" not a "Must Have". It seems to me that the
> benefit is quite marginal, and so any more than the most trivial cost to
> implement this is likely to be too much for the benefit gained.
> 
> I don't just mean the effort of implementing your suggested change. I
> mean, if there is even a small chance that in the future people
> will expect me (or require me!) to write code like this:
> 
>      raise NameError('foo', template='missing "{}"')
> 
> instead of
> 
>      raise NameError('missing "foo"')
> 
> then the cost of this new feature is more than the benefit to me, and I
> don't want it.

There have been many proposals for what we might call RichExceptions, 
with more easily access information.  But as Raymond Hettinger keeps 
pointing out, Python does not use exceptions only for (hopefully rare) 
errors.  It also uses them as signals for flow control, both as an 
alternative form for alternation and for iteration.  Alternation with 
try:except instead of if:else is common.  In the try: unicode example 
above, the NameError is not an error.  Until 2.2, IndexError served the 
role of StopIteration today, and can still be used for iteration.  For 
flow control, richer exceptions just slow code execution.

> I see little benefit and a small cost (more of a nuisance than a major
> drama, but a nusiance is still a nagative). I wouldn't use the new style
> if it were available (why should I bother?) and I'd be annoyed if I were
> forced to use it.
> 
> Contrast that to OSError, where the ability to extract the errno and
> errstr separately are *very* useful. When you catch an OSError or
> IOError, you typically have *no idea* what the underlying errno is, it
> could be one of many. I don't object to writing this:
> 
>      raise OSError(errno, 'message')
> 
> because I see real benefit.

In other words, the richness of the exception should depend on the 
balance between the exception class's use as flow signal versus error 
reporting.  Note that the exception reported usually does not know what 
the exception use will be, so raising a bare signal exception versus a 
rich report exception is not an option.  An additional consideration, as 
Raymond has also pointed out, is the fractional overhead, which depends 
on the context of the exception raising.

IndexError: list index out of range

This is probably the most common flow signal after StopIteration.  Also, 
as Raymond has noted, IndexError may be come from loops with short 
execution per loop.  Attaching a *constant* string is very fast, to the 
consternation of people who would like the index reported.  I believe 
there should usually be the workaround of naming a calculated index and 
accessing it in the exception clause.

NameError: name 'xyz' is not defined

This is less commonly a flow signal. (But not never!)
Interpolating the name is worth the cost.

OSError: whatever

I believe this is even less commonly used as a flow signal.  In any 
case, the context is usually a relatively slow operation, like opening a 
file (and reading it if successful).



>> The above is an example. It is a bit contrived. I simply wanted to illustrate
>> the basic issue in a few lines of code. However, my intent was also to
>> illustrate what I see as a basic philosophical problem in the way we approach
>> exceptions in Python:
>>
>>      It is a nice convenience that an error message is provided by the source of
>>      the error, but it should not have the final say on the matter.
>>      Fundamentally, the code that actually presents the error to the user is in
>>      a better position to produce a message that is meaningful to the user.  So,
>>      when we raise exceptions, not only should we provide a convenient human
>>      readable error message, we should anticipate that the exception handler may
>>      need to reformat or reinterpret the exception and provide it with what it
>>      need to do so.
> 
> That argument makes a certain amount of sense, but its still a niche
> use-case. In *general*, we don't do our users any favours by
> reinterpreting standard error messages into something that they can't
> google, so even if this were useful, I can't see it becoming used in a
> widespread manner.

I hope not.  One of the frustrations of trying to answer StackOverflow 
question is when the user used an environment that suppresses 
stacktraces and mangles exception names and messages.

-- 
Terry Jan Reedy