[Python-ideas] the error that raises an AttributeError should be passed to __getattr__

[Steven D'Aprano]

On Mon, Jun 19, 2017 at 04:06:56PM -0500, Jason Maldonis wrote:
> Hi everyone,
> 
> A while back I had a conversation with some folks over on python-list. I
> was having issues implementing error handling of `AttributeError`s using
> `__getattr__`.
[...]
> For example, we cannot tell the difference between `A.x` not existing
> (which would raise an AttributeError) and some attribute inside `A.x` not
> existing (which also raises an AttributeError).

I didn't understand what you were talking about here at first. If you 
write something like

A.x.y

where y doesn't exist, it's A.x.__getattr__ that is called, not 
A.__getattr__. But I went and looked at the thread in Python-Ideas and 
discovered that you're talking about the case where A.x is a descriptor, 
not an ordinary attribute, and the descriptor leaks AttributeError.

Apparently you heavily use properties, and __getattr__, and find that 
the two don't interact well together when the property getters and 
setters themselves raise AttributeError. I think that's relevant 
information that helps explain the problem you are hoping to fix.

So I *think* this demonstrates the problem:

class A(object):
    eggs = "text"
    def __getattr__(self, name):
        if name == 'cheese':
            return "cheddar"
        raise AttributeError('%s missing' % name)
    @property
    def spam(self):
        return self.eggs.uper() # Oops.


a = A()
a.spam



Which gives us 

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in __getattr__
AttributeError: spam missing


But you go on to say that:

> This is evident from the
> stack trace that gets printed to screen, but `__getattr__` doesn't get that
> stack trace.

I can't reproduce that! As you can see from the above, the stack trace 
doesn't say anything about the actual missing attribute 'uper'.

So I must admit I don't actually understand the problem you are hoping 
to solve. It seems to be different from my understanding of it.


 
> I propose that the error that triggers an `AttributeError` should get
> passed to `__getattr__` (if `__getattr__` exists of course).  Then, when
> handling errors, users could dig into the problematic error if they so
> desire.

What precisely will be passed to __getattr__? The exception instance? 
The full traceback object? The name of the missing attribute? Something 
else? It is hard to really judge this proposal without more detail.

I think the most natural thing to pass would be the exception instance, 
but AttributeError instances don't record the missing attribute name 
directly (as far as I can tell). Given:

try:
    ''.foo
except AttributeError as e:
    print(e.???)

there's nothing in e we can inspect to get the name of the missing 
exception, 'foo'. (As far as I can see.) We must parse the error message 
itself, which we really shouldn't do, because the error message is not 
part of the exception API and could change at any time.

So... what precisely should be passed to __getattr__, and what exactly 
are you going to do with it?

Having said that, there's another problem: adding this feature (whatever 
it actually is) to __getattr__ will break every existing class that uses 
__getattr__. The problem is that everyone who writes a __getattr__ 
method writes it like this:

    def __getattr__(self, name):

not:

    def __getattr__(self, name, error):

so the class will break when the method receives two arguments 
(excluding self) but only has one parameter.

*If* we go down this track, it would probably require a __future__ 
import for at least one release, probably more:

- in 3.7, use `from __future__ import extra_getattr_argument`

- in 3.8, deprecate the single-argument form of __getattr__

- in 3.9 or 4.0 no longer require the __future__ import.


That's a fairly long and heavy process, and will be quite annoying to 
those writing cross-version code using __getattr__, but it can be done. 

But only if it actually helps solve the problem. I'm not convinced that 
it does. It comes down to the question of what this second argument is, 
and how do you expect to use it?



-- 
Steve