On Nov 27, 2015, at 20:25, 王珺 <wjun77@gmail.com> wrote:

I don't have realistic code that blindly returns a value now, but showing unhelpful and confusing traceback messages.

Well, yes. If you catch an AttributeError and raise a different one that hides all the relevant information, it will be unhelpful and confusing. But just don't do that, and you don't have that problem.

(That being said, the way you've written it, I think the new AttributeError will contain the old one, so you should still be able to debug it--unless you're using Python 2, but in that case, obviously you need to migrating to Python 3... But anyway, it would be simpler to just not write code that makes debugging harder and provides no benefits.)

As a side note: why are you writing @staticmethods that take a self parameter? The whole point of static methods is that they don't get passed self. If you need to pass in the "owner", calling it "self" is misleading; give it a name that makes it clear what's being passed. But it looks like you don't even need that, since you never use it. Is this code by chance an attempt to directly port some Java code to Python?

> The code below doesn't even have a __getattr__ in it
>> In the window example
I've already post the __getattr__-related code when discussing the use case of __getattr__, so I omitted those code here.
Sorry for the misleading code. Here's the complete version:

class ActiveState(State):
    @staticmethod
    def rightClick(self):
        print('right clicked')
class InactiveState(State):
    @staticmethod
    def rightClick(self):
        pass

class Window():
    def __init__(self):
        self.__state = ActiveState
    def __getattr__(self, name):
        try:
            return partial(getattr(self.__state, name), self)
        except AttributeError:
            raise AttributeError("'{}' object has no attribute '{}'".format(self.__class__.__name__, name)) from None

    @property
    def backgroundImg(self):
        if self._backgroundImg is None: #need update, while the number of items changes
            self.set_backgroundImg()
        return self._backgroundImg

    def set_backgroundImg(self):
        self._backgroundImg = loadImg('white.bmp')
        for widget in self.widgets:
            widget.show(self._backgroundImg)

Class Widget():
    def show(self, img):
        img.draw(self.item.img, self.pos)


> Why? If you think that erroneous uses are rare or nonexistent, while intentional uses are rare but not nonexistent, "fixing" it means breaking code gratuitously for no benefit.
I mean if we don't consider any backward compatibility, maybe when creating a new language other than python, I think it's a better choice to 'fix' it. Only my personal opinion.

I have something urgent to do now, so I'll read the rest part of your post carefully later. Anyway thanks for your attention.

2015-11-28 11:52 GMT+08:00 Andrew Barnert <abarnert@yahoo.com>:
On Friday, November 27, 2015 3:23 PM, 王珺 <wjun77@gmail.com> wrote:

>> Do you have any examples that actually do demonstrate the problem to be solved?

>So you want more details about AttributeError in property?

No. I'm assuming Paul wanted an example that demonstrates the problem (a @property or other descriptor that raises or passes AttributeError, and a __getattr__ that blindly returns a value for anything). Just like your toy example on the bug tracker does, but realistic code rather than a toy example.

What you've provided is an example that doesn't demonstrate the problem at all. The code below doesn't even have a __getattr__ in it, and it does exactly what you should expect it to do (assuming you fill in the missing bits in any reasonable way), so it can't possibly demonstrate why interactions with __getattr__ are a problem.

> I thought property is widely used, and AttributeError occurs at all times. Maybe I've used property too heavy.>In the window example, a simplified demonstration:
>
>class Window():
>    @property
>    def backgroundImg(self):
>        if self._backgroundImg is None: #need update, while the number of items changes
>            self.set_backgroundImg()
>        return self._backgroundImg
>
>    def set_backgroundImg(self):
>        self._backgroundImg = loadImg('white.bmp')
>        for widget in self.widgets:
>            widget.show(self._backgroundImg)
>
>Class Widget():
>    def show(self, img):
>        img.draw(self.item.img, self.pos)
>

>However, widget.item may be None, while e.g. there are four widgets but only three items in total. In this case I should fill the area with white. But in this version of show, I just FORGET item can be None. So the traceback infomation: 'Window' object has no attribute 'backgroundImg'.

No, that can't possibly be your problem. If that were the case, the AttributeError will say that 'NoneType' object has no attribute 'img'. And the traceback would run from the Widget.show method, where the self.item.img is, back up the chain through your @property method.

I'm guessing your actual problem is that you forgot to set self._backgroundImg = None somewhere (e.g., in the __init__ method). In that case, you would get an error that looks more like the one you're claiming to get (but the attribute mentioned is '_backgroundImg', not 'backgroundImg'), with only one level of traceback and everything.


Or maybe there's a typo in your actual code, and you really don't have a 'backgroundImg' at all on Window objects; that would give exactly the error you're describing.

No matter which case it is, the problem has nothing to do with @property or descriptors in general, or with __getattr__ (obviously, since there is no __getattr__ in the code), much less with the interaction between them, so any fix to that interaction couldn't possibly help this example.


> In fact it takes a while before I find the cause is the AttributeError/__getattr__ mechanism.


Since that isn't the cause, it would be bad if Python pointed you to look in that direction sooner...
>> But surely breaking that isn't the same as breaking code that's been explicitly stated to work, and used as sample code, for decades.
>I don't know this is such a severe problem. I used to think raising AttributeError in __getattribute__ to trigger __getattr__ is rare.
>
>> any solution that can fix descriptors without also "fixing" __getattribute__ is a lot better

>In practice I don't concern __getattribute__. But in my opinion it's better to 'fix' this in python4.

Why? If you think that erroneous uses are rare or nonexistent, while intentional uses are rare but not nonexistent, "fixing" it means breaking code gratuitously for no benefit.
>> all of your solutions make it too hard to trigger __getattr__ from a descriptor when you really _do_ want to do so
>This is a big problem, OK.
>
>> You didn't comment on the alternative I suggested; would it not satisfy your needs, or have some other problem that makes it unacceptable?
>I don't quite understand, you mean adding a subclass of object with the only difference of this behavior?


No, adding a subclass of AttributeError, much like the one you mentioned in the bug report, but with the opposite meaning: the existing AttributeError continues to trigger __getattr__, but the new subclass doesn't. This makes it trivial to write new code that doesn't accidentally trigger __getattr__, without breaking old code (or rare new code) that wants to trigger __getattr__.

The code currently does something like this pseudocode:

    try:
        val = obj.__getattribute__(name)
    except AttributeError:
        __getattr__ = getattr(type(obj), '__getattr__', None)
        if __getattr__: return __getattr__(name)

I'm cheating a bit, but you get the idea. The problem is that we have no idea whether __getattribute__ failed to find anything (in which case we definitely want __getattr__ called), or found a descriptor whose __get__ raised an AttributeError (in which case we may not--e.g., a write-only attribute should not all through to __getattr__).

My suggestion is to change it like this:


    try:
        val = obj.__getattribute__(name)
    except AttributeDynamicError:
        raise
    except AttributeError:
        __getattr__ = getattr(type(obj), '__getattr__', None)
        if __getattr__: return __getattr__(name)


Now, if __getattribute__ found a descriptor whose __get__ raised an AttributeDynamicError, that passes on to the user code. (And, since it's a subclass of AttributeError, the user code should have no problem handling it.) And the Descriptor HOWTO will be changed to suggest raising AttributeDynamicError, except when you explicitly want it to call __getattr__, which you usually don't. And examples like simulating a write-only attribute will raise AttributeDynamicError. And maybe @property will automatically convert any AttributeError to AttributeDynamicError (not sure about that part).

So, new code can easily be written to act the way you want, but existing code using descriptors that intentionally raise or pass an AttributeError continues to work the same way it always has, and new code that does the same can also be written easily.

Obviously, the downside of any backward-compat-friendly change is that someone who has old code with a hidden bug they didn't know about will still have that same bug in Python 3.7; they have to change their code to take advantage of the fix. But I don't think that's a serious problem. (Especially if we decide @property is buggy and should be changed--most people who are writing actual custom descriptors, not just using the ones in the stdlib, probably understand this stuff.)