Case study: library class inheritance with property declarations

[cdleary at gmail.com]

On Aug 2, 7:05 am, Bruno Desthuilliers
<bdesth.quelquech... at free.quelquepart.fr> wrote:
> cdle... at gmail.com a écrit :
> (snip)
>
> > Last post -- I swear.
>
> > I failed to realize that it's all part of an extremely well defined
> > attribute resolution protocol, and handled via the descriptor
> > specification. Discussing descriptors was on the TODO list for the
> > type/class unification document, but there's a very fulfilling
> > explanation by Raymond Hettinger:http://users.rcn.com/python/download/Descriptor.htm
> > Also, the official doc is here:http://docs.python.org/ref/descriptors.html
>
> > Maybe the documentation for the property builtin should make reference
> > to the descriptor specification? If nobody thinks this is silly, I'll
> > submit a documentation patch in a few days.
>
> What to say ? As an old-time pyton user (well... 7+ years now) and
> meta-programming addict, it's nothing new to me, but the fact that you
> didn't realize it is certainly a clear indication that documentation is
> not really up to date, so yes, submitting a patch is certainly a pretty
> good idea.
>
> > Sorry for the spam -- hope someone besides me learns from it!
>
> What's to be learned is mostly that we (the whole community) should take
> more care of the doc (which I never contributed FWIW, so I'm the first
> one that should feel in guilt here).

Actually I was wrong about this! Of course, the problem was
insufficient test coverage :)

The key addition: assert spam.useful_attr == 12 # FAILS!!! I failed to
realize it in the last posts...

When you don't declare a descriptor as a class attribute the
descriptor protocol isn't invoked! The descriptor protocol looks in
the /class/ attributes -- it transforms spam.useful_attr to
type(spam).__dict__['useful_attr'].__get__(spam, type(spam)), the
caveat being /type(spam)/.

>From the doc: (http://users.rcn.com/python/download/Descriptor.htm)
The implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__ if provided.

The data descriptors are only looked up in the class attributes -- the
instance attributes are returned as-is, so the above assertion fails!

The addition of 'print spam.useful_attr' produces <property object at
0x[something]>, because the portion of the descriptor protocol that
invokes __get__ is does not cover instance attributes (as declared in
__init__).

So, we're back to the original problem, with a bit more insight. What
if we declare the useful_attr property as a class attribute in the
constructor after the LibraryClass initialization function? It only
works once! Once you set a class attribute on the first instantiation
it's obviously there the next time you try to instantiate. Now our
test looks like this:

def test():
    class _Fake(object):
        pass
    external_obj = _Fake()
    external_obj.proxy_useful_attr = 12
    spam = MyInheritedClass(external_obj)
    assert spam.useful_attr == 12
    eggs = MyInheritedClass(external_obj)
    assert eggs.useful_attr == 12

Which provides coverage for the INCORRECT class-attribute-on-
instantiation solution, since eggs won't be able to instantiate. The
solution which fails this test looks like this:

class MyInheritedClass(LibraryClass):
    # WRONG! Don't use this.
    def __init__(self, external_obj):
        LibraryClass.__init__(self)
        def get_useful_attr(would_be_self): # has to be unbound
            return would_be_self._external_obj.proxy_useful_attr
        MyInheritedClass.useful_attr = property(get_useful_attr)
        self._external_obj = external_obj

At this point it seems like the only viable option is to hack
getattribute to add special cases. To do this, we continue to use the
unbound method getter as our argument to property, but add the
getattribute hack which allows for specified instance attributes that
have the descriptor protocol invoked on them:

class MyInheritedClass(LibraryClass):
    # ...
    def __init__(self, external_obj):
        LibraryClass.__init__(self)
        def get_useful_attr(would_be_self):
            return would_be_self._external_obj.proxy_useful_attr
        self.useful_attr = property(fget=get_useful_attr)
        self._external_obj = external_obj

    def __getattribute__(self, attr_name):
        """
        Hacks getattribute to allow us to return desired instance
        attributes with the descriptor protocol.
        """
        instance_descrs = ['useful_attr']
        if attr_name in instance_descrs and attr_name in
self.__dict__:
            attr = self.__dict__[attr_name]
            return attr.__get__(self, type(self))
        return object.__getattribute__(self, attr_name)

This passes the above test, but obviously requires a hack -- is there
a more elegant way to do it?

- Chris