Understanding descriptors

[Bruno Desthuilliers]

Brian Allen Vanderburg II a écrit :
> I'm trying to better understand descriptors and I've got a few questions 
> still after reading some sites.  Here is what I 'think', but please let 
> me know if any of this is wrong as I'm sure it probably is.
> 
> 
> First when accessing an attribute on a class or instance it must be 
> found.  For an instance, it's __dict__ is first search.

Actually, this is not true - binding descriptors are searched (in the 
class and it's mro) _before_ the instance's __dict__ :

 >>> class Foo(object):
...     @apply
...     def bar():
...         def fget(self):
...             return self._bar
...         def fset(self, v):
...             self._bar = v
...         return property(**locals())
...
 >>> f = Foo()
 >>> f.bar = 42
 >>> f.bar
42
 >>> f.__dict__['bar'] = "boo"
 >>> f.bar
42
 >>> f.__dict__['bar']
'boo'
 >>>

Note that this works a bit differently for non-binding descriptors 
(descriptors that only implement the __get__ method), which are looked 
up after the instance's __dict__

So the lookup chain is:

1/ lookup the class and bases for a binding descriptor
2/ then lookup the instance's __dict__
3/ then lookup the class and bases for a non-binding descriptor or plain 
attribute
4/ then class __getattr__

Also and FWIW, there's a "step zero" : calls __getattribute__. All the 
above lookup mechanism is actually implemented by object.__getattribute__.

>  If not found 
> the class and base class __dict__ are searched.  For a class, the 
> __dict__ and base classes __dict__ are search.
> 
> If assigning, and the attribute is found and is not a descriptor

a binding descriptor

> or if 
> the attribute is not found, then the assignment will occur in the 
> __dict__ of the class or instance.  If it is found and is a descriptor, 

a binding descriptor

> then __set__ will be call.


> For reading, if the attribute is found and is a descriptor, __get__ will 
> be called, passing the object (if it is an instance) and class.  If it 
> is not a descriptor, the attribute will be returned directly.
> 
> Class methods are just functions:

In Python, "classmethod" has a definite meaning - it's a method that 
takes the class (and not the instance) as first argument. So your 
assertion should be "Methods are just functions which are attributes of 
the class". Which is still not quite true. Sure, the def statement 
always create function objects, regardless of whether it happens within 
a class statement or not. OTHO, what you get when looking up the 
corresponding attribute is definitly a method object. So the correct 
formulation here is (IMHO of course) that methods are implemented by 
functions that are attributes of the class. Also, this is not restricted 
to functions : any callable object correctly implementing the descriptor 
protocol can be used (cf the classmethod and staticmethod objects).

> class C(object):
>    def F(self):
>       pass
> 
> C.__dict__['F'] # function object ...

yes.

> But functions are descriptors:

yes.

> C.__dict__['F'].__get__ # method wrapper ...
> 
> def f1():
>    pass
> 
> f1.__get__ # method wrapper ...
> 
> When a lookup is done it uses this descriptor to make a bound or unbound 
> method:

yes.

> c=C()
> 
> C.F # unbound method object, expects explicit instance when calling the 
> function
> c.F # bound method object provides instance implicitly when calling the 
> function
> 
> This is also done when adding to the classes:

Indeed. Whether the def statement happens within the class statement or 
not is irrelevant here.

> C.f1 = f1
> 
> f1 # function
> C.f1 # unbound method
> c.f1 # bound method
> 
> To prevent this it has to be decorated so the descriptor doesn't cause 
> the binding:
> 
> C.f2 = staticmethod(f1)
> C.f2 # functon
> c.f2 # function

yes.

> Here is a question, why don't instance attributes do the same thing?

Because the descriptor protocol is only invoked on class attributes.

> c.f3 = f1
> c.f3 # function, not bound method
> 
> So it is not calling the __get__ method for c.f3  After it finds c.f3 in 
> c.__dict__, and since it has a getter, shouldn't it call the __get__ to 
> return the bound method.

Nope, cf above.

>  It is good that it doesn't I know, but I just 
> want to know why it doesn't from an implementation view.

If by "from an implementation view", you mean "how it works", then the 
answer is above. If your question was about "why this design choice", 
I'll leave the answer to someone else...