Add a way to test for a descriptor in Python Code
Greetings,
I am trying to emulate attribute lookup, and want to test if symbol found
in __dict__ is an attribute or not in python code (i.e.: does its have a
`tp_descr_get` slot?)
Reading the documentation carefully, I am supposed to test if the attribute
has a `.__get__` method; however, I see no way to test for this properly
(easily).
Currently the only way I know to test for this is (See code at end of this
message):
any('__get__' in m.__dict__ for m in type(v).__mro__)
Which seems terribly inefficient.
The documentation at:
https://docs.python.org/2/howto/descriptor.html
https://docs.python.org/3/howto/descriptor.html
Both says:
"""
For classes, the machinery is in type.__getattribute__() which transforms
B.x into B.__dict__['x'].__get__(None, B). In pure Python, it looks like:
def __getattribute__(self, key):
"Emulate type_getattro() in Objects/typeobject.c"
v = object.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v
"""
However, the call to `hasattr(v, '__get__')` appears to me to be incorrect.
The question is *NOT* whether 'v' has an attribute '__get__'; *BUT* whether
`v` has a symbol `__get__` in any of the classes in it's method resolution
order.
Looking at `type_getattro` in "Objects/typeobject.c" here:
https://github.com/python/cpython/blob/master/Objects/typeobject.c#L3177
Reads:
meta_get = Py_TYPE(meta_attribute)->tp_descr_get
So I really want to know if the `tp_descr_get` slot is set or not.
(Which is a different question than whether `v` has a `__get__` attribute).
The code below shows that:
1. The valid value of `Point.y` is
On Tue, Oct 30, 2018 at 04:40:40AM -0400, Joy Diamond wrote:
""" For classes, the machinery is in type.__getattribute__() which transforms B.x into B.__dict__['x'].__get__(None, B). In pure Python, it looks like:
def __getattribute__(self, key): "Emulate type_getattro() in Objects/typeobject.c" v = object.__getattribute__(self, key) if hasattr(v, '__get__'): return v.__get__(None, self) return v
""" [...] However, the call to `hasattr(v, '__get__')` appears to me to be incorrect.
I agree, but only because it fails to take into account that dunder methods like __get__ are only looked up on the class, not the instance. I believe a more accurate eumulation would be: if hasattr(type(v), '__get__'): return type(v).__get__(None, self) Actually, on further investigation, I think it ought to be: if inspect.hasattr_static(type(v), '__get__') except that there is no hasattr_static, there's only a getattr_static. So perhaps there ought to be a hasattr_static as well.
The question is *NOT* whether 'v' has an attribute '__get__'; *BUT* whether `v` has a symbol `__get__` in any of the classes in it's method resolution order.
What's the difference as you see it? -- Steve
Clarifications:
1. I miswrote part of my first post where I wrote "I want to test if
symbol found in __dict__ is an attribute or not in python code".
I meant to write "is a DESCRIPTOR" or not.
2. The example in https://docs.python.org/3/howto/descriptor.html for
reproducing `type.__getattribute__` has a second bug, in that it does not
look at the class inheritance properly. (See fixed example below named
`type_getattro` and based on the same function in the C code).
In the example below if you call ` __getattribute__(Child, 'x')` it will
incorrectly fail with "Catch `AttributeError: 'type' object has no
attribute 'x'`"
Responding to Steve:
On Tue, Oct 30, 2018 at 6:31 AM Steven D'Aprano
Actually, on further investigation, I think it ought to be:
if inspect.hasattr_static(type(v), '__get__')
except that there is no hasattr_static, there's only a getattr_static. So perhaps there ought to be a hasattr_static as well.
`inspect.hasattr_static` gets me half way there (but it still looks in two chains of inheritance, where I only want to look in one). In particular if the metaclass of `type(v)` has a `.__get__` method it will incorrectly find that. So it will still misidentify if an instance is a descriptor or not.
The question is *NOT* whether 'v' has an attribute '__get__'; *BUT* whether `v` has a symbol `__get__` in any of the classes in it's method resolution order.
What's the difference as you see it?
I want to be able to look in the method resolution order (one inheritance
chain).
`getattr` and `inspect.getattr_static` both look in two inheritance chains
(the instance & it's type; or the case of a class, the class and it's
metaclass).
I need to look in only one chain (and disable descriptors like
`inspect.getattr_static` does).
To put it succinctly: I am trying to reproduce the behavior of
`_PyType_Lookup` from "Objects/typeobject.c" (see example below).
Below is a full reproduction of `object.__getattribute__` and
`type.__getattribute__` based on reading the Python source code.
Note this reproduction of `type.__getattribute__` is much more accurate
than what is at:
https://docs.python.org/2/howto/descriptor.html
https://docs.python.org/3/howto/descriptor.html
Both of which need to be updated.
(This is not yet filed as a bug report; as first I am requesting a call to
something like `_PyType_Lookup` that is efficent; and once we agree on
that, we can created an updated reproduction of `type.__getattribute__`).
Thanks,
Joy Diamond.
#
# The following reproduces (and tests) `object.__getattribute__` and
`type.__getattribute__` based on reading the C source code.
#
absent = object()
def _PyType_Lookup(model, name):
'''Based on `_PyType_Lookup` in "Objects/typeobject.c"'''
mro = model.__mro__
if mro is None:
return absent
for m in mro:
symbol_table = m.__dict__
if name in symbol_table:
return symbol_table[name]
return absent
def lookup__tp_descr_get(model):
tp_descr_get = _PyType_Lookup(model, '__get__')
return tp_descr_get
def has__tp_descr_set(model):
tp_descr_set = _PyType_Lookup(model, '__set__')
return tp_descr_set is not absent
#
# Reproduction of `object.__getattribute__`
#
def PyObject_GenericGetAttr(instance, name):
'''Based on `PyObject_GenericGetAttr` in "Objects/object.c"'''
instance_type = type(instance)
descriptor = _PyType_Lookup(instance_type, name)
if descriptor is absent:
get = absent
else:
descriptor_type = type(descriptor)
get = lookup__tp_descr_get(descriptor_type)
if (get is not absent) and (has__tp_descr_set(descriptor_type)):
#
# "Data Descriptor" (a `__set__` method exists) has
precedence.
#
return get(descriptor, instance, instance_type)
if instance_type.__dictoffset__:
instance__mapping = instance.__dict__
if name in instance__mapping:
return instance__mapping[name]
if get is not absent:
return get(descriptor, instance, instance_type)
raise AttributeError("cannot find attribute `{}` in instance of
`{}`".format(name, instance_type.__name__))
#
# Reproduction of `type.__getattribute__`
#
def type_getattro(model, name):
'''Based on `type_getattro` in "Objects/type_object.c"'''
metatype = type(model)
descriptor = _PyType_Lookup(metatype, name)
if descriptor is absent:
get = absent
else:
descriptor_type = type(descriptor)
get = lookup__tp_descr_get(descriptor_type)
if (get is not absent) and (has__tp_descr_set(descriptor_type)):
#
# "Data Descriptor" (a `__set__` method exists) has
precedence.
#
return get(descriptor, instance, instance_type)
symbol = _PyType_Lookup(model, name)
if symbol is not absent:
#
# Implement descriptor functionality, if any
#
symbol_type = type(symbol)
symbol_get = lookup__tp_descr_get(symbol_type)
if symbol_get is not absent:
#
# None 2nd argument indicates the descriptor was
# found on the target object itself (or a base)
#
return symbol_get(symbol, None, model)
return symbol
if get is not absent:
return get(descriptor, instance, instance_type)
raise AttributeError("cannot find attribute `{}` in class
`{}`".format(name, instance_type.__name__))
#
# object Example
#
print('=== object example ===')
class Readonly_Descriptor(object):
__slots__ = ((
'value',
))
def __init__(self, value):
self.value = value
def __get__(self, object, object_type):
return self.value
class Data_Descriptor(object):
__slots__ = ((
'value',
))
def __init__(self, value):
self.value = value
def __get__(self, object, object_type):
return self.value
def __set__(self, object, value):
self.value = value
class Point(object):
__slots__ = ((
'__dict__',
'_y',
))
def __init__(self, x, y):
self.x = x
self.y = y
self._y = 1
p23 = Point(2, 3)
Point.x = Readonly_Descriptor(4)
Point.y = Data_Descriptor(5)
Point.z = Readonly_Descriptor(6)
p23.y = 7 # Uses the Data_Descriptor
print("p23.x: %d # p23.__dict__['x']; *IGNORES* ReadOnly_Descriptor" %
p23.x)
print("p23.y: %d # type(p23).__dict__['y'].__get__(p23, Point)" % p23.y)
print("p23.z: %d # type(p23).__dict__['z'].__get__(p23, Point)" % p23.z)
print('')
print("PyObject_GenericGetAttr(p23, 'x'): %d" %
PyObject_GenericGetAttr(p23, 'x'))
print("PyObject_GenericGetAttr(p23, 'y'): %d" %
PyObject_GenericGetAttr(p23, 'y'))
print("PyObject_GenericGetAttr(p23, 'z'): %d" %
PyObject_GenericGetAttr(p23, 'z'))
#
# Type Example
#
print('')
print('=== Type example ===')
def get_1(self, a, b):
return 1
def get_2(a, b):
return 2
class Descriptor(object):
__get__ = get_1
class Not_A_Descriptor(object):
def __init__(self):
self.__get__ = get_2
def __repr__(self):
return '
participants (2)
-
Joy Diamond
-
Steven D'Aprano