Would it possible to define abstract read/write properties with decorators?
I'm a big fan of the decorator syntax introduced in python-2.6 to define properties: class Foo(object): @property def bar(self): return 1 @bar.setter def bar(self, val): pass Lately I've been learning about ABCs at http://www.python.org/dev/peps/pep-3119/ and http://docs.python.org/py3k/library/abc.html . The documentation states that abstract read-only properties can be defined with the decorator syntax, but the "long-form" [bar=property(getbar, setbar)] property declaration is required for read/write properties. Would it be possible to support the decorator syntax for read/write properties in python-3.3? In python-3.2, this is valid code that (sort-of) produces an abstract read/write property: class Foo(metaclass=ABCMeta): @abstractproperty def bar(self): return 1 @bar.setter def bar(self, val): pass but subclasses of Foo can be instantiated even if they do not define a bar.setter: class Baz(Foo): @property def bar(self): return 1 baz=Baz() which must have been the reason for requiring the long-form property declaration syntax. It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported: class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass I thought that would work with Python-3.2, but Foo is instantiable even though there are abstractmethods. If python's property could be tweaked to recognize those abstract methods and raise the usual TypeError, then we could subclass the abstract base class Foo in the usual way: class Baz(Foo): @Foo.bar.getter def bar(self): return 2 this is not yet instantiable, Baz.bar.setter is still abstract... @bar.setter def bar(self, val): pass Now Baz could be instantiated. Is this feasible? Darren
On Sun, Mar 13, 2011 at 11:18 AM, Darren Dale <dsdale24@gmail.com> wrote: [...]
It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported:
class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass
I thought that would work with Python-3.2, but Foo is instantiable even though there are abstractmethods. If python's property could be tweaked to recognize those abstract methods and raise the usual TypeError, then we could subclass the abstract base class Foo in the usual way:
Here is a working example!: import abc class Property(object): def __init__(self, getter, setter=None): self._getter = getter self._setter = setter if (getattr(getter, '__isabstractmethod__', False) or getattr(setter, '__isabstractmethod__', False)): self.__isabstractmethod__ = True def __get__(self, instance, owner): if instance is None: return self return self._getter(instance) def __set__(self, instance, value): return self._setter(instance, value) def getter(self, func): return Property(func, self._setter) def setter(self, func): return Property(self._getter, func) class C(metaclass=abc.ABCMeta): @Property @abc.abstractmethod def x(self): return 1 @x.setter @abc.abstractmethod def x(self, val): pass try: c=C() except TypeError as e: print(e) class D(C): @C.x.getter def x(self): return 2 try: d=D() except TypeError as e: print(e) class E(D): @D.x.setter def x(self, val): pass print(E())
On Sun, Mar 13, 2011 at 12:49 PM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 11:18 AM, Darren Dale <dsdale24@gmail.com> wrote: [...]
It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported:
class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass
I thought that would work with Python-3.2, but Foo is instantiable even though there are abstractmethods. If python's property could be tweaked to recognize those abstract methods and raise the usual TypeError, then we could subclass the abstract base class Foo in the usual way:
Here is a working example!:
The modifications to "property" to better support abstract base classes using the decorator syntax and @abstractmethod (rather than @abstractproperty) are even simpler than I originally thought: class Property(property): def __init__(self, *args, **kwargs): super(Property, self).__init__(*args, **kwargs) for f in (self.fget, self.fset, self.fdel): if getattr(f, '__isabstractmethod__', False): self.__isabstractmethod__ = True break
class C(metaclass=abc.ABCMeta): @Property @abc.abstractmethod def x(self): return 1 @x.setter @abc.abstractmethod def x(self, val): pass
try: c=C() except TypeError as e: print(e)
class D(C): @C.x.getter def x(self): return 2
try: d=D() except TypeError as e: print(e)
class E(D): @D.x.setter def x(self, val): pass
print(E())
running this example yields: Can't instantiate abstract class C with abstract methods x Can't instantiate abstract class D with abstract methods x <__main__.E object at 0x212ee10> Wouldn't it be possible to include this in python-3.3? Darren
On Fri, Mar 18, 2011 at 10:29 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 12:49 PM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 11:18 AM, Darren Dale <dsdale24@gmail.com> wrote: [...]
It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported:
class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass
I thought that would work with Python-3.2, but Foo is instantiable even though there are abstractmethods. If python's property could be tweaked to recognize those abstract methods and raise the usual TypeError, then we could subclass the abstract base class Foo in the usual way:
Here is a working example!:
The modifications to "property" to better support abstract base classes using the decorator syntax and @abstractmethod (rather than @abstractproperty) are even simpler than I originally thought:
class Property(property):
def __init__(self, *args, **kwargs): super(Property, self).__init__(*args, **kwargs) for f in (self.fget, self.fset, self.fdel): if getattr(f, '__isabstractmethod__', False): self.__isabstractmethod__ = True break
class C(metaclass=abc.ABCMeta): @Property @abc.abstractmethod def x(self): return 1 @x.setter @abc.abstractmethod def x(self, val): pass
try: c=C() except TypeError as e: print(e)
class D(C): @C.x.getter def x(self): return 2
try: d=D() except TypeError as e: print(e)
class E(D): @D.x.setter def x(self, val): pass
print(E())
running this example yields:
Can't instantiate abstract class C with abstract methods x Can't instantiate abstract class D with abstract methods x <__main__.E object at 0x212ee10>
Wouldn't it be possible to include this in python-3.3?
Sounds good to me. -- --Guido van Rossum (python.org/~guido)
On Fri, Mar 18, 2011 at 2:36 PM, Guido van Rossum <guido@python.org> wrote:
On Fri, Mar 18, 2011 at 10:29 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 12:49 PM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 11:18 AM, Darren Dale <dsdale24@gmail.com> wrote: [...]
It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported:
class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass [...] The modifications to "property" to better support abstract base classes using the decorator syntax and @abstractmethod (rather than @abstractproperty) are even simpler than I originally thought:
class Property(property):
def __init__(self, *args, **kwargs): super(Property, self).__init__(*args, **kwargs) for f in (self.fget, self.fset, self.fdel): if getattr(f, '__isabstractmethod__', False): self.__isabstractmethod__ = True break
class C(metaclass=abc.ABCMeta): @Property @abc.abstractmethod def x(self): return 1 @x.setter @abc.abstractmethod def x(self, val): pass
try: c=C() except TypeError as e: print(e)
class D(C): @C.x.getter def x(self): return 2
try: d=D() except TypeError as e: print(e)
class E(D): @D.x.setter def x(self, val): pass
print(E())
running this example yields:
Can't instantiate abstract class C with abstract methods x Can't instantiate abstract class D with abstract methods x <__main__.E object at 0x212ee10>
Wouldn't it be possible to include this in python-3.3?
Sounds good to me.
I took a stab at this, but unfortunately I have not been able to perform a complete build of python from the mercurial checkout on either ubuntu 11.04 or OS X 10.6.6, for reasons that appear unrelated to the changes below (undefined setlocale symbols on OS X, Could not find platform dependent libraries <exec_prefix> segfault on ubuntu). I'm an experienced python programmer, but not an experienced python hacker. Would anyone care to comment on (or test) the changes?: diff -r e34b09c69dd3 Objects/descrobject.c --- a/Objects/descrobject.c Sat Mar 12 22:31:06 2011 -0500 +++ b/Objects/descrobject.c Sat Mar 19 11:22:14 2011 -0400 @@ -1117,6 +1121,7 @@ PyObject *prop_set; PyObject *prop_del; PyObject *prop_doc; + PyObject *prop_isabstract; int getter_doc; } propertyobject; @@ -1128,6 +1133,8 @@ {"fset", T_OBJECT, offsetof(propertyobject, prop_set), READONLY}, {"fdel", T_OBJECT, offsetof(propertyobject, prop_del), READONLY}, {"__doc__", T_OBJECT, offsetof(propertyobject, prop_doc), READONLY}, + {"__isabstractmethod__", T_OBJECT, + offsetof(propertyobject, prop_isabstract), READONLY}, {0} }; @@ -1180,6 +1187,7 @@ Py_XDECREF(gs->prop_set); Py_XDECREF(gs->prop_del); Py_XDECREF(gs->prop_doc); + Py_XDECREF(gs->prop_isabstract); self->ob_type->tp_free(self); } @@ -1213,7 +1221,7 @@ PyErr_SetString(PyExc_AttributeError, value == NULL ? "can't delete attribute" : - "can't set attribute"); + "can't set attribute"); return -1; } if (value == NULL) @@ -1263,6 +1271,21 @@ return new; } +static void +property_identify_abstract_method(PyObject *self, PyObject *method) +{ + /* Set self.__isabstractmethod__ if method is abstract */ + if (method != NULL){ + PyObject *is_abstract = PyObject_GetAttrString(method, + "__isabstractmethod__"); + if (PyObject_IsTrue(is_abstract) > 0){ + Py_INCREF(Py_True); + PyObject_SetAttrString(self, "__isabstractmethod__", Py_True); + } + Py_DECREF(is_abstract); + } +} + static int property_init(PyObject *self, PyObject *args, PyObject *kwds) { @@ -1285,11 +1308,13 @@ Py_XINCREF(set); Py_XINCREF(del); Py_XINCREF(doc); + Py_INCREF(Py_False); prop->prop_get = get; prop->prop_set = set; prop->prop_del = del; prop->prop_doc = doc; + prop->prop_isabstract = Py_False; prop->getter_doc = 0; /* if no docstring given and the getter has one, use that one */ @@ -1320,6 +1345,11 @@ } } + /* set __isabstractmethod__ if fget, fset, or fdel are abstract methods */ + property_identify_abstract_method(self, get); + property_identify_abstract_method(self, set); + property_identify_abstract_method(self, del); + return 0; }
Thanks much for your contribution! In order to get it reviewed and submitted, can you please create a bug for this issue (mention the python-ideas thread), upload your patch there, and perhaps ping python-dev? --Guido On Sat, Mar 19, 2011 at 8:29 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Fri, Mar 18, 2011 at 2:36 PM, Guido van Rossum <guido@python.org> wrote:
On Fri, Mar 18, 2011 at 10:29 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 12:49 PM, Darren Dale <dsdale24@gmail.com> wrote:
On Sun, Mar 13, 2011 at 11:18 AM, Darren Dale <dsdale24@gmail.com> wrote: [...]
It seems like it should be possible for Python to support the decorator syntax for declaring abstract read/write properties. The most elegant approach might be the following, if it could be supported:
class Foo(metaclass=ABCMeta): # Note the use of @property rather than @abstractproperty: @property @abstractmethod def bar(self): return 1 @bar.setter @abstractmethod def bar(self, val): pass [...] The modifications to "property" to better support abstract base classes using the decorator syntax and @abstractmethod (rather than @abstractproperty) are even simpler than I originally thought:
class Property(property):
def __init__(self, *args, **kwargs): super(Property, self).__init__(*args, **kwargs) for f in (self.fget, self.fset, self.fdel): if getattr(f, '__isabstractmethod__', False): self.__isabstractmethod__ = True break
class C(metaclass=abc.ABCMeta): @Property @abc.abstractmethod def x(self): return 1 @x.setter @abc.abstractmethod def x(self, val): pass
try: c=C() except TypeError as e: print(e)
class D(C): @C.x.getter def x(self): return 2
try: d=D() except TypeError as e: print(e)
class E(D): @D.x.setter def x(self, val): pass
print(E())
running this example yields:
Can't instantiate abstract class C with abstract methods x Can't instantiate abstract class D with abstract methods x <__main__.E object at 0x212ee10>
Wouldn't it be possible to include this in python-3.3?
Sounds good to me.
I took a stab at this, but unfortunately I have not been able to perform a complete build of python from the mercurial checkout on either ubuntu 11.04 or OS X 10.6.6, for reasons that appear unrelated to the changes below (undefined setlocale symbols on OS X, Could not find platform dependent libraries <exec_prefix> segfault on ubuntu). I'm an experienced python programmer, but not an experienced python hacker. Would anyone care to comment on (or test) the changes?:
diff -r e34b09c69dd3 Objects/descrobject.c --- a/Objects/descrobject.c Sat Mar 12 22:31:06 2011 -0500 +++ b/Objects/descrobject.c Sat Mar 19 11:22:14 2011 -0400 @@ -1117,6 +1121,7 @@ PyObject *prop_set; PyObject *prop_del; PyObject *prop_doc; + PyObject *prop_isabstract; int getter_doc; } propertyobject;
@@ -1128,6 +1133,8 @@ {"fset", T_OBJECT, offsetof(propertyobject, prop_set), READONLY}, {"fdel", T_OBJECT, offsetof(propertyobject, prop_del), READONLY}, {"__doc__", T_OBJECT, offsetof(propertyobject, prop_doc), READONLY}, + {"__isabstractmethod__", T_OBJECT, + offsetof(propertyobject, prop_isabstract), READONLY}, {0} };
@@ -1180,6 +1187,7 @@ Py_XDECREF(gs->prop_set); Py_XDECREF(gs->prop_del); Py_XDECREF(gs->prop_doc); + Py_XDECREF(gs->prop_isabstract); self->ob_type->tp_free(self); }
@@ -1213,7 +1221,7 @@ PyErr_SetString(PyExc_AttributeError, value == NULL ? "can't delete attribute" : - "can't set attribute"); + "can't set attribute"); return -1; } if (value == NULL) @@ -1263,6 +1271,21 @@ return new; }
+static void +property_identify_abstract_method(PyObject *self, PyObject *method) +{ + /* Set self.__isabstractmethod__ if method is abstract */ + if (method != NULL){ + PyObject *is_abstract = PyObject_GetAttrString(method, + "__isabstractmethod__"); + if (PyObject_IsTrue(is_abstract) > 0){ + Py_INCREF(Py_True); + PyObject_SetAttrString(self, "__isabstractmethod__", Py_True); + } + Py_DECREF(is_abstract); + } +} + static int property_init(PyObject *self, PyObject *args, PyObject *kwds) { @@ -1285,11 +1308,13 @@ Py_XINCREF(set); Py_XINCREF(del); Py_XINCREF(doc); + Py_INCREF(Py_False);
prop->prop_get = get; prop->prop_set = set; prop->prop_del = del; prop->prop_doc = doc; + prop->prop_isabstract = Py_False; prop->getter_doc = 0;
/* if no docstring given and the getter has one, use that one */ @@ -1320,6 +1345,11 @@ } }
+ /* set __isabstractmethod__ if fget, fset, or fdel are abstract methods */ + property_identify_abstract_method(self, get); + property_identify_abstract_method(self, set); + property_identify_abstract_method(self, del); + return 0; } _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (python.org/~guido)
On Sat, Mar 19, 2011 at 12:24 PM, Guido van Rossum <guido@python.org> wrote:
Thanks much for your contribution! In order to get it reviewed and submitted, can you please create a bug for this issue (mention the python-ideas thread), upload your patch there, and perhaps ping python-dev?
I did so, at http://bugs.python.org/issue11610 . The first reviewer directed me to discussion concerning the implementation of abstract classmethods at http://bugs.python.org/issue5867 . In that case, you objected to extending the implementation of classmethod to allow assigning to an __isabstractmethod__ attribute, which would have allowed the same syntax I suggested, combining @abstractmethod and @classmethod. The patch I made (which doesn't work yet) also attempts to extend the implementation of a builtin. Do you still object to this approach? There are two somewhat related issues: * api: The first reviewer objects to using a single decorator for methods (regular, class, and static) but a combination of two decorators for properties. * implementation: The current abc.abstractproperty has some issues beyond not supporting the decorator syntax, which are due to the fact that properties are composite objects, and it is the methods which compose the property that should imbue "abstractness". - To provide an implementation for an abstract property, one currently has to completely respecify a concrete property and rebind it. If an ABC defines an abstract read/write property and a subclass mistakenly redefines it as a read-only property, the ABC mechanisms will not catch the error. - I can imagine cases where an abstract base class may define a simple concrete getter but an abstract setter. This is not possible with the current abstractproperty. - An abstractproperty cannot be made concrete through the use of the decorators: class D(MyABC): @MyABC.my_abstract_property.setter def my_abstract_property(self): ... because @MyABC.my_abstract_property.setter returns another instance of abstractproperty. I think the general approach I suggested resolves all of these issues. If you have reservations about extending builtins, an alternative might be to improve the definition of abstractproperty so it supports the features and addresses the issues we have been discussing, and has decorators implemented such that once all of the abstract methods have been replaced with concrete ones, they return an instance of the built-in property rather than abc.abstractproperty. Does this sound like it could be an acceptable alternative? Darren
It looks like you have moved on to a different strategy; let me comment on the code review instead. On Sun, Mar 20, 2011 at 7:06 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Sat, Mar 19, 2011 at 12:24 PM, Guido van Rossum <guido@python.org> wrote:
Thanks much for your contribution! In order to get it reviewed and submitted, can you please create a bug for this issue (mention the python-ideas thread), upload your patch there, and perhaps ping python-dev?
I did so, at http://bugs.python.org/issue11610 . The first reviewer directed me to discussion concerning the implementation of abstract classmethods at http://bugs.python.org/issue5867 . In that case, you objected to extending the implementation of classmethod to allow assigning to an __isabstractmethod__ attribute, which would have allowed the same syntax I suggested, combining @abstractmethod and @classmethod. The patch I made (which doesn't work yet) also attempts to extend the implementation of a builtin. Do you still object to this approach? There are two somewhat related issues:
* api: The first reviewer objects to using a single decorator for methods (regular, class, and static) but a combination of two decorators for properties. * implementation: The current abc.abstractproperty has some issues beyond not supporting the decorator syntax, which are due to the fact that properties are composite objects, and it is the methods which compose the property that should imbue "abstractness". - To provide an implementation for an abstract property, one currently has to completely respecify a concrete property and rebind it. If an ABC defines an abstract read/write property and a subclass mistakenly redefines it as a read-only property, the ABC mechanisms will not catch the error. - I can imagine cases where an abstract base class may define a simple concrete getter but an abstract setter. This is not possible with the current abstractproperty. - An abstractproperty cannot be made concrete through the use of the decorators:
class D(MyABC): @MyABC.my_abstract_property.setter def my_abstract_property(self): ...
because @MyABC.my_abstract_property.setter returns another instance of abstractproperty.
I think the general approach I suggested resolves all of these issues. If you have reservations about extending builtins, an alternative might be to improve the definition of abstractproperty so it supports the features and addresses the issues we have been discussing, and has decorators implemented such that once all of the abstract methods have been replaced with concrete ones, they return an instance of the built-in property rather than abc.abstractproperty. Does this sound like it could be an acceptable alternative?
Darren
-- --Guido van Rossum (python.org/~guido)
participants (2)
-
Darren Dale -
Guido van Rossum