Read descriptors (defining only __get__) and data descripttors have (defining __get__ and __set__) different semantics. In particular, read descriptors are overridden by instance data, while data descriptors are not. A common use of read descriptors is for lazily computed data: class readproperty(object): "Create a read descriptor from a function" def __init__(self, func): self.func = func def __get__(self, inst, class_): if inst is None: return self return self.func(inst) class Spam: @readproperty def eggs(self): ... expensive computation of eggs self.eggs = result return result When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor. I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5. Thoughts? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On 9/28/05, Jim Fulton <jim@zope.com> wrote:
Read descriptors (defining only __get__) and data descripttors have (defining __get__ and __set__) different semantics. In particular, read descriptors are overridden by instance data, while data descriptors are not. A common use of read descriptors is for lazily computed data:
class readproperty(object): "Create a read descriptor from a function"
def __init__(self, func): self.func = func
def __get__(self, inst, class_): if inst is None: return self
return self.func(inst)
class Spam:
@readproperty def eggs(self): ... expensive computation of eggs
self.eggs = result return result
When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor.
I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5.
Thoughts?
I like the idea of adding this functionality somewhere. I do note that the semantics were somewhat of an accident, required by backwards compatibility with pre-2.2 method lookup semantics. I think we need to be real careful with chosing a name -- in Jim's example, *anyone* could assign to Spam().eggs to override the value. The name "readproperty" is too close to "readonlyproperty", but read-only it ain't! "Lazy" also doesn't really describe what's going on. I believe some folks use a concept of "memo functions" which resemble this proposal except the notation is different: IIRC a memo function is always invoked as a function, but stores its result in a private instance variable, which it returns upon subsequent calls. This is a common pattern. Jim's proposal differs because the access looks like an attribute, not a method call. Still, perhaps memoproperty would be a possible name. Another way to look at the naming problem is to recognize that the provided function really computes a default value if the attribute isn't already set. So perhaps defaultproperty? (Sorry to turn this into a naming game, which is bound to produce 100s of competing proposals that are totally unintuitive except to the proposer. But naming is important.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
I think we need to be real careful with chosing a name
In Eiffel, the keyword "once" is used for something analogous -- a method that is called once the first time it's referenced, and the return value cached. So perhaps this could be called a "once_property". -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
At 01:34 PM 9/29/2005 +1200, Greg Ewing wrote:
Guido van Rossum wrote:
I think we need to be real careful with chosing a name
In Eiffel, the keyword "once" is used for something analogous -- a method that is called once the first time it's referenced, and the return value cached.
So perhaps this could be called a "once_property".
I was actually going to propose that before I thought of defaultproperty, and in fact my equivalent descriptor was originally called "Once", but the truth is that it gets called every time you delete it and reuse it, whereas Eiffel's once functions are called once, period, with no way to reset them. (At least, IIRC.)
Guido van Rossum wrote:
On 9/28/05, Jim Fulton <jim@zope.com> wrote:
...
I think we need to be real careful with chosing a name -- in Jim's example, *anyone* could assign to Spam().eggs to override the value. The name "readproperty" is too close to "readonlyproperty",
In fact, property creates read-only properties for new-style classes. (I hadn't realized, until reading this thread, that for classic classes, you could still set the attribute.)
but read-only it ain't! "Lazy" also doesn't really describe what's going on.
I agree.
I believe some folks use a concept of "memo functions" which resemble this proposal except the notation is different: IIRC a memo function is always invoked as a function, but stores its result in a private instance variable, which it returns upon subsequent calls. This is a common pattern. Jim's proposal differs because the access looks like an attribute, not a method call. Still, perhaps memoproperty would be a possible name.
Another way to look at the naming problem is to recognize that the provided function really computes a default value if the attribute isn't already set. So perhaps defaultproperty?
Works for me. Oleg Broytmann wrote:
On Wed, Sep 28, 2005 at 10:16:12AM -0400, Jim Fulton wrote:
class readproperty(object):
[skip]
I do this often enough
I use it since about 2000 often enough under the name CachedAttribute:
Steven Bethard wrote:
Jim Fulton wrote:
...
I've also needed behavior like this a few times, but I use a variant of Scott David Daniel's recipe[1]:
class _LazyAttribute(object):
Yup, the Zope 3 sources have something very similar: http://svn.zope.org/Zope3/trunk/src/zope/cachedescriptors/property.py?view=m... I actually think this does too much. All it saves me, compared to what I proposed is one assignment. I'd rather make that assignment explicit. Anyway, all I wanted with readproperty was a property that implemented only __get__, as opposed to property, which implements __get__, __set__, and __delete__. I'd be happy to call it readproprty or getproperty or defaulproperty or whatever. :) I'd prefer that it's semantics stay fairly simple though. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Based on the discussion, I think I'd go with defaultproperty. Questions: - Should this be in builtins, alongside property, or in a library module? (Oleg suggested propertytools.) - Do we need a short PEP? Jim Jim Fulton wrote:
Guido van Rossum wrote:
On 9/28/05, Jim Fulton <jim@zope.com> wrote:
...
I think we need to be real careful with chosing a name -- in Jim's example, *anyone* could assign to Spam().eggs to override the value. The name "readproperty" is too close to "readonlyproperty",
In fact, property creates read-only properties for new-style classes. (I hadn't realized, until reading this thread, that for classic classes, you could still set the attribute.)
but
read-only it ain't! "Lazy" also doesn't really describe what's going on.
I agree.
I believe some folks use a concept of "memo functions" which resemble this proposal except the notation is different: IIRC a memo function is always invoked as a function, but stores its result in a private instance variable, which it returns upon subsequent calls. This is a common pattern. Jim's proposal differs because the access looks like an attribute, not a method call. Still, perhaps memoproperty would be a possible name.
Another way to look at the naming problem is to recognize that the provided function really computes a default value if the attribute isn't already set. So perhaps defaultproperty?
Works for me.
Oleg Broytmann wrote:
On Wed, Sep 28, 2005 at 10:16:12AM -0400, Jim Fulton wrote:
class readproperty(object):
[skip]
I do this often enough
I use it since about 2000 often enough under the name CachedAttribute:
Steven Bethard wrote:
Jim Fulton wrote:
...
I've also needed behavior like this a few times, but I use a variant of Scott David Daniel's recipe[1]:
class _LazyAttribute(object):
Yup, the Zope 3 sources have something very similar:
http://svn.zope.org/Zope3/trunk/src/zope/cachedescriptors/property.py?view=m...
I actually think this does too much. All it saves me, compared to what I proposed is one assignment. I'd rather make that assignment explicit.
Anyway, all I wanted with readproperty was a property that implemented only __get__, as opposed to property, which implements __get__, __set__, and __delete__.
I'd be happy to call it readproprty or getproperty or defaulproperty or whatever. :)
I'd prefer that it's semantics stay fairly simple though.
Jim
-- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
Based on the discussion, I think I'd go with defaultproperty.
Questions:
- Should this be in builtins, alongside property, or in a library module? (Oleg suggested propertytools.)
- Do we need a short PEP?
The much-discussed never-created decorators module, perhaps? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
On 10/9/05, Jim Fulton <jim@zope.com> wrote:
Based on the discussion, I think I'd go with defaultproperty.
Great.
Questions:
- Should this be in builtins, alongside property, or in a library module? (Oleg suggested propertytools.)
- Do we need a short PEP?
I think so. From the responses I'd say there's at most lukewarm interest (including from me). You might also want to drop it and just add it to your personal (or Zope's) library. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
On 10/9/05, Jim Fulton <jim@zope.com> wrote:
Based on the discussion, I think I'd go with defaultproperty.
Great.
Questions:
- Should this be in builtins, alongside property, or in a library module? (Oleg suggested propertytools.)
- Do we need a short PEP?
I think so. From the responses I'd say there's at most lukewarm interest (including from me).
Hm, I saw several responses from people who'd built something quite similar. This suggests to me that this is a common need.
You might also want to drop it and just add it to your personal (or Zope's) library.
I have something like this in Zope's library. I end up with a very small package that isn't logically part of other packages, but that is a dependency of lots of packages. I don't like that, but I guess I should get over it. I must say that I am of 2 minds about things like this. On the one hand, I'd like Python's standard library to be small with packaging systems to provide "extra batteries". OTOH, I often find small tools like this that would be nice to have readily available. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
At 10:16 AM 9/28/2005 -0400, Jim Fulton wrote:
I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5.
Thoughts?
You mean something like defaultproperty(func), where func(ob) is called at most once when there is no dictionary entry for the attribute? I started using such properties with PEAK a few years ago, and found some corner cases that made me decide to stick with ones that have both __get__ and __set__. Mostly those cases have to do with creating class-level properties, which can end up being inherited by subclasses if you don't include __set__. Also of course you can't hook the setting of attributes without a __set__. Of course, most people aren't likely to be creating metaclass properties, so it's probably not a big issue for the stdlib. But I thought *you* might want to know about it, in case you hadn't already encountered the issue. :) The other issue I found with such properties is that I really wanted to be able to use functions that didn't rebind the attribute value directly, i.e., I wanted to be able to use lambdas for short computed property descriptions. However, to make this work you have to know what attribute name(s) the property is stored under in the class, which then leads to other interesting complications. So now I use a custom C type that knows its name and takes two functions (a filter for values set, and a function to compute the default). Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice if there were a descriptor hook __bind__(cls,name) that was called by classes during cls.__new__ or assignment to a class attribute, and which you could define to return a replacement descriptor. It seems like one of the first metaclasses I end up writing in any new project is something to do this, and I believe Ian Bicking has encountered the same thing in e.g. SQLObject.
"Phillip J. Eby" <pje@telecommunity.com> writes:
Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice if there were a descriptor hook __bind__(cls,name) that was called by classes during cls.__new__ or assignment to a class attribute, and which you could define to return a replacement descriptor. It seems like one of the first metaclasses I end up writing in any new project is something to do this, and I believe Ian Bicking has encountered the same thing in e.g. SQLObject.
I've done this many times too, but I've never really felt the need to propose a change to Python for it. I guess one could modify the descriptor protocol slightly, but is it worth it? Hmm, dunno. Cheers, mwh -- First time I've gotten a programming job that required a drug test. I was worried they were going to say "you don't have enough LSD in your system to do Unix programming". -- Paul Tomblin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
Michael Hudson <mwh <at> python.net> writes:
"Phillip J. Eby" <pje <at> telecommunity.com> writes:
Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice if there were a descriptor hook __bind__(cls,name) that was called by classes during cls.__new__ or assignment to a class attribute, and which you could define to return a replacement descriptor. It seems like one of the first metaclasses I end up writing in any new project is something to do this, and I believe Ian Bicking has encountered the same thing in e.g. SQLObject.
I've done this many times too, but I've never really felt the need to propose a change to Python for it. I guess one could modify the descriptor protocol slightly, but is it worth it? Hmm, dunno.
Cheers, mwh
You can use something like this to find a descriptor's name: class Private(property): def __init__(self, permission, fget=None, fset=None, fdel=None, doc=None): fget = fget or 'r' in permission and self.default_fget or None fset = fset or 'w' in permission and self.default_fset or None fdel = fdel or 'd' in permission and self.default_fget or None super(private, self).__init__(fget, fset, fdel, doc) def get_attribute_name(self, instance): my_name = None for cls in instance.__class__.mro(): for attribute_name, attribute_object in cls.__dict__.iteritems(): if attribute_object is self: my_name = attribute_name break if my_name is not None: class_name = cls.__name__ break attribute_name = '_%s__%s' % (class_name, my_name) self.attribute_name = attribute_name return attribute_name def default_fget(self, instance): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) return getattr(instance, attribute_name) def default_fset(self, instance, value): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) setattr(instance, attribute_name, value) def default_fdel(self, instance): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) delattr(instance, attribute_name)
At 05:28 PM 9/28/2005 +0000, iga Seilnacht wrote:
You can use something like this to find a descriptor's name: <snip>
The given code fails if the same property appears under more than one name or is used in more than one class. It also requires the property object to be mutable, and is subject to inter-thread race conditions in the case of modification of a class. It also doesn't work for class-level descriptors (added to the class' class), and I'd prefer it to use the logical equivalent of ob.__dict__.setdefault() rather than setattr(ob,...) so that it's free of (simple) race conditions in the case where one thread sets the attribute while another is computing the default value. While these aren't crippling limitations in a given application, I think the stdlib implementation should be a bit more robust, especially since none of these features is very hard to implement, once you know what's needed. (And I already have a robust implementation that could be cribbed from.) In any case, having a stdlib version that doesn't address those issues wouldn't be very useful for me, as it wouldn't allow me to replace my custom C descriptor and metaclass. If we added __bind__ (or something like it) to the descriptor protocol I could get rid of *at least* that metaclass, and maybe others.
Jim Fulton wrote:
A common use of read descriptors is for lazily computed data:
class readproperty(object): "Create a read descriptor from a function"
def __init__(self, func): self.func = func
def __get__(self, inst, class_): if inst is None: return self
return self.func(inst)
class Spam:
@readproperty def eggs(self): ... expensive computation of eggs
self.eggs = result return result
I've also needed behavior like this a few times, but I use a variant of Scott David Daniel's recipe[1]: class _LazyAttribute(object): def __init__(self, calculate_function): self._calculate = calculate_function def __get__(self, obj, _=None): if obj is None: return self try: value = self._calculate(obj) except AttributeError, e: # I don't like this, but if _calculate raises an # AttributeError and I don't catch it, the descriptor # machinery hides it and I can't debug my code raise Exception(e) setattr(obj, self._calculate.func_name, value) return value It uses the .func_name attribute to put the "self.eggs = result" into the property. I like that I don't have to do the set at the end of every function, and I'm never doing anything complicated enough that I don't want the attribute named the same as the function that I passed in. [1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/363602 STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy
On Wed, Sep 28, 2005 at 10:16:12AM -0400, Jim Fulton wrote:
class readproperty(object): [skip] I do this often enough
I use it since about 2000 often enough under the name CachedAttribute: http://cvs.sourceforge.net/viewcvs.py/ppa/qps/qUtils.py (The current maintainer of the code is Denis Otkidach, http://sourceforge.net/users/ods/ .) It seems many people reinvent this particular wheel. Which is understandable. propertytools.py, anyone? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
It seems many people reinvent this particular wheel. Which is understandable.
propertytools.py, anyone?
"import magic" ;) Cheers, Nick. P.S. Such a module may actually be a good idea, if it reduces the variation in metaclass and descriptor hacks across some of the more complex framework. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
On Wed, 2005-09-28 at 10:16, Jim Fulton wrote:
When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor.
I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5.
I /must/ be missing something. Why not just use property as a decorator? class C: @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs
c = C() c.eggs in eggs 7 c.eggs 7
-Barry
It doesn't work that way for new-style classes. On 9/28/05, Barry Warsaw <barry@python.org> wrote:
On Wed, 2005-09-28 at 10:16, Jim Fulton wrote:
When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor.
I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5.
I /must/ be missing something. Why not just use property as a decorator?
class C: @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs
c = C() c.eggs in eggs 7 c.eggs 7
-Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux)
iQCVAwUAQzsX03EjvBPtnXfVAQK43QP/dRjW11myDXRdjXcPPuaxRQ2qtUlMyAJG 26sedhmrF00rvKVh7U0RaGJ/Cq5iwgEbQRmXm1pbS8UKzNZxz55qGjVDXjp7Rwgr KJpJzz/UWVqVClRJJGDdgasRO8GUfxTYh2YPrmXaTDPLh3uscIwpwq1oapT1R4OH 6xJYLrjAs9M= =mCyw -----END PGP SIGNATURE-----
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
At 06:23 PM 9/28/2005 -0400, Barry Warsaw wrote:
I /must/ be missing something. Why not just use property as a decorator?
class C: @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs
c = C() c.eggs in eggs 7 c.eggs 7
Because it only works in classic classes due to a bug in descriptor handling:
class C(object): @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs
c=C() c.eggs in eggs
Traceback (most recent call last): File "<pyshell#12>", line 1, in -toplevel- c.eggs File "<pyshell#10>", line 4, in eggs self.eggs = 7 AttributeError: can't set attribute
participants (10)
-
Barry Warsaw -
Greg Ewing -
Guido van Rossum -
Jim Fulton -
Michael Hudson -
Nick Coghlan -
Oleg Broytmann -
Phillip J. Eby -
Steven Bethard -
Žiga Seilnacht