[Python-ideas] The Descriptor Protocol...

Raymond Hettinger raymond.hettinger at gmail.com
Wed Mar 2 20:35:00 CET 2011

On Mar 2, 2011, at 9:09 AM, Martin Chilvers wrote:

> Hi Raymond,
> On 02/03/2011 16:10, Raymond Hettinger wrote:
>> I've gotten this question a couple times. There usual answers are:
>> * Why? Because Guido implemented it that way ;-)
>> * Why did he pick a signature that dropped the "key"? Mostly likely, it
>> is because none of the use-cases he had in mind required it. The keyless
>> API suffices to implement property(), super(), classmethod(),
>> staticmethod(), bound method, __slots__, various features of new-style
>> classes, and a reimplementation old-style classes.
> Probably so, but it still smells in terms of the information flow through the various levels of the API :^)

Maybe.  You'll have to take it up with Guido.  He invented it.  I just documented it.

A huge class of problems he was trying to solve required only a hammer and screwdriver, not the whole toolbox ;-)

>> Yes, option b is the usual way to do it (there are a number of recipes
>> that use this approach).
>> For the most part, that shouldn't be an unfamiliar pattern to Python
>> developers. For example, named tuples repeat the name:
>> Point = namedtuple('Point', ('x', 'y')).
>> Sometimes the language provides syntax to do the work for us, like
>> "class A: ..." instead of "A = type('A', (), {})", but for other
>> situations, it isn't unusual to pass in a name as a string literal so
>> that an object will know its own name.
>> For Python 4, perhaps you can convince Guido to add a key argument to
>> the signature for __get__ and __set__. It would make a handful of
>> recipes a bit prettier at the expense of being a total waste for all the
>> other use cases that don't need it.
> Just because some use cases don't use it doesn't mean it is a total waste ;^)

Okay, let's say "mostly wasted", meaning that all the common use cases (just about everything currently implemented with descriptors) would pay a price (an extra argument being passed around) in order to benefit rare use cases (none of the recipes needing a key have yet found their way into the standard library despite having been around since Py2.2).

> The current design mean that descriptors can't be (sensibly) shared across differently named attributes which has major implications for scaleability...
>> I think everyone who writes a descriptor that needs the "key" will chafe
>> at the current design. It bugged me at first, but the workaround is easy
>> and after a while it doesn't seem as bothersome.
> Yep - workarounds generally are easy, but if the workarounds appear often enough its usually a symptom that the underlying code might be improved...

"... might be improved" suggests that the API can be changed without breaking every piece of descriptor code currently in existence.  Until Python4.0, the point is moot.  So, a better phrasing might be, "oh i wish the API had been originally designed differently".

Also, I don't agree with the antecedent, "if the workarounds appear often enough".  I'm a heavy user of descriptors and have needed this only twice; both times, the workaround idiom sufficed for getting the job done.  The implementation of CPython is itself a heavy user of descriptors and has not needed this functionality even once.

All that being said, if the key were passed along, I would find uses for it.   So, I sympathize with your post.


P.S.  Python is very customizable.  The descriptor protocol in implemented by __getattribute__, so it's not a hard exercise to write your own __getattribute__ to implement your own variant of the descriptor protocol.  Here's a quick and dirty example:

class Str:
    'Descriptor using a custom protocol'
    def __my_get__(self, obj, key):
        print(key * 5)
        return obj

class A:

    @property        # old protocol
    def x(self):
        return 10

    y = Str()        # new protocol

    def __getattribute__(self, key):
        'Implement an alternative descriptor protocol'
        attr = object.__getattribute__(self, key)
        if hasattr(attr, '__my_get__'):
            return attr.__my_get__(attr, key)
        return attr

a = A()

More information about the Python-ideas mailing list