[Python-Dev] Hooking into super() attribute resolution

[Steve Dower]

The only real advantage is a simpler signature and more easily explained use (assuming the person you're explaining it to is familiar with metaclasses, so most of the hard explaining has been done).

I'm still not sure that this isn't simply a bug in super. If the superclass's metaclass provides a __getattr__ then it should probably use it and abandon it's own MRO traversal.

I still haven't thought the edge cases through, and it seems like there'd be some with that change, so that's where __getattribute_super__ comes in - super can call it without abandoning its MRO traversal.

AFAICT, the difference between that and __getlocalattribute__ is that the latter would be implemented on a metaclass while the former takes extra parameters. I think this functionality is advanced enough that requiring a metaclass isn't unreasonable.

(The proxy objects idea was a red herring, sorry :) )

Steve

Sent from my Windows Phone
________________________________
From: Ronald Oussoren<mailto:ronaldoussoren at mac.com>
Sent: ‎7/‎7/‎2013 12:37
To: Steve Dower<mailto:Steve.Dower at microsoft.com>
Cc: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Hooking into super() attribute resolution


On 7 Jul, 2013, at 17:17, Steve Dower <Steve.Dower at microsoft.com> wrote:

> Could the same result be achieved by hooking the MRO that super uses and returning a list of proxy objects?

What is the advantage over adding a hook to the class itself? That seems to be the right place to add such a hook, super already looks in the classes along the MRO and my proposal would add a formal interface for that instead of having super peek into the class __dict__. I have thought of using a custom mapping object for the tp_dict slot to intercept this, but that won't work because super assumes that tp_dict is an actual PyDictObject (and likely other parts of the interpreter do so as well).


> And then wouldn't you only really need a __getattribute__ that doesn't recurse (__getlocalattribute__)? The end result may be conceptually simpler, but you've thought through the edge cases better than I have.

__getattribute_super__ already is a kind of __getlocalattribute__, the primairy difference being __getattribute_super__ is a staticmethod instead of an instance method. To be honest I'm not sure if a staticmethod is the right solution, I'm having a hard time to determine if this should be a class, instance or static method.

Currently super(StartClass, x) basicly does (assuming x is an instance method):


def __getattribute__(self, name):
    mro = type(x).mro()
    idx = mro.index(StartClass)
    while idx < len(mro):
       dct = mro[idx].__dict__
       try:
          result = dct[name]
          # deal with descriptors here
          return result

       except KeyError:
           continue
    return object.__getattribute__(self, name)

With my proposal 'dct' would no longer be needed and 'result = dct[name]' would be 'mro[idx].__getattribute_super__(mro[idx], name, x, StartClass)' (I may have the last argument for the call to __getattribute_super__ wrong, but that's the idea). Given that the first argument of __get...super__ is the same as the object the method get getattr-ed from I guess the method should be a classmethod instead of an staticmethod. Changing that would be easy enough.

I'm still interested in feedback on the basic idea, I'd love to here that my proposal isn't necessary because there is already a way to get the behavior I'm looking for although that's nog going to happen ;-).

Ronald


>
> (Apologies for the HTML top-post)

I don't mind.

PS. Does anyone know if the pep editors are away (conferences, holidays, ...)? I could just check in my proposal in the peps repository, but as this is my first PEP I'd prefer to follow the documented procedure and have someone that knows what he's doing look at the metadata before checking in.

>
> Sent from my Windows Phone
> From: Ronald Oussoren
> Sent: ‎7/‎6/‎2013 0:47
> To: Ronald Oussoren
> Cc: python-dev at python.org Dev
> Subject: Re: [Python-Dev] Hooking into super() attribute resolution
>
> I've updated the implementation in issue 18181 <http://bugs.python.org/issue18181> while adding some tests, and have updated the proposal as well.
>
> The proposal has some open issues at the moment, most important of which is the actual signature for the new special method; in particular I haven't been able to decide if this should be an instance-, class- or static method. It is a static method in the proposal and prototype, but I'm not convinced that that is the right solution.
>
> Ronald
>
>
>
>
> PEP: TODO
> Title: Hooking into super attribute resolution
> Version: $Revision$
> Last-Modified: $Date$
> Author: Ronald Oussoren <ronaldoussoren at mac.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 12-Jun-2013
> Post-History: 2-Jul-2013, ?
>
>
> Abstract
> ========
>
> In current python releases the attribute resolution of the `super class`_
> peeks in the ``__dict__`` attribute of classes on the MRO to look
> for attributes. This PEP introduces a hook that classes can use
> to override that behavior for specific classes.
>
>
> Rationale
> =========
>
> Peeking in the class ``__dict__`` works for regular classes, but can
> cause problems when a class dynamicly looks up attributes in a
> ``__getattribute__`` method.
>
> The new hook makes it possible to introduce the same customization for
> attribute lookup through the `super class`_.
>
>
> The superclass attribute lookup hook
> ====================================
>
> In C code
> ---------
>
> A new slot ``tp_getattro_super`` is added to the ``PyTypeObject`` struct. The
> ``tp_getattro`` slot for super will call this slot when it is not ``NULL``,
> and will raise an exception when it is not set (which shouldn't happen because
> the method is implemented for :class:`object`).
>
> The slot has the following prototype::
>
>     PyObject* (*getattrosuperfunc)(PyTypeObject* cls, PyObject* name,
>         PyObject* object, PyObject* owner);
>
> The function should perform attribute lookup on *object* for *name*, but only
> looking in type *tp* (which will be one of the types on the MRO for *self*)
> and without looking in the instance *__dict__*.
>
> The function returns ``NULL`` when the attribute cannot be found, and raises and
> exception. Exception other than ``AttributeError`` will cause failure of super's
> attribute resolution.
>
> The implementation of the slot for the :class:`object` type is
> ``PyObject_GenericGetAttrSuper``, which peeks in the ``tp_dict`` for *cls*.
>
> Note that *owner* and *object* will be the same object when using a
> class-mode super.
>
>
> In Python code
> --------------
>
> A Python class can contain a definition for a static method
> ``__getattribute_super__`` with the following prototype::
>
>    def __getattribute_super__(cls, name, object, owner): pass
>
> The method should perform attribute lookup for *name* on instance *self* while
> only looking at *cls* (it should not look in super classes or the instance
> *__dict__*
>
> XXX: I haven't got a clue at the moment if the method should be an
> instance-, class- or staticmethod. The prototype uses a staticmethod.
>
> XXX: My prototype automagicly makes this a static method, just like __new__ is
> made into a static method. That's more convenient, but also (too?) magical.
>
> XXX: Should this raise AttributeError or return a magic value to signal that
> an attribute cannot be found (such as NotImplemented, used in the comparison
> operators)? I'm currently using an exception, a magical return value would
> be slightly more efficient because the exception machinery is not invoked.
>
>
> Alternative proposals
> ---------------------
>
> Reuse ``tp_getattro``
> .....................
>
> It would be nice to avoid adding a new slot, thus keeping the API simpler and
> easier to understand.  A comment on `Issue 18181`_ asked about reusing the
> ``tp_getattro`` slot, that is super could call the ``tp_getattro`` slot of all
> methods along the MRO.
>
> AFAIK that won't work because ``tp_getattro`` will look in the instance
> ``__dict__`` before it tries to resolve attributes using classes in the MRO.
> This would mean that using ``tp_getattro`` instead of peeking the class
> dictionaries changes the semantics of the `super class`_.
>
>
> Open Issues
> ===========
>
> * The names of the new slot and magic method are far from settled.
>
> * I'm not too happy with the prototype for the new hook.
>
> * Should ``__getattribute_super__`` be a class method instead?
>
>   -> Yes? The method looks up a named attribute name of an object in
>      a specific class. Is also likely needed to deal with @classmethod
>      and super(Class, Class)
>
> * Should ``__getattribute_super__`` be defined on object?
>
>   -> Yes: makes it easier to delegate to the default implementation
>
> * This doesn't necessarily work for class method super class
>    (e.g. super(object, object))...
>
>
> References
> ==========
>
> * `Issue 18181`_ contains a prototype implementation
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> .. _`Issue 18181`: http://bugs.python.org/issue18181
>
> .. _`super class`: http://docs.python.org/3/library/functions.html?highlight=super#super
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130708/ec8c9d0b/attachment.html>