[Python-Dev] dealing with decorators hiding metadata of decorated functions

[Brett Cannon]

On 3/17/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Josiah Carlson wrote:
> > "Brett Cannon" <brett at python.org> wrote:
> >> With the discussion of a possible @decorator to help set the metadata
> >> of the decorator to that of what the wrapped function has, I had an
> >> idea that I wanted to toss out there (this dicussion stems from a blog
> >> post I made: http://sayspy.blogspot.com/2006/03/how-to-handle-object-identity-issues.html).
> >
> > [snip]
> >
> > Edward Loper suggested this way back on September 5, 2004.
> >
> >     http://mail.python.org/pipermail/python-dev/2004-September/048626.html
> >
> > I was and continue to be +1 on this,
>
> +1 here, too. Unlike Brett, though, I have no problem with overwriting
> __name__ and updating __dict__ unconditionally, and overwriting __doc__ if it
> hasn't already been set.
>

I have no problem with updating __dict__ since decorators that just 
do annotations will need to have that information passed forward.  But
if Josiah's suggestion is taken and decorators are just part of a
lookup chain of attributes and they just fall through then this would
not be needed.

> The first two are needed if we expect "print f" and "f.a" to work properly.

I don't know if having ``print f`` work like that is important.  And
if it is print can be tweaked to follow __decorates__ properly.

> The function's name is set on the 'def' line, and I'd be much happier seeing
> that at all levels of the decorator chain, rather than seeing something like
> "wrapper", "wrapper", ... "wrapper", "f". Annotating decorators will modify
> the functions' attributes, and this needs to be visible in the final
> function's dictionary.
>
> If a wrapper doesn't set a docstring explicitly, it makes a lot more sense to
> me to re-uses the original function's docstring rather than leave it at None.
>

But this is still a loss of information since you will no longer know
that the decorator had no docstring.

> My real interest is that it should be possible to get at all the details of
> the original function (such as its code object), and the obvious way to do
> that is with a standard attribute that links to the original.
>

Exactly.  __decorates__ would provide this.  Otherwise the last key
piece of information I think missing that decorators cannot copy from
the decorated function is the function parameters, and that is where
my __signature__ object proposal (I think Philip proposed the main
idea originally) steps in if the __decorates__ idea doesn't go
anywhere.

> > though I would go farther and state,
> > like I did at the time, that one shouldn't copy any of the function
> > attributes, they should come 'free', similar to the way that class
> > attributes are 'free' on subclasses.
>
> Well, that's the idea behind the decorator decorator - simply put @decorator
> on your decorator function and it will automatically do the right thing.
>
> >     http://mail.python.org/pipermail/python-dev/2004-September/048631.html
> >
> > What would make this _really_ nice is if one didn't need to do anything
> > manually; that the attribute that pointed to the decorated
> > function/object would be automatically applied - though I realize that
> > this may not be generally possible.
>
> A slight problem is that not all decorators will wrap the function they
> decorate - some will only annotate it.
>
> However, here's an idea for the @decorator decorator that would make it pretty
> much automatic, leaves the docstring alone if the decorator has already set it
> on the wrapper, and builds up a record of the decorators that are wrapping the
> the original function:
>
> def _link_decorated(decorated, orig, decorator):
>        """Link a decorated function with the original"""
>        decorated.__name__ = orig.__name__
>        decorated.__dict__.update(orig.__dict__)
>        if decorated.__doc__ is None:
>            decorated.__doc__ = orig.__doc__
>        decorated.__decorates__ = orig
>        decorated.__decorator__ = decorator
>
> def decorator(orig_decorator):
>        """Decorator to create a well-behaved decorator"""
>        # Wrapper function that links a decorated function
>        # to the original function if necessary
>        def wrapper(f):
>            decorated = orig_decorator(f)
>            if decorated is not f:
>                # Link wrapper function to the original
>                _link_decorated(decorated, f, wrapper)
>            return decorated
>        _link_decorated(wrapper, orig_decorator, decorator)
>        return wrapper
>
> Decorators that only do annotations aren't recorded because there isn't
> anywhere to record them. Wrapping decorators, on the other hand, allow the
> references to both the decorated function and the applied decorator to be
> stored on the new function object.
>

If you do the copying of data and provide a __signature__ object you
have 90% of the metadata one would want for introspection on a
decorated function, so I don't know if __decorates__ will be that
important in that situation (but one extra pointer to the function
object is not that expensive, so it can still be provided very
cheaply).

I guess we need to decide if we want to promote the copying of
metadata from the decorated function into the decorator or not.  If we
do support copying the metadata, then should we provide a
__signature__ object as well to help with that or not.  __decorates__
is an innocuous suggestion that it should probably be promoted
regardless of what we end up suggesting for use.  The real difference
will be whether 'inspect' and friends get tweaked to follow
__decorates__ or just to use the metadata on the decorator.

-Brett

> Cheers,
> Nick.
>
> P.S. Example usage:
>
> Py> @decorator
> ... def annotated(f):
> ...     f.note = 1
> ...     return f
> ...
> Py> @decorator
> ... def wrapped(f):
> ...     def wrapper(*args, **kwds):
> ...         return f(*args, **kwds)
> ...     return wrapper
> ...
>
> Py> @wrapped
> ... @annotated
> ... @wrapped
> ... @wrapped
> ... def show(*args, **kwds):
> ...     print args, kwds
> ...
>
> Py> while hasattr(obj, "__decorates__"):
> ...     print obj
> ...     print "  Decorates:\t%s" % obj.__decorates__
> ...     print "  Using:\t%s" % obj.__decorator__
> ...     print "  Annotated?:\t%s" % hasattr(obj, "note")
> ...     print
> ...     obj = obj.__decorates__
> ...
> <function show at 0x00AE9D30>
>    Decorates:    <function show at 0x00AE9AB0>
>    Using:        <function wrapped at 0x00AE9CF0>
>    Annotated?:   True
>
> <function show at 0x00AE9AB0>
>    Decorates:    <function show at 0x00AE9B30>
>    Using:        <function wrapped at 0x00AE9CF0>
>    Annotated?:   True
>
> <function show at 0x00AE9B30>
>    Decorates:    <function show at 0x00AE99F0>
>    Using:        <function wrapped at 0x00AE9CF0>
>    Annotated?:   False
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>