[Python-Dev] Re: decorators and 2.4

[Phillip J. Eby]

At 01:26 PM 6/24/04 -0400, David Abrahams wrote:
>Anthony Baxter <anthony at interlink.com.au> writes:
>
> > So, let the floodgates open. Remember, we _can_ change this any
> > time up until 2.4b1, if there's a decision that the chosen form
> > sucks. :-)
>
>Here's thinking in a different direction altogether:
>
>    No special syntax
>
>    Instead, expose enough functionality in standard library functions
>    that an appropriately-written pure-python "decorate" function can
>    do it.
>
>
>    decorate(staticmethod, my_decorator)
>    def f(x):
>        whatever()
>
>    Since function definitions are executable statements, it should in
>    principle be possible to arrange something that allows one to hook
>    the execution of that statement.  Perhaps it's already doable with
>    the debugger hook?

Hmmm.  You probably *could* create such a function in CPython, maybe even 
as far back as 2.1 (since all it needs is sys._getframe and the tracing 
hook), but it wouldn't be portable to Jython.

And, boy, would it be a sick and twisted piece of code.  You'd need to do 
something like keep a copy of the locals and on each "new line" event from 
the trace hook, you'd need to both call the old trace hook (so that 
debugging could still take place) and check to see if the locals had a new 
function object.  As soon as the function object appeared, you could do 
your dirty work (after first restoring the *old* value bound to that name, 
so the decorators would have access to the previous binding of the name).

About the only bit you couldn't do in pure Python would be decorating a 
function defined in a nested scope, because "fast locals" aren't accessible 
from pure Python.  But for module-level functions and methods in classes, 
this might actually work.  Heck, I'm rather tempted to try to write it and 
see if it does.  At worst, it might be a fun way to learn the ins and outs 
of sys.settrace().

[goes away for 30 minutes or so]

Hm, interesting.  After a bit of twiddling, I've got a rough draft of the 
trace mechanism.  It seems to be possible to chain tracer functions in the 
manner described, and I got it to work even with pdb running.  That is, if 
the debugger was already tracing a frame, adding co-operative trace 
functions didn't appear to disturb the debugger's operation.

However, there are certainly some "interesting" peculiarities in the trace 
function behavior.  There seems to be a bit of voodoo 
necessary.  Specifically, it seems as though one must call 'sys.settrace()' 
*and* set 'frame.f_trace' in order to change a local frame trace function, 
even though in theory just returning a new trace function from a 'line' 
event should work.

Ah well.  Here's a proof-of-concept version, that uses right-to-left 
application and simple calling for the decorators, but as you'll see, the 
bulk of the code is devoted to mechanism rather than policy.  You'll see 
that it's easy to write different policies, although the way I have the 
trace mechanism set up, later tracers execute *before* earlier 
tracers.  I'm not sure if this is the right way to go, but it seems to read 
best with this syntax.

Anyway, it works with CPython 2.2.  Caveat: do *not* use listcomps or any 
other assignment statements between your 'decorate()' invocation and the 
function definition, or the decorators will be applied to the bound-to 
variable!

Personally, I think the 'decorate()' function is probably actually not that 
useful, compared to making specialized decoration functions like, say:

    classmethod_follows()
    def foo(klass, x, y, z):
        # ...

Only with better names.  :)  Also, using this approach means you can do 
something like:

     [classmethod_follows()]

     def foo(klass, x, y, z):
         # ...

which comes awfully close to Guido's preferred syntax!

Anyway, concept code follows, and I'm quite tempted to add this to 
PyProtocols, which already has a similar function for class decoration.


# =======

import sys

def add_assignment_advisor(callback,depth=2):

     frame = sys._getframe(depth)

     oldtrace = [frame.f_trace]
     old_locals = frame.f_locals.copy()

     def tracer(frm,event,arg):

         if event=='call':
             if oldtrace[0]:
                 return oldtrace[0](frm,event,arg)
             else:
                 return None

         try:
             if frm is frame and event !='exception':
                 for k,v in frm.f_locals.items():
                     if k not in old_locals:
                         del frm.f_locals[k]
                         break
                     elif old_locals[k] is not v:
                         frm.f_locals[k] = old_locals[k]
                         break
                 else:
                     return tracer

                 callback(frm,k,v)

         finally:
             if oldtrace[0]:
                 oldtrace[0] = oldtrace[0](frm,event,arg)

         frm.f_trace = oldtrace[0]
         sys.settrace(oldtrace[0])
         return oldtrace[0]

     frame.f_trace = tracer
     sys.settrace(tracer)


def decorate(*decorators):

     if len(decorators)>1:
         decorators = list(decorators)
         decorators.reverse()

     def callback(frame,k,v):
         for d in decorators:
             v = d(v)
         frame.f_locals[k] = v

     add_assignment_advisor(callback)



decorate(classmethod)

def f(klass,x,y):
     pass

print f