[Python-ideas] History on proposals for Macros?

[Andrew Barnert]

On Apr 1, 2015, at 15:17, Nathaniel Smith <njs at pobox.com> wrote:
> 
> On Apr 1, 2015 3:02 PM, "Andrew Barnert" <abarnert at yahoo.com.dmarc.invalid> wrote:
> >
> > On Wednesday, April 1, 2015 12:40 PM, Ron Adam <ron3200 at gmail.com> wrote:
> > > When exec is given a code object, it call's PyEval_EvalCodeEx in
> > > ceval.c directly with local and global dictionaries.  (That should answer
> > > some of your comments as to why I uses the dictionary.)
> >
> > Yes, and running the block of code directly with the local and global dictionaries is exactly what you want it to do, so why are you telling it not to do that?
> >
> > For example:
> >
> >
> >     def macro():
> >         x += 1
> >     code = fix_code(macro.__code__)
> >
> >     def f(code_obj):
> >         x = 1
> >         loc = locals()
> >         eval(code_obj)
> >         return loc['x']
> >
> > (Or, if you prefer, use "run_code_obj" instead of "eval".)
> >
> > The problem here is that if you just "return x" at the end instead of "return loc['x']", you will likely see 1 instead of 2. It's the same problem you get if you "exec('x += 1')", exactly as described in the docs.
> >
> > That happens because f was compiled to look up x by index in the LOAD_FAST locals array, instead of by name in the locals dict, but your modified code objects mutate only the dict, not the array. That's the big problem you need to solve. Adding more layers of indirection doesn't get you any closer to fixing it.
> 
> You can propagate changes to the dict back to the array by calling the c api function PyFrame_LocalsToDict. It's pretty easy to do via ctypes, see e.g.
> 
You mean PyFrame_LocalsToFast, not the other way around, right? That's a good idea. There might be problems executing two code blocks (or a code block and a normal eval/exec statement) in the same function, but for a prototype that's fine...
> http://pydev.blogspot.com/2014/02/changing-locals-of-frame-frameflocals.html?m=1
> 
> I guess you could append some byte code to do this to your modified function bodies.
> 
That would be painful without byteplay--you have to insert the new instructions before every return and raise bytecode, which means renumbering jumps, etc.

But do you really need to? Can you do it in the wrapper?

    def call_code(code):
        frame = sys._getframe(1)
        PyFrame_LocalsToDict(py_object(frame))
        try:
            return eval(code, frame.f_locals(), frame.f_globals())
        finally:
            PyFrame_LocalsToFast(py_object(frame))

(Doing the matched pair like this might avoid the problem with multiple code blocks in one function. I'm not sure, but... worth a try, right?)

I think there will still be problem with cell vars (that is, updating a local in the caller which is used in a closure by a local function in the caller). And there's definitely still the problem of magically guessing which variables are meant to be local, closure, or global. But again, for a prototype, that all may be fine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150401/9c091349/attachment-0001.html>