[Python-ideas] History on proposals for Macros?

[Andrew Barnert]

On Mar 31, 2015, at 22:11, Ron Adam <ron3200 at gmail.com> wrote:
> 
> 
> 
>> On 03/30/2015 04:54 PM, Andrew Barnert wrote:
>> [snip]
>> 
>>>> And the reason for bringing this idea up here was I think it could
>>>> be used to implement the lite macro behaviour that was suggested
>>>> with a bit of added syntax.
>>>> 
>>>> On the other hand it appears to me, that Python is going in the
>>>> direction of making it easier to compile to C code.  More dynamic
>>>> features may not be helpful in the long run.
> 
>> I don't think this is true. There definitely isn't any momentum in that
>> direction--shedskin and its two competitors are dead, while PyPy and
>> numba are alive and kicking ass. And I don't think it's desirable, or
>> that the core developers think it's desirable.
> 
> Good to know.
> 
> BTW... here is a first try.  Works barely.  Don't use the decorators inside a function yet.  I'm sure there's quite a few byte code changes that need to be done to make this dependable, but it does show it's possible and is something interesting to play with. ;-)
> 
> This is closely related to continuations, but rather than try to capture a frame and then continue the code, it just applies code objects to a name space.  That in it self ins't new, but getting the parts from decorated functions is a nice way to do it.
> 
> 
> import inspect
> 
> def fix_code(c):
>    """
>        Fix code object.
> 
>        This attempts to make a code object from a function be
>        more like what you would get if you gave exec a string.
> 
>        * There is more that needs to be done to
>        make this work dependably.  But it's a start.
> 
>    """
>    varnames = c.co_varnames
>    names = c.co_names
> 
>    xchg = [(124, 0x65),     #LOAD_FAST to LOAD_NAME
>            (125, 0x5a)]     #STORE_FAST to STORE_NAME

The first problem here is that any 124 or 125 in an operand to any opcode except 124 or 125 will be matched and converted (although you'll usually probably get an IndexError trying to treat the next two arbitrary bytes as an index...).

To solve this, you need to iterate opcode by opcode, not byte by byte. The dis module gives you the information to tell how many bytes to skip for each opcode's operands. (It also maps between opcode numbers and names, so you don't have to use magic numbers with comments.) Using it will solve this problem (and maybe others I didn't spot) and also make your code a lot simpler.

Another problem is that this will only catch local variables. Anything you don't assign to in the function but do reference is liable to end up a global or cell, which will still be a global or cell when you try to run it later. I'm not sure exactly how you want to handle these (if you just convert them unconditionally, then it'll be a bit surprising if someone writes "global spam" and doesn't get a global...), but you have to do something, or half your motivating examples (like "lambda: x+1") won't work. (Or is that why you're doing the explicit namespace thing instead of using the actual scope later on, so that this won't break, because the explicit namespace is both your locals and your globals?)

>    bcode = []
>    bgen = iter(c.co_code)
>    for b in bgen:
>        for bx1, bx2 in xchg:
>            if b == bx1:
>                i1 = next(bgen)
>                i2 = next(bgen)
>                index = i1  + i2 * 256
>                if b in [124, 125]:
>                    b = bx2
>                    char = varnames[index]
>                    names = names + (char,)
>                    index = names.index(char)
>                    i2 = index // 256
>                    i1 = index - i2
>                bcode.append(b)
>                bcode.append(i1)
>                bcode.append(i2)
>                break
>        else:
>            bcode.append(b)
>    co_code = bytes(bcode)
> 
>    Code = type(c)
>    co = Code(
>               0,      #co_argcount,
>               0,      #co_kwonlyargcount,
>               0,      #co_nlocals,
>               c.co_stacksize,
>               64,     #co_flags,
>               co_code,
>               c.co_consts,
>               names,  #co_names
>               (),     #co_varnames,
>               c.co_filename,
>               c.co_name,
>               c.co_firstlineno,
>               c.co_lnotab
>             )
>    return co
> 
> 
> class fn_signature:
>    """
>       Hold a signature from a function.
> 
>       When called with argumetns it will bind them
>       and return a mapping.
>    """
>    def __init__(self, fn):
>        self.sig = inspect.signature(fn)
> 
>    def __call__(self, *args, **kwds):
>        return dict(self.sig.bind(*args, **kwds).arguments)
> 
> 
> class fn_code:
>    """
>        Create a relocatable code object that can
>        be applied to a name space.
>    """
>    def __init__(self, fn):
>        self.co = fix_code(fn.__code__)
> 
>    def __call__(self, ns):
>        return eval(self.co, ns)

Why are you applying these to a dict? I thought the whole point was to be able to run it inside a scope and affect that scope's variables? If you just leave out the ns, won't that be closer to what you want? (And it also means you don't need the get_sig thing in the first place, and I'm not sure what that adds. Using a function signature plus a call expression as a fancy way of writing a dict display seems like just obfuscation. Maybe with default values, *args, binding partials and then calling get_sig on them, etc. is interesting for something, but I'm not sure what?)

> def get_sig(fn):
>    """ Decorator to get a signature. """
>    return fn_signature(fn)
> 
> 
> def get_code(fn):
>    """ Decorator to get a code object. """
>    return fn_code(fn)
> 
> 
> 
> 
> 
> # Example 1
> # Applying code to a namespace created by a signature.
> 
> @get_sig
> def foo(x):
>    pass
> 
> @get_code
> def inc_x():
>    x += 1
> 
> @get_code
> def dec_x():
>    x -= 1
> 
> @get_code
> def get_x():
>    return x
> 
> 
> ns = foo(3)         # creates a namespace
> 
> inc_x(ns)           # Apply code to namespace
> inc_x(ns)
> print(get_x(ns))    # --> 5
> 
> dec_x(ns)
> dec_x(ns)
> print(get_x(ns))    # --> 3
> 
> 
> 
> # Example 2
> # Code + signature <---> function
> 
> def add_xy(x, y):
>    return x + y
> 
> sig = get_sig(add_xy)
> co = get_code(add_xy)
> print(co(sig(3, 5)))     # --> 8
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/