[Python-ideas] History on proposals for Macros?

[Matthew Rocklin]

Lisps like Scheme do indeed have an easier time with these due to the whole
code-is-data thing, it's quite doable in languages with real syntax
though.  R and Julia would be good examples of syntactic languages with
full macros.  The Julia implementation might be a good model for what
Python could do.   Their docs
<http://julia.readthedocs.org/en/latest/manual/metaprogramming/> are also a
nice read if anyone isn't familiar with the topic.

Macropy represents unevaluated expressions with the objects from the ast
module.  This seems like a sane choice.

To be a little pedantic I'll give a brief example loosely showing what a
macro is, then I'll talk about assert statements as a use case where macros
might help with a pain point in normal Python programming.

*Brief Educational Blurb*

We write code as text

defmacro f(x):
    ...

f(a + b) * sin(c)

We then parse parts of that text into syntax trees.

[image: Inline image 1]
Usually we translate these trees into byte-code and evaluate bottom-up,
starting with a + b, then applying f, etc... Macros stop this process.
They capture the subtrees beneath them before execution.  Whenever we see a
macro (f), we don't evaluate its subtree (a + b).  Instead we transform the
subtree into an in-memory representation (perhaps ast.BinOp(a, ast.Add(), b))
and hand that to f to do with as it will.  Lets see an example with
assertions.

*Use case with Assertions*

When testing we often want to write statements like the following

assert x == y
assert x in y
etc...

When these statements fail we want to emit statements that are well
informed of the full expression, e.g.

    5 != 6
    5 was not found in {1, 2, 3}

In Python we can't do this; assert only gets True or False and doesn't
understand what generated that value .  We've come up with a couple of
workarounds.  The first is the venerable unittest.TestCase methods that
take the two sides of the comparison explicitly e.g. assertEquals(a, b),
assertContains(a, b).  This was sufficiently uncomfortable that projects
like py.test arose and gained adoption.  Py.test goes through the trouble
of parsing the python test_.py files in order to generate nicer error
messages.

Having macros around would allow users to write this kind of functionality
directly in Python rather than resorting to full text parsing and code
transformation.  Macros provide an escape out of pure bottom-up evaluation.


On Sun, Mar 29, 2015 at 5:53 PM, Guido van Rossum <guido at python.org> wrote:

> On Sat, Mar 28, 2015 at 9:53 AM, Matthew Rocklin <mrocklin at gmail.com>
> wrote:
>
>> Responding to comments off list:
>>
>> I'm not referring to C-style preprocessor macros, I'm referring to macros
>> historically found in functional languages and commonly found in many
>> user-targeted languages built in the last few years.
>>
>
> Do you have examples and references? IIRC there's something named macros
> in Scheme but Scheme, unlike Python, completely unifies code and data, and
> there is a standard in-memory representation for code.
>
>
>> The goal is to create things that look like functions but have access to
>> the expression that was passed in.
>>
>> Some examples where this is useful:
>>
>>     plot(year, miles / gallon)  # Plot with labels determined by
>> input-expressions, e.g. miles/gallon
>>
>>     assertRaises(ZeroDivisionError, 1/0)  # Evaluate the rhs 1/0 within
>> assertRaises function, not before
>>
>>     run_concurrently(f(x), f(y), f(z))  # Run f three times in three
>> threads controlled by run_concurrently
>>
>> Generally one constructs something that looks like a function but, rather
>> than receiving a pre-evaluated input, receives a syntax tree along with the
>> associated context.  This allows that function-like-thing to manipulate the
>> expression and to control the context in which the evaluation occurs.
>>
>
> None of the examples need the syntax tree though. The first wants the
> string, the last probably just want a way to turn an argument into a lambda.
>
>
>> There are lots of arguments against this, mostly focused around potential
>> misuse.  I'm looking for history of such arguments and for a general "Yes,
>> this is theoretically possible" or "Not a chance in hell" from the
>> community.  Both are fine.
>>
>
> I don't think this is a mainline need in Python, so it's probably both. :-)
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/57d05b51/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: macro.png
Type: image/png
Size: 17961 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/57d05b51/attachment-0001.png>