[Python-ideas] History on proposals for Macros?

[Matthew Rocklin]

Also, just to credentialize myself, I am not a huge Lisp lover.  I don't
want macros to do crazy logic programming or whatever.  I write numeric
code in the scientific Python ecosystem.  I want macros to build better
interfaces for downstream users.  This seems to be the modern use case in
user-focused languages rather than lisp-magic-hell.

On Mon, Mar 30, 2015 at 8:20 PM, Matthew Rocklin <mrocklin at gmail.com> wrote:

> Lisps like Scheme do indeed have an easier time with these due to the
> whole code-is-data thing, it's quite doable in languages with real syntax
> though.  R and Julia would be good examples of syntactic languages with
> full macros.  The Julia implementation might be a good model for what
> Python could do.   Their docs
> <http://julia.readthedocs.org/en/latest/manual/metaprogramming/> are also
> a nice read if anyone isn't familiar with the topic.
>
> Macropy represents unevaluated expressions with the objects from the ast
> module.  This seems like a sane choice.
>
> To be a little pedantic I'll give a brief example loosely showing what a
> macro is, then I'll talk about assert statements as a use case where macros
> might help with a pain point in normal Python programming.
>
> *Brief Educational Blurb*
>
> We write code as text
>
> defmacro f(x):
>     ...
>
> f(a + b) * sin(c)
>
> We then parse parts of that text into syntax trees.
>
> [image: Inline image 1]
> Usually we translate these trees into byte-code and evaluate bottom-up,
> starting with a + b, then applying f, etc... Macros stop this process.
> They capture the subtrees beneath them before execution.  Whenever we see a
> macro (f), we don't evaluate its subtree (a + b).  Instead we transform
> the subtree into an in-memory representation (perhaps ast.BinOp(a,
> ast.Add(), b)) and hand that to f to do with as it will.  Lets see an
> example with assertions.
>
> *Use case with Assertions*
>
> When testing we often want to write statements like the following
>
> assert x == y
> assert x in y
> etc...
>
> When these statements fail we want to emit statements that are well
> informed of the full expression, e.g.
>
>     5 != 6
>     5 was not found in {1, 2, 3}
>
> In Python we can't do this; assert only gets True or False and doesn't
> understand what generated that value .  We've come up with a couple of
> workarounds.  The first is the venerable unittest.TestCase methods that
> take the two sides of the comparison explicitly e.g. assertEquals(a, b),
> assertContains(a, b).  This was sufficiently uncomfortable that projects
> like py.test arose and gained adoption.  Py.test goes through the trouble
> of parsing the python test_.py files in order to generate nicer error
> messages.
>
> Having macros around would allow users to write this kind of functionality
> directly in Python rather than resorting to full text parsing and code
> transformation.  Macros provide an escape out of pure bottom-up evaluation.
>
>
> On Sun, Mar 29, 2015 at 5:53 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>> On Sat, Mar 28, 2015 at 9:53 AM, Matthew Rocklin <mrocklin at gmail.com>
>> wrote:
>>
>>> Responding to comments off list:
>>>
>>> I'm not referring to C-style preprocessor macros, I'm referring to
>>> macros historically found in functional languages and commonly found in
>>> many user-targeted languages built in the last few years.
>>>
>>
>> Do you have examples and references? IIRC there's something named macros
>> in Scheme but Scheme, unlike Python, completely unifies code and data, and
>> there is a standard in-memory representation for code.
>>
>>
>>> The goal is to create things that look like functions but have access to
>>> the expression that was passed in.
>>>
>>> Some examples where this is useful:
>>>
>>>     plot(year, miles / gallon)  # Plot with labels determined by
>>> input-expressions, e.g. miles/gallon
>>>
>>>     assertRaises(ZeroDivisionError, 1/0)  # Evaluate the rhs 1/0 within
>>> assertRaises function, not before
>>>
>>>     run_concurrently(f(x), f(y), f(z))  # Run f three times in three
>>> threads controlled by run_concurrently
>>>
>>> Generally one constructs something that looks like a function but,
>>> rather than receiving a pre-evaluated input, receives a syntax tree along
>>> with the associated context.  This allows that function-like-thing to
>>> manipulate the expression and to control the context in which the
>>> evaluation occurs.
>>>
>>
>> None of the examples need the syntax tree though. The first wants the
>> string, the last probably just want a way to turn an argument into a lambda.
>>
>>
>>> There are lots of arguments against this, mostly focused around
>>> potential misuse.  I'm looking for history of such arguments and for a
>>> general "Yes, this is theoretically possible" or "Not a chance in hell"
>>> from the community.  Both are fine.
>>>
>>
>> I don't think this is a mainline need in Python, so it's probably both.
>> :-)
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/fd2c3a60/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: macro.png
Type: image/png
Size: 17961 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/fd2c3a60/attachment-0001.png>