Lisps like Scheme do indeed have an easier time with these due to the whole code-is-data thing, it's quite doable in languages with real syntax though. R and Julia would be good examples of syntactic languages with full macros. The Julia implementation might be a good model for what Python could do. Their docs <http://julia.readthedocs.org/en/latest/manual/metaprogramming/> are also a nice read if anyone isn't familiar with the topic. Macropy represents unevaluated expressions with the objects from the ast module. This seems like a sane choice. To be a little pedantic I'll give a brief example loosely showing what a macro is, then I'll talk about assert statements as a use case where macros might help with a pain point in normal Python programming. *Brief Educational Blurb* We write code as text defmacro f(x): ... f(a + b) * sin(c) We then parse parts of that text into syntax trees. [image: Inline image 1] Usually we translate these trees into byte-code and evaluate bottom-up, starting with a + b, then applying f, etc... Macros stop this process. They capture the subtrees beneath them before execution. Whenever we see a macro (f), we don't evaluate its subtree (a + b). Instead we transform the subtree into an in-memory representation (perhaps ast.BinOp(a, ast.Add(), b)) and hand that to f to do with as it will. Lets see an example with assertions. *Use case with Assertions* When testing we often want to write statements like the following assert x == y assert x in y etc... When these statements fail we want to emit statements that are well informed of the full expression, e.g. 5 != 6 5 was not found in {1, 2, 3} In Python we can't do this; assert only gets True or False and doesn't understand what generated that value . We've come up with a couple of workarounds. The first is the venerable unittest.TestCase methods that take the two sides of the comparison explicitly e.g. assertEquals(a, b), assertContains(a, b). This was sufficiently uncomfortable that projects like py.test arose and gained adoption. Py.test goes through the trouble of parsing the python test_.py files in order to generate nicer error messages. Having macros around would allow users to write this kind of functionality directly in Python rather than resorting to full text parsing and code transformation. Macros provide an escape out of pure bottom-up evaluation. On Sun, Mar 29, 2015 at 5:53 PM, Guido van Rossum <guido@python.org> wrote:
On Sat, Mar 28, 2015 at 9:53 AM, Matthew Rocklin <mrocklin@gmail.com> wrote:
Responding to comments off list:
I'm not referring to C-style preprocessor macros, I'm referring to macros historically found in functional languages and commonly found in many user-targeted languages built in the last few years.
Do you have examples and references? IIRC there's something named macros in Scheme but Scheme, unlike Python, completely unifies code and data, and there is a standard in-memory representation for code.
The goal is to create things that look like functions but have access to the expression that was passed in.
Some examples where this is useful:
plot(year, miles / gallon) # Plot with labels determined by input-expressions, e.g. miles/gallon
assertRaises(ZeroDivisionError, 1/0) # Evaluate the rhs 1/0 within assertRaises function, not before
run_concurrently(f(x), f(y), f(z)) # Run f three times in three threads controlled by run_concurrently
Generally one constructs something that looks like a function but, rather than receiving a pre-evaluated input, receives a syntax tree along with the associated context. This allows that function-like-thing to manipulate the expression and to control the context in which the evaluation occurs.
None of the examples need the syntax tree though. The first wants the string, the last probably just want a way to turn an argument into a lambda.
There are lots of arguments against this, mostly focused around potential misuse. I'm looking for history of such arguments and for a general "Yes, this is theoretically possible" or "Not a chance in hell" from the community. Both are fine.
I don't think this is a mainline need in Python, so it's probably both. :-)
-- --Guido van Rossum (python.org/~guido)
