On Sat, Mar 28, 2015 at 12:50:09PM -0700, Andrew Barnert wrote:
On Mar 28, 2015, at 10:26, Steven D'Aprano steve@pearwood.info wrote:
On Sat, Mar 28, 2015 at 09:53:48AM -0700, Matthew Rocklin wrote: [...] The goal is to create things that look like functions but have access to the expression that was passed in.
assertRaises(ZeroDivisionError, 1/0) # Evaluate the rhs 1/0 within assertRaises function, not before
Generally one constructs something that looks like a function but, rather than receiving a pre-evaluated input, receives a syntax tree along with the associated context. This allows that function-like-thing to manipulate the expression and to control the context in which the evaluation occurs.
How will the Python compiler determine that assertRaises should receive the syntax tree rather than the evaluated 1/0 expression (which of course will raise)? The information that assertRaises is a "macro" is not available at compile time.
Well, it _could_ be available. At the time you're compiling a scope (a function, class, or top-level module code), if it uses an identifier that's the name of a macro in scope, the compiler expands the macro instead of compiling in a function call.
Perhaps I trimmed out too much of Matthew's comment, but he did say he isn't talking about C-style preprocessor macros, so I think that if you are imagining "expanding the macro" C-style, you're barking up the wrong tree. From his description, I don't think Lisp macros are quite the right description either. (As always, I welcome correction if I'm the one who is mistaken.)
C macros are more or less equivalent to a source-code rewriter: if you define a macro for "foo", whenever the compiler sees a token "foo", it replaces it with the body of the macro. More or less.
Lisp macros are different, and more powerful:
http://c2.com/cgi/wiki?LispMacro http://cl-cookbook.sourceforge.net/macros.html
but I don't think that's what Matthew wants either. The critical phrase is, I think:
"one constructs something that looks like a function but, rather than receiving a pre-evaluated input, receives a syntax tree along with the associated context"
which I interpret in this way:
Suppose we have this chunk of code creating then using a macro:
macro mymacro(expr): # process expr somehow
mymacro(x + 1)
That is *roughly* equivalent (ignoring the part about context) to what we can do today:
def myfunction(expr): assert isinstance(expr, ast.Expression) # process expr somehow
tree = ast.parse('x + 1', mode='eval') myfunction(tree)
The critical difference being that instead of the author writing code to manually generate the syntax tree from a string at runtime, the compiler automatically generates the tree from the source code at compile time.
This is why I think that it can't be done by Python. What should the compiler do here?
callables = [myfunction, mymacro] random.shuffle(callables) for f in callables: f(x + 1)
If that strikes you as too artificial, how about a simpler case?
from mymodule import name name(x + 1)
If `name` will refer to a function at runtime, the compiler needs to generate code which evaluates x+1 and passes the result to `name`; but if `name` will refer to a macro, the compiler needs to generate an ast (plus context) and pass it without evaluating it. To do that, it needs to know at compile-time which objects are functions and which are macros, but that is not available until runtime.
But we might be able to rescue this proposal by dropping the requirement that the compiler knows when to pass the syntax tree and when to evaluate it. Suppose instead we had a lightweight syntax for generating the AST plus grabbing the current context:
x = 23 spam(x + 1, !(x+1)) # macro syntax !( ... )
Now the programmer is responsible for deciding when to use an AST and when to evaluate it, not the compiler, and "macros" become regular functions which just happen to expect an AST as their argument.
[...]
If such a light-lambda syntax reduced the desire for macros down to the point where it could be ignored, and if that desire weren't _already_ low enough that it can be ignored, it would be worth adding. I think the second "if" is where it fails, not the first, but I could be wrong.
I presume that Matthew wants the opportunity to post-process the AST, not merely evaluate it. If all you want is to wrap some code an an environment in a bundle for later evaluation, you are right, a function will do the job. But it's hard to manipulate byte code, hence the desire for a syntax tree.