No macros in Python

Mike Meyer mwm at mired.org
Mon Dec 16 23:11:01 EST 2002


In <20021216192625.A30427 at unpythonic.net>, jepler at unpythonic.net typed:
> into a macro.  If your proposal is all there is to support macros, then I
> have to write
> 	def body(): suite
> 	with_lock(lock, body)
> but that can be implemented as a function.  I'd rather just write 'with
> lock: suite'.  So some additional (non-function) syntax has to be introduced 
> for this purpose.  It's here that things start to get hairy: your proposal
> *could* be implemented by leaving the grammar unchanged, but changing the
> compiler's behavior when the production that now means "function call" is
> encountered.

You want the *second* half of the proposal, which I haven't talked
about yet, because I want to see the reaction to the first half.

The second half involves throwing out the __macro__ function and
adding a real, live "defmacro" statement. The reason that it's a
statement is because the identifier for the macro name can have ":"'s
in it. Their presence changes the invocation, but not the
definition. You still write it with a couple of arguments for the
context, and then arguments that are going to be passed blocks of code
for you to pass around.

> The biggest omission is how to allow 'suite's in macros.  For instance,
> one recent request that might be well served by macros would be a way to
> abstract
> 	lock.acquire()
> 	try: body
> 	finally: lock.release()
> It's at the point that this syntax must be added that things seem to get
> ugly.  For instance, you might permit at most one 'suite' to be given to a
> macro, and mark the beginning of a macro invocation with a statement that
> starts with a symbol that can't now begin a statement, or with a new token.
> For instance,
> 	macro with_lock lock: body

Hmm. I like the idea of adding a macro statement. That makes it more
likely that the "macros are evil" crowd will accept it, and less
likely that it will cause the balkanization of the python community.

So, how about this:

defmacro with_lock:(g, l, lock_code, body_code):
    lock = eval(lock)
    lock.acquire()
    try: exec body
    finally: lock.release

You then invoke it exactly like you said for the one-line version. But
you can also invoke it as:

macro with_lock lock:
   body_expression_1()
   body_expression_2()

etc.

The compiler sees this, and knows that lock - if present, that
argument is optional - and the body expressions are to be compiled and
passed to the with_lock: macro without evaluation.

This can be extended to multiple expressions/code segments by using
multiple colons:

define if:else:(g, l, if_test, if_code, else_test, else_code):
    ......

macro if expression:
    code body
else:
    code body

You get two arguments for every ':' in the name, plus the two context
arguments. Failure to have the right number of arguments is a syntax
error.

> So maybe you need to keep the function-call syntax, but find another way to
> introduce a suite.  Well, how about making ': suite' an expression that is
> equal to the function created by the statement
> 		execute if lock is already held)
> ?  I'm pretty sure the compiler current suppresses creation of INDENT,
> DEDENT, and NEWLINE tokens while inside grouping characters ('()', etc) so
> this is another big change, this time to the tokenizer.  What about the
> line ", :"?  That's NEWLINE DEDENT COMMA COLON, but with what will the
> tokenizer match indentation to find the correct dedent?  Maybe the "," or
> ")" will have to be on the same column as the beginning of the macro, but
> that looks even worse to me:
> 	with_lock(lock, :
> 		executed_with_lock_held
> 	, if_already_locked=:
> 		execute if lock is already held)

It's wanting to avoid having to do indentation levels in the middle of
an expression that caused me to work out the idea of the ":-ified"
macro names. If you're going to do that, you might as well pass
strings around.

> I think with_lock can still be written as a function, not a macro, by the
> way, if an extension like this is adopted:
> 	def with_lock(lock, body, if_already_locked=lambda: None):
> 		if lock.acquire():
> 			try: return body()
> 			finally: lock.release()
> 		else: return if_already_locked()

What do you think of my proposal - with credit to you for the idea of
using the macro keyword to distinguish macros?

> (I think of a true macro as something that would actually affect the parse
> tree of the function before it's byte-compiled, I guess, but I guess I
> don't actually see this reflected in your original post)

Correct. This is somewhere between lisp macros and C macros. You can't
twiddle the parse tree like you can in lisp, but you get the entire
power of the language to work with the code fragments and the results
of evaluating them in the body of the macro.

> If you don't want to fancy transformations of parse trees (and when I tried
> it, it got ugly quick) then why not settle for
> 	def ternary(cond, lt, lf):
> 		if cond: return lt()
> 		else: return lf()
> 	x = ternary(f(), lambda: g(1), lambda: g(2))
> as a substitute for
> 	a = f() ? g(1) : g(2)

Well, you don't get multi-line functions. And I'd really rather write
quoted strings than lambdas for this, even though that means that
syntax errors aren't caught at the point of

> I guess that in my mind, we don't need macros, but a nice way to create
> anonymous multi-line functions (and possibly a way to modify variables
> visible thanks to nested scopes).  If you have that, macros seem to simply
> become a static tool for getting more performance-- and everybody knows
> that nobody cares about performance in python.

That's one correct way of looking at it. Come to think of it, making
them real lambdas instead of code objects would simplify things. You'd
no longer need the extra two parameters for context. So instead of
having the compiler parse the code into a code object and pass that,
it should parse the code into a code object, wrap that into a lambda,
and then pass *that*. Much cleaner all the way around.

Thank you for the insightful questions and commentary.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.




More information about the Python-list mailing list