[Python-Dev] A Hygienic Macro System in Python?

[Guido van Rossum]

> Has there been any discussion about adding a macro system to Python,
> such as the hygienic macros supported by Scheme (R^4RS and later) and
> Dylan?  Dylan made a lot of use of macros, providing a very powerful
> environment for them. Many useful constructs can be implemented with
> macros.
> 
> Anyway, if this has been discussed and shot down, I'll end the
> discussion now. However, if this something that has been considered,
> or might be, I'd be interested in pursuing it further.

I've considered it, and rejected it.  That doesn't mean you shouldn't
bring it up, but I expect it would turn Python into an entirely
different language.

I recently had a thought about how Python and C++ represent extreme
ends of the spectrum of how much happens in the parser/compiler
vs. how much happens at run time (although Lisp would be even more
extreme than Python on this scale).

- In C++, the compiler is incredibly powerful (especially with the new
  templates).  This means you can use the compiler for very
  sophisticated compile-time processing.  But the downside is that
  if you don't know a lot about how compilers work, or how the
  specific C++ compiler you're using works, it can be quite baffling
  to figure out how you're supposed to accomplish even simple things
  (e.g. examples like "istream_iterator<string> i(cin);" are quite
  baffling).

- In Python, the parser and compiler are extremely stupid -- e.g. the
  compiler doesn't even know the type of variables, and translates
  everything to very simple bytecode.

This means that (ideally) when using Python, you only need to learn
the intricacies of one system: the runtime, and that one is relatively
well-behaved: when something's wrong, all the context is available,
and can be shown in the debugger or in the traceback.  Contrast that
to debugging C++: when the problem occurs at runtime, it may be
difficult to correlate it to the relevant portions of the source
(e.g. because of template expansion).  When a C++ problem occurs at
compile time, it may *still* be hidden by layers of template
expansion, and even if it isn't, there are so many things to talk
about that the compiler may have a hard time telling you what's wrong
in a language you can understand -- again, you often end up learning a
lot about compiler internals if you want to understand its error
messages.

I guess I'm wary of anything that adds compile time complexity,
especially if the user will be aware of it -- I'm not set against
adding optimizations, as long as they don't change the semantics and
don't add new error messages or warnings.

--Guido van Rossum (home page: http://www.python.org/~guido/)