Python Macros

Lonnie Princehouse finite.automaton at
Tue Oct 5 15:38:56 EDT 2004

Josiah Carlson <jcarlson at> wrote in message news:<mailman.4259.1096910844.5135.python-list at>...
> > How hard/easy is it to extend the Python syntax?  Perhaps there are 
> Check out the tokenizer module.

Sort of OT for this thread, but I've found that tokenize isn't well
suited to extending syntax, unless you actually want to copy and edit
the module's source.

The module has a bunch of code to generate its regular expressions
that executes in its global scope at import time, so you'd have to
duplicate that code in order to modify these expressions.

One might expect something a little more object-oriented out of Python
(e.g. a Tokenizer class that can be inherited and behavior of the
subclass modified), but that's not how it's currently set up.

More information about the Python-list mailing list