Skip Montanaro <skip@pobox.com>:
This reminds me of something which tickled my interesting bone the other day. The SpamAssassin folks are starting to look at Flex for much faster regular expression matching in situations where large numbers of static re's must be matched.
*snort* Took'em long enough. No, I shouldn't be snarky. Flex is only obvious to Unix old-timers -- the traditions that gave rise to it have fallen into desuetitude in the last decade.
...insert Flex rules into the string, call a slightly higher level function which calls flex to generate the scanner code and use a slightly different bit of module sticky stuff to make it callable from Python.
Lexers are painful in Python. They hit the language in a weak spot created by the immutability of strings. I've found this an obstacle more than once, but then I'm a battle-scarred old compiler jock who attacks *everything* with lexers and parsers. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>