[Python-Dev] A standard lexer?
Sun, 08 Oct 2000 23:04:55 -0700
Tim Peters wrote:
> Reviewing this before 2.0 has been on my todo list for 3+ months, and
> finally got to it. Good show! I converted some of my by-hand scanners to
> use lastgroup, and like it a whole lot. I know you understand why this is
> Good, so here's a simple example of an "after" tokenizer for those who don't
> (this one happens to tokenize REXX-like PARSE stmts):
Is there a standard technique for taking a regexp like this and applying
it to data fed in a little at a time? Other than buffering the data
forever? That's something else I would like in a "standard Python
lexer", if that's the goal.
Paul Prescod - Not encumbered by corporate consensus
Simplicity does not precede complexity, but follows it.