[Python-Dev] A standard lexer?

M.-A. Lemburg mal@lemburg.com
Mon, 03 Jul 2000 00:44:55 +0200

Daniel Berlin wrote:
> Actually, if it helps, i'm working on porting re2c to python.

Pointers ?

> Because it was written properly, it's rather simple (in fact, i've only
> needed to modify one file, add some if's to ee if we want python
> generation, and output the python code instead of c code)
> The lexers it generates for c/C++ are much faster than flex lexers,
> because they are directly coded.
> I haven't benchmarked it against SPARK yet, but i would imagine it would
> blow it away, for the same reason it blows away flex.

Perhaps you should also look at the tagging engine in
mxTextTools (you know where...) ?! It's very low-level, but
it makes nice target for optimizing parser generators since
it provides a Python interface to raw C speed.

Marc-Andre Lemburg
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/