[Python-Dev] A standard lexer?

Daniel Berlin dberlin@cygnus.com
02 Jul 2000 18:49:37 -0400

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Daniel Berlin wrote:
> > 
> > Actually, if it helps, i'm working on porting re2c to python.
> Pointers ?


> > Because it was written properly, it's rather simple (in fact, i've only
> > needed to modify one file, add some if's to ee if we want python
> > generation, and output the python code instead of c code)
> > The lexers it generates for c/C++ are much faster than flex lexers,
> > because they are directly coded.
> > I haven't benchmarked it against SPARK yet, but i would imagine it would
> > blow it away, for the same reason it blows away flex.
> Perhaps you should also look at the tagging engine in
> mxTextTools (you know where...) ?! It's very low-level, but
> it makes nice target for optimizing parser generators since
> it provides a Python interface to raw C speed.

re2c is simply a scanner generator, not a parser.