lexing nested parenthesis

[François Pinard]

[Kristian Ovaska]

> The language category you're dealing with is called context-free grammars.
> It's more expressive than regular languages, and it can parse any
> programming language.

Any? :-)

I would not go that far.  There are strange beasts out there.  Moreover,
people usually throw good parts of the complexity in "semantic analysis"
passes.  In the caricatural case, your grammar may parse individual
characters and leave it all to later compilation stages, so a fortiori, any
kind of parser will do.  We are diving in the complex art of compromises! :-)

> Yacc is a classic C tool for this. PLY is a Lex+Yacc implementation in
> pure Python. SPARK is another Python tool, and there are some others.

I found SPARK and PLY to be interesting for parsing, as Yacc replacements.
Scanning with these tools falls back to building big regular expressions,
and relying on the `re' module.  I just tried Greg Ewing's PLEX yesterday,
which for a scanner, is much closer to Lex.

John Aycock's SPARK is surprisingly elegant, powerful and compact.
PLEX and PLY are not so compact, they also require some more work to use,
but they reward us with speedier analysis, especially for bigger grammars.

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard