Parser project

[Michael Dyck]

Paul Prescod wrote:
> 
> Here is a project for a parser hacker:
> 
> Write a Python module that can read a grammar in the syntax that Python
> uses (see Grammar/Grammar in the source distribution) and generate a
> parser object without any special compilation stage:

This itself would not be too difficult, but I'm not sure it would be as
helpful as you think.

> from newparser import Parser
> 
> parser=Parser( grammar="/python/src/Grammar/Grammar" )
> parsetree=parser.parse( sys.argv[1] )
> code=compileast( parsetree )
> eval( code )
>
> With this tool, people could easily write Python-like languages in
> Python by making changes to the grammar. This would be great for those
> who want to experiment with new language features without building (and
> distributing) a special version of Python. Let's say you invented a
> Python-like language for matrices. You would tell people to install
> Python and then invoke your code like this:
> 
> python matrixLang.py helloworld.mtrx

So matrixLang.py would look like the above code, but with
"/python/src/Grammar/Grammar" replaced with the name of a file containing
the MatrixLang grammar, yes? So `parsetree' would be a parse-tree for the
content of helloworld.mtrx. The problem is that it would be a parse-tree
corresponding to the *MatrixLang* grammar (e.g., it might contain
MatrixLiteral nodes, and TransposeOperator nodes). And you can't give such
a parse-tree to compileast (or actually, to sequence2ast or tuple2ast,
whose result you would then pass to compileast): it wouldn't know anything
about such nodes.  Instead, you need another step that transforms
`parsetree' into a parse-tree that's acceptable to sequence2ast/tuple2ast.
In the example, this would have to transform MatrixLiteral and
TransposeOperator nodes into the equivalent standard-Python constructs.

Well, I guess that wouldn't be too difficult either (and maybe you had
this in mind anyway and glossed over it as the "easy" part), but it's
something that would have to be written on a per-language-invention basis,
as opposed to the run-time parser-constructor, which need only be written
once.

> ... It's not
> that doing this is impossible today, it just isn't as easy as it would
> be if there were a module that could parse things at runtime and was
> compatible with Python's grammar syntax.

Yes, I suppose it would be easier. Now if only I had some time to do some
parser hacking...

-Michael Dyck
 MichaelDyck at home.com