Parser project

[paul at prescod.net.bbs]

Huaiyu Zhu wrote:
>
> ...
>
> This is a good idea, although MatPy could not wait for it to happen, and I
> do not foresee much additional syntax being requested. For the current
> matrix issue, a limited parser might be easier and more useful.

Not as much easier as you think. And anyhow, my proposal was to make a
module useful to Python programmers from a variety of domains. That
seems more likely than the infix operators which are not going to be as
widely popular. For instance, someone wanting to experiment with list
comprehensions or embedded SQL could do so.

> I'm not sure if it can be made as easy as
>
> import NewOperator
> NewOperator.define(".+", "__dotadd__")
> class A: def __dotadd__(self, other) ...
> a=A(); b=A()
> a.+b
> NewOperator.define("compares", "__cmp__")
>  ------> Exception: "compares" contains characters not allowed in operator.

No, it can't be made that easy using traditional parser/compiler
techniques. Using non-standard techniques, yes it could be but I'm not
sure if someone out there is interested enough to do it.

If you want to get started quickly, I would suggest you download all of
the Python parser toolkits and look for those that have Python
lexers/parsers built in. Choose one and use it. You may already be 80%
of the way there. You might want to ask around for help in the compiler
sig mailing list:

http://www.python.org/sigs/compiler-sig/

Here is a list of parsing tools I collected once:

<slide><title>PyLR</title>
<points>
<point>For efficient <a
href="http://starship.skyport.net/crew/scott/PyLR.html">parsers</a> in
python</point>
<point>Includes lexer and parser</point>
<point>Supports LR(1) and LALR(1)</point>
<point>By Scott Cotton</point>
</points>
</slide>

<slide><title>PyLR example</title>
<points>
<pre>expression: expression
		PLUS term (addfunc)
		| term;
term: term
	TIMES factor (timesfunc)
	| factor;
factor: LPAREN expression RPAREN (parenfunc)
	| INT;
</pre>
</points>
</slide>

<slide><title>kwParsing</title>
<points>
<point><a href="http://www.chordate.com/kwParsing/">by Aaron
Watters</a></point>
<!-- More here -->
<pre>
GRAMMARSTRING ="""
Value ::
 @R SetqRule :: Value >> ( setq var Value )
 @R ListRule :: Value >> ( ListTail
 @R TailFull :: ListTail >> Value ListTail
 @R TailEmpty :: ListTail >> )
 @R Varrule :: Value >> var
 @R Intrule :: Value >> int
 @R Strrule :: Value >> str
 @R PrintRule :: Value >> ( print Value )
"""
</pre>
</points>
</slide>

<slide><title>Simpleparse</title>
<points>
<point>Built on top of <a
href="http://starship.skyport.net/~lemburg/mxTextTools.html">mxTextTools</a></point>
<point>Generates tables that drive mxTextTools</point>
<point>Reputed to be very fast</point>
<point>by <a
href="http://members.home.net/mcfletch/programming/simpleparse/simpleparse.html">Michael
Fletcher</a></point>
<point>Gentle introduction to more complex mxTextTools package</point>
</points>
</slide>

<slide><title>Simpleparse example</title>
<pre>
file := [ \t\n]*, section+
section := '[',identifier,']', ts,'\n', body
body := statement*
statement := (ts,';',comment,'\n')/equality/nullline
nullline := ts,'\n'
ts := [ \t]*

...
</pre>
</slide>

<slide><title>YAPPS</title>
<points>
<point>Yet Another Python Parser <a
href="http://theory.stanford.edu/~amitp/Yapps/yapps-doc/yapps-doc.html">system</a>.</point>
<point>Simple, easy to use</point>
<point>Generates human-readable parsers</point>
<point>Recursive descent</point>
<point>Not as fast, powerful or flexible as others</point>
<point>Supports LL(1)</point>
</points>
</slide>

<slide><title>Spark</title>
<point><a href="http://www.csr.uvic.ca/~aycock/python/">Scanning,
Parsing, and Rewriting Kit</a></point>
<point>John Aycock</point>
<point>Very powerful, easy, slow</point>
<pre>
def p_ifelse_stmt(self, args):
'''
stmt ::=
  IF expr
  THEN stmt
  ELSE stmt
'''
    somePythonCode()
</pre>
</slide>

</div>
</slides>
</slideshow>

--
 Paul Prescod - Not encumbered by corporate consensus
It's difficult to extract sense from strings, but they're the only
communication coin we can count on.
	- http://www.cs.yale.edu/~perlis-alan/quotes.html