<div dir="ltr"><div>Ryan: I'm trying to figure out how the parsing library should be done — not trying to work around other designs.</div><div>Stefan: maybe this is a better answer to your question.</div><div><br></div>So thinking about this more, this is how I think it should be done:<div><br></div><div>Each grammar rule is expressed as an Iterable.</div><div><div><br></div><div><br></div><div>class FileInput:</div><div> def __init__(self):</div><div> self.indent_level = None</div><div><br></div><div> def match(self):</div><div> while True:</div><div> matched = yield Disjunction(</div><div> '\n',</div><div> [Whitespace(self.indent_level, indent=False), Statement()])</div><div> if matched == '\n':</div><div> break</div><div> yield EndOfFile()</div><div><br></div><div><br></div><div>class Suite:</div><div> def __init__(self, indent_level):</div><div> self.indent_level = indent_level</div><div><br></div><div> def match(self):</div><div> yield Disjunction(</div><div> SimpleStatement(),</div><div> ['\n', Whitespace(self.indent_level, indent=True),</div><div> Repeat(Statement)])</div><div> # dedent is not required because the next statement knows its indent</div><div> # level.</div></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jun 6, 2015 at 9:36 AM, s.krah <span dir="ltr"><<a href="mailto:stefan@bytereef.org" target="_blank">stefan@bytereef.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u><div><div style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif"><br><div><div dir="ltr"><br><b>Neil Girdhar <<a href="mailto:mistersheik@gmail.com" target="_blank">mistersheik@gmail.com</a>></b> wrote: <br><div><span class="">> Along with the grammar, you also give it code that it can execute as it matches each symbol in a rule. In Python for example, as it matches each argument passed to a function, it would keep track of the count of *args, **kwargs, and keyword arguments, and regular arguments, and then raise a syntax error if it encounters anything out of order. Right now that check is done in validate.c, which is really annoying.<br><br></span>Agreed. For 3.4 it was possible to encode these particular semantics into the grammar<br>itself, but it would no longer be LL(1).<br><br>If I understood correctly, you wanted to handle lexing and parsing together. How<br>would the INDENT/DEDENT tokens be generated?<br><br>For my private ast generator, I did the opposite: I wanted to formalize the token<br>preprocessing step, so I have:<br><br> lexer -> parser1 (generates INDENT/DEDENT) -> parser2 (generates the ast directly)<br><br><br>It isn't slower than what is in Python right now and you can hook into the token stream<br>at any place.<br><br><br></div><div><br>Stefan Krah<br></div><br><br></div></div></div></div><div class="HOEnZb"><div class="h5">
<p></p>
-- <br>
<br>
--- <br>
You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.<br>
To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/python-ideas/WTFHSUbfU20/unsubscribe" target="_blank">https://groups.google.com/d/topic/python-ideas/WTFHSUbfU20/unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:python-ideas+unsubscribe@googlegroups.com" target="_blank">python-ideas+unsubscribe@googlegroups.com</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/optout</a>.<br>
</div></div><br>_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" target="_blank">http://python.org/psf/codeofconduct/</a><br>
<br>
--<br>
<br>
---<br>
You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.<br>
To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/python-ideas/WTFHSUbfU20/unsubscribe" target="_blank">https://groups.google.com/d/topic/python-ideas/WTFHSUbfU20/unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:python-ideas%2Bunsubscribe@googlegroups.com">python-ideas+unsubscribe@googlegroups.com</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/optout</a>.<br>
<br></blockquote></div><br></div>