[Python-Dev] Modifying Grammar/grammar and other foul acts

[Jeremy Hylton]

On Sat, Mar 6, 2010 at 11:27 AM, Gregg Lind <gregg.lind at gmail.com> wrote:
> Python-devs,
>
> I'm writing to you for some help in understanding the Python grammar.  As an
> excuse to deep dive into Python's tokenizer / grammar, I decided (as a
> hideous, hideous joke) to want to allow braces where colons are allowed (as
> flow control).
>
> Starting from PEP 306 (and branch r311), I hacked on Grammar/Grammer
>
> As a first example:
>
> funcdef: ('def' NAME parameters ['->' test] ':' suite |
>           'def' NAME parameters ['->' test] '{' suite '}' )
>
> I reran Parser/pgen and the dfa changes, but python (3.1) when recompiled,
> throws errors on things like:
>
> def a() { None }
>
> Strangely enough:
>
> lambdef: ( 'lambda' [varargslist] ':' test  |
>                'lambda' [varargslist] '{' test '}' )
>
> works fine!  I this simplely some difference between "test" and "suite".
>
> I have tried tackling this with gdb, looking at err_input clearly isn't
> enough.
>
>     (gdb) break err_input
>     (gdb) break PyParser_ASTFromString
>     import sys
>     b = compile("def a() {pass}","sys.stdout","single")
>     # yet a simple grammar fix is enough for this!
>     c = compile("lambda x {None}","sys.stdout","single")
>
> I'm in over my head!

You don't say what errors occur when you try to compile strings in
your new language.  You may have changed the Grammar, which allows you
to tokenize the input.  That isn't enough to get the input to compile.
 You also need to change the compiler to understand the new tokens.

Jeremy

> Any insights / help would be appreciated.  Full-on flaming is also
> appropriate, but would be less appreciated.
>
> Specific questions
>
> 1.)  I assume the Grammar/grammar is read top to bottom.  Confirm?
> 2.)  More help figuring out how to debug what python *thinks* it's seeing
> when it see "def a() {pass}".  It's not getting to the ast construction
> stage, as near as I can tell.  What additional breakpoints can I set to see
> where it's failing.
>
> Gregg L.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
>