[Types-sig] updated grammar (was: updated proposal)

Greg Stein gstein@lyra.org
Tue, 4 Jan 2000 20:37:50 -0800 (PST)


On Mon, 3 Jan 2000, Tony Lownds wrote:
> I just tried to compile Python 1.5.2 with your syntax changes. I got
> pretty far but your syntax needed a bit of tweaking. Here is a tweaked
> version of the "type declarators" code snippet that worked[1]:

Excellent! Very cool.

> 
> # first NAME should be: 'member', 'var', 'class', etc
> decl_stmt: 'decl' NAME NAME ':' typedecl
> typedecl: item_tdecl ('or' item_tdecl)*
> item_tdecl: param_tdecl | list_tdecl | dict_tdecl | func_tdecl |
> tuple_tdecl
> param_tdecl: dotted_name ['(' arglist_tdecl ')']
> tuple_tdecl: '(' typedecl (',' typedecl)* [','] ')'
> list_tdecl: '[' typedecl ']'
> dict_tdecl: '{' typedecl ':' typedecl '}'
> func_tdecl: 'def' '(' [varargs_tdecl] ')' '->' typedecl
> varargs_tdecl: (arg_tdecl ',')* ('*' ':' typedecl [',' '**' ':' typedecl] 
> | '**' ':' typedecl) | arg_tdecl (',' arg_tdecl)* [',']
> arg_tdecl: typedecl
> 
> 
> The changes are:
> - the grouping alternative in item_tdecl conflicted with tuple_tdecl, so
>   tuple_tdecl handles grouping and tuples

Ah. Right. I added the grouping when I wrote the interface parser in my
prototype and realized that:

  decl member foo: def()->Int or String

required parentheses so that a person could differentiate between the two
possible meanings. In the prototype parser, there is no tuple syntax :-).
I copied over the grouping to the type-proposal without realizing the
conflict.

Well... I will note that Guido resolves grouping vs. tuple construction at
compile time, too :-)

> - arg_tdecl cannot have a name, just a type. I don't see a way to have:
> 
>     arg_tdecl: [NAME ':'] typedecl 
> 
>   in python's parser, because typedecl can start with a
>   NAME via dotted_name) so the rule makes an ambiguous DFA. You 
>   can have:
> 
>     arg_tdecl: typedecl [':' typedecl]
> 
>   and constrain the first typedecl in the AST-consuming code.

I think I can re-jigger the grammar. I'll work on that later tonite.

> I'm going to continue to add the syntax you've laid out. I've
> independently been writing grammar for in-line function definitions stuff
> so that'll be the next piece. 

Woo! That is awesome.

Adding decl statements and typedecls should have not required a compile.c
change (unless you use them in source code), as you probably noted.
Similarly, adding the function definition changes and "typedef" operator
would not require compile.c changes unless you use those things in source
code.

Adding the type-assert operator will require a change whether you use the
operator or not (because it introduces "or_test" into the parse tree).

> However, I'd like to hear from people who have an opinion if using a
> modified build is even adviseable. My recent experience suggests it is not
> because when you change the grammer you often trip up the compiler and end
> up with a broken python.

Only with the type-assert operator. The others are quite safe, unless you
truly feed that source into the Python compiler (rather than just the
parser).

I would *really* like to see a patch set for your changes. That would be
awesome. I'd also like to make them available via my web pages.

> If it *is* adviseable, then the thing to do is to completely specify the
> syntax and then make a Grammar and compile.c that works with current
> syntax (ignoring any type declarations).

I think it is. It is very helpful to actually work with code, rather than
sit around and talk for weeks on end. Gotta get in an really try things!

> If it is *not* adviseable, then the thing to do is to make check.py go
> through an interface instead of accessing the parser, token and symbol
> modules directly, and look for another way to turn Python programs into
> parse trees.

I don't want to replicate the Python parser :-) ... instead, we would have
to rely entirely on external files and/or string-based or functional
replacements for the syntax (as described in my recent feedback to the
PyDL RFC).

> Either way I'm pretty interested in getting check.py to work on a modified 
> Python syntax, if only to test ideas.

Me too!

I will note that check.py could easily handle working with the standard
parser or you modified parser. It can degrade gracefully.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/