[Types-sig] development approach (was: Syntax)

[Greg Stein]

On Sun, 19 Dec 1999, Paul Prescod wrote:
>...
> I am still guessing that for the first version it will also use Yet
> Another Parser because I don't want to change the real Python parser
> while we are in development mode. Are we going to set up our own CVS
> tree and have all of our testers install a new binary when we change the
> precedent of the ! operator or add a feature to the decl statement? I
> would rather send them one or two new Python files.

Our own CVS tree? Nah. I think that once we reach consensus and have Guido
Approval, then it goes right into the main CVS tree. I think the issue
here would be whether or how much we find we must iterate. I don't see any
iteration happening with the grammar, especially once we have Guido
Approval.

> *If* 1.6 is coming when we need it to, then we could give it a very

What do you mean "when we need it to" ? I didn't realize that there was
any "need" being discussed. I certainly am not interested in building a
system that is some kind of hackery add-on to 1.5. Design, build, and
integrate into 1.6, IMO.

> informal grammar for decl that basically stops at a comment or line
> boundary. That could invoke our (Python coded) decl-parser. More likely,
> we will want to test things out before 1.6 so we will probably stuff
> decl statements into expression statement-strings or shadow files.
> Either way, we have our own (sub-)grammar and (sub-)parser based, it
> seems, on Haskell.

Well... whoever codes it gets to decide :-). I'm just stating for the
record, that I believe the best approach is to directly start working on
the grammar changes [rather than use a short-term, throw-away solution].
If I do any coding on this, then it will use that approach.

> > Regardless: I'd hope that the first step to any implementation is to
> > update the Python grammar and allow us to annotate existing Python
> > programs (i.e. to use inline syntax). Updating the grammar is not super
> > difficult, but I hear you about wanting to not use another binary. But
> > I'll just shrug that off and say that's your problem :-)
> 
> Updating the grammar is not super-difficult but getting it right the
> first time is difficult.

I disagree, but that's okay.

> I cannot believe that nobody in parser-land has written a Python-based
> Python parser that we can hack. Whatever happened to the ethic that a
> parser-generator was not done until it could parse the language it was
> written in? That a Real Programming Language was not done until it could
> compile itself? :)

I think a number of people have done this. Go take a look for it.

One of my projects for 1.6 is to insert a hook for the parsing and the
compilation process. This would allow Python-level code to replace the
parse step, and/or Python code to replace the bytecode compilation. Once
those hooks are in, then it will be pretty much a given to have a Python
parser written in Python. And a bytecode compiler written in Python.
Specifically: can you replace each step, and parse the same files,
producing the same set of bytecodes?
[ I'll be doing the hooks; not sure if I want to write the replacements,
  though ]

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/