I've recently written a short Informational PEP describing all the places that may need updating when you change Python's Grammar.
Naturally, it's on the web at
I'd be very grateful if people could scan through it and tell me about or just add anything I've missed, particularly the Jython issues.
PEP: 306 Title: How to Change Python's Grammar Version: $Revision: 1.2 $ Last-Modified: $Date: 2003/01/30 10:27:45 $ Author: Michael Hudson firstname.lastname@example.org Status: Draft Type: Informational Content-Type: text/plain Created: 29-Jan-2003 Post-History:
There's more to changing Python's grammar than editing Grammar/Grammar and Python/compile.c. This PEP aims to be a checklist of places that must also be fixed.
It is probably incomplete. If you see omissions, just add them if you can -- you are not going to offend the author's sense of ownership. Otherwise submit a bug or patch and assign it to mwh.
This PEP is not intended to be an instruction manual on Python grammar hacking, for several reasons.
People are getting this wrong all the time; it took well over a year before someone noticed that adding the floor division operator (//) broke the parser module.
__ Grammar/Grammar: OK, you'd probably worked this one out :)
__ Python/compile.c: you'll most likely need to edit both the actual compiler and the symtable builder (which is to say both the com_foo and the sym_foo functions).
__ You may need to regenerate Lib/symbol.py and/or Lib/token.py.
__ The parser module. Add some of your new syntax to test_parser, bang on parsermodule.c until it passes.
__ The compiler package. A good test is to compile the standard library and test suite with the compiler package and then check it runs. You did add some of your new syntax to the test suite, didn't you? There's a script in Tools/compiler that does this.
__ If you've gone so far as to change the token structure of Python, then the tokenizer library module will need to be changed.
__ Certain changes may require tweaks to the library module pycblr.
__ Jython too will need work, but I don't know what. Explanations gratefully received.
 SF Bug #676521, parser module validation failure http://www.python.org/sf/676521
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: