changing the Python grammar ?

[Michael Hudson]

mis6 at pitt.edu (Michele Simionato) writes:

> Browsing on the on-line refencence manual you can find the formal
> definition of the Python grammar: http://python.org/doc/current/ref/grammar.txt

Bear in mind that that is a descriptive grammar.  The real one is
Grammar/Grammar in the source distribution.

> But now a question arise to me: can I modify the grammar ??

If yo're prepared to build from source, yes.

> Suppose for instance I want to make able Python to recognize identifiers
> starting with some funny symbol, say "@". All I must do is to modify
> the first line of the grammar as 

Ah, that's the lexer.

> identifier ::= 
>                ["@"] (letter|"_") (letter | digit | "_")*
> 
> In the best of the worlds, I would look for some file in the Python
> source distribution containing the grammar, I would change that line, 
> and recompiling I would have a new Python able to understand identifiers
> starting with "@". Is this possible in practice ?

You're asking two questions here (though you might not realize it):

Hacking the Grammar is certainly possible.  You then usually need to
hack Python/compile.c too.  This can be hard.  Python/compile.c is not
the friendliest code in the world.

However, to what you want to do, you need to hack Parser/tokenizer.c
(I think).  This also isn't the friendliest code in the world, but
it's not too bad.  I've done similar things without too much wailing
and gnashing of teeth.

> Notice that I do NOT want to introduce ugly perlish-like identifiers in
> Python, I simply want to know how much it is possible to customize the
> Python grammar: is this a simple hack or a nightmare ?

Somewhere between those.  As your customizations get more ambitious,
you'll probably find yourself rapidly heading for the "nightmare" end
of that scale.

Cheers,
M.

-- 
 Two decades later, well-known hacker Henry Spencer described the 
 Perl scripting language as a "Swiss-Army chainsaw", intending to 
 convey his evaluation of the language as exceedingly powerful but 
 ugly and noisy and prone to belch noxious fumes.   -- the jargon file