[Python-Dev] Policy for making changes to the AST

[Nick Coghlan]

On Tue, Apr 5, 2011 at 7:49 PM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> Historically, that's incorrect. The AST structure was introduced to
> simplify the implementation of (C)Python. Exposing it to Python modules
> was done primarily because it's neat and was easy to do - not because
> any specific use was expected, and certainly not as the primary
> motivation for having an AST. It was clear (to me, at least) back then
> that the AST will change over time (hence the module was originally
> called _ast).

_ast is actually still there under the hood - the Python just added
some nicer tools for working with it.

It's probably worth mentioning the specific non-new-feature-related
changes to the AST that Eugene proposed on the tracker:

1. Making "docstring" an attribute of the Function node rather than
leaving it embedded as the first statement in the suite (this avoids
issues where AST-based constant folding could potentially corrupt the
docstring)
2. Collapsing Num, Str, Bytes, Ellipsis into a single Literal node
type (the handling of those nodes is the same in a lot of cases)
3. Since they're keywords now, pick up True, False, None at the
parsing stage and turn them into instances of the Literal node type,
allowing the current Name-based special casing to be removed.

These are the proposed changes that would be visible to someone using
the ast.PyCF_ONLY_AST flag. Any further changes (i.e. the actual
constant folding) aren't exposed in the AST produced by that flag -
those changes happen as a pre-transformation step in the process of
turning the submitted AST into CPython bytecode.

As I said in the tracker item, I actually *like* those 3 changes and
think they streamline the AST (and the associated code) quite nicely.
However, I didn't want to tell Eugene to proceed on that basis without
getting feedback from a wider audience first.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia