[Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Brett Cannon]

On Mon, Nov 13, 2017, 13:37 Greg Ewing, <greg.ewing at canterbury.ac.nz> wrote:

> Guido van Rossum wrote:
> > But Python's syntax changes in nearly every release.
>
> The changes are almost always additions, so there's no
> reason why the AST can't remain backwards compatible.
>
> > the AST level ... elides many details
> > (such as whitespace and parentheses).
>
> That's okay, because the AST is only expected to
> represent the semantics of Python code, not its
> exact lexical representation in the source. It's
> the same with Lisp -- comments and whitespace have
> been stripped out by the time you get to Lisp
> data.
>
> > Lisp had almost no syntax so I presume the mapping to data structures
> > was nearly trivial compared to Python.
>
> Yes, the Python AST is more complicated, but we
> already have that much complexity in the AST being
> used by the compiler.
>
> If I understand correctly, we also have a process
> for converting that internal structure to and from
> an equally complicated set of Python objects, that
> isn't needed by the compiler and exists purely for
> the convenience of Python code.
>

The internal and stdlib AST are generated from the AST definition which is
written in a DEAL. The conversion code is also auto-generated. (The
devguide has the details.)


> I can't see much complexity being added if we were
> to decide to standardise the Python representation.
>

Do you have a specific example in a recent release where a change was made
that you disapproved of? Those of us who have mutated the AST have tried to
not be gratuitous in the changes to begin with while also not letting the
AST make maintaining Python harder. Plus there are external libraries like
typed_ast and asteroid that try to make the AST uniform across releases so
we don't have to worry quite as much about this explicitly.

-brett


> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171113/9560e209/attachment.html>