[Python-Dev] Parser module in the stdlib

[Steve Dower]

On 16May2019 1548, Pablo Galindo Salgado wrote:
>  > Will the folks using forks be happy to switch to the stdlib version?
>>For example I can imagine that if black wants to process 3.7 input
>>code while running on 3.6, it might prefer a parser on PyPI even if
>>he stdlib version were public, since the PyPI version can be updated
>>independently of the host Python.
> The tool can parse arbitrary grammars, the one that is packed into is 
> just one of them.
> 
> I think it would be useful, among other things because the standard library
> lacks currently a proper CST solution. The ast module is heavily 
> leveraged for
> things like formatters, static code analyzers...etc but CST can be very 
> useful as
> Łukasz describes here:
> 
> https://bugs.python.org/issue33337
> 
> I think is missing an important gap in the stdlib and the closest thing 
> we have
> (the current parser module) is not useful for any of that. Also, the 
> core to generating
> the hypothetical new package (with some new API over it may be) is 
> already undocumented
> as an implementation detail of lib2to3 (and some people are already 
> using it directly).

We still have the policy of not removing modules that exist in the 
Python 2 standard library. But 3.9 won't be covered by that :)

But I'm in favor of having a proper CST module that matches the version 
of Python it's in. It doesn't help people on earlier versions (yet), but 
given how closely tied it is to the Python version you're on I think it 
makes sense in the stdlib.

Cheers,
Steve