> About the migration, can I ask who is going to (help
to) fix projects which rely on the AST?
Whoops, I send the latest email before finishing it by mistake. Here is the extended version of the answer:
I think there is a misunderstanding here: The new parser generates the same AST as the old parser so
calling ast.parse() or compile() will yield exactly the same result. We have extensive testing around that
and that was a goal from the beginning. Projects using the ast module will not need to do anything special.
I think that's only half true. It's true if they already work with Python 3.9 (master/HEAD). But probably some of these packages have not yet started testing with 3.9 nightly runs or even alphas, so it's at least *conceivable* that some of the fixes we applied to the AST could require (small) adjustments. And I think *that* was what Victor was referring to. (For example, I'm not 100% sure that mypy actually works with the latest 3.9. But there seems to be something else wrong there so I can't even test it.)
The difference is that the new parser does not generate a CST (Concrete Syntax Tree). The concrete syntax tree
is an immediate structure from where the AST is generated. This structure is only partially exposed via the
"parser" module but otherwise is only used in the parser itself so it should not be a problem. On the other hand:
as explained in the PEP, the lack of the CST greatly simplifies the AST generation among other advantages.