[Python-Dev] Memory management in the AST parser & compiler

[Guido van Rossum]

On 11/28/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> [...] My mental model
> of parsing & compiling in the presence of a parse tree
> is like this:
>
>    [source] -> scanner -> [tokens]
>      -> parser -> [AST] -> code_generator -> [code]
>
> The fact that there still seems to be another kind of
> parse tree in between the scanner and the AST generator
> is an oddity which I hope will eventually disappear.

Have a look at http://python.org/sf/1337696 -- a reimplementation of
pgen in Python that I did for Elemental and am contributing to the
PSF. It customizes the tree generation callback so as to let you
produce an style of AST you like.

> > I know
> > Guido has said he doesn't like punishing the performance of small
> > scripts in the name of large-scale apps
>
> To me, that's an argument in favour of always generating
> a .pyc, even for scripts.

I'm not sure I follow the connection. But I wouldn't mind if someone
contributed code that did this. :)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)