[Python-Dev] ast-objects branch created

[Jeremy Hylton]

On 12/5/05, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> Jeremy Hylton wrote:
> I would expect that you allocate in the process memory that needs to
> outlive the arena, e.g. python strings. The fundamental problem is that
> the arena deallocation cannot know whether such memory exists, and what
> to do with it.

I can see that problem occurring with an all-or-nothing solution, but
not if you have the freedom to allocate from an arena or from some
other mechanism.  If there are multiple ways to allocate memory, there
is some increased programming burden (you have to remember how each
pointer was allocated) but you gain flexibility.  The ast-arena branch
allocates most memory from an arena, but allocates identifiers on the
regular heap as PyObjects.  It does keep a list of these PyObjects so
that it can DECREF them later.

> IOW, there would be no problem if you were *completely* done when
> you throw away the arena. This is not the case, though: eventually
> you end up with byte code that need to persist.

Right.  The bytecode can't be allocated from the arena, but the AST
can.  The AST is an internal abstraction.

> > Sure does.  It seems like the code generation from the AST description
> > also makes this kind of experimentation easier.
>
> Indeed. I wish there was a way to generate ast.c from a transformation
> description as well.

I'm sure there's a way to generate a parser from the the description,
but that seemed like a bigger project than I wanted to tackle.  GIven
how long it took to finish the AST without a new parser, I think it
was a wise choice :-).

Jeremy