[Python-Dev] Memory management in the AST parser & compiler

[Brett Cannon]

On 11/16/05, Thomas Lee <krumms at gmail.com> wrote:
> Just messing around with some ideas. I was trying to avoid the ugly
> macros (note my earlier whinge about a learning curve) but they're the
> cleanest way I could think of to get around the problem without
> resorting to a mass deallocation right at the end of the AST run. Which
> may not be all that bad given we're going to keep everything in-memory
> anyway until an error occurs ... anyway, anyway, I'm getting sidetracked :)
>
> The idea is to ensure that all allocations within a single function are
> made using the pool so that a function finishes what it starts. This
> way, if the function fails it alone is responsible for cleaning up its
> own pool and that's all. No funkyness needed for sequences, because each
> member of the sequence belongs to the pool too. Note that the stmt_ty
> instances are also allocated using the pool.
>
> This breaks interfaces all over the place though. Not exactly a pretty
> change :) But yeah, maybe somebody smarter than I will come up with
> something a bit cleaner.
>
> --
>
> /* snip! */
>
> #define AST_SUCCESS(pool, result) return result
> #define AST_FAILURE(pool, result) asdl_pool_free(pool); return result
>

This is actually exactly what I was thinking of; macros that handle
returns and specify whether the return signals a success or failure.

One tweak I would do is posibly lock down the the variable name with
AST_POOL_ALLOC() at the start of a function that creates _arena_pool. 
That way you don't need to pass in the specific pool.  I don't see why
we will need to have multiple pools within a function.  This also
allows the VISIT_* macros to be easily modified and not suddenly
require another argument to specify the arena name.

And all of this is easy to police since you can grep for 'return' and
make sure that it is meant to be there and not in actuality be one of
the macros.  Basically gives us the mini-language that Nick mentioned
way back at the beginning of this thread.

Oh, and tweak the macros to be within ``do { ... } while(0)`` (``if
(1) AST_FAILURE(pool, NULL);`` will not expand properly otherwise).

-Brett