[Python-Dev] ast-objects branch created

[James Y Knight]

On Dec 5, 2005, at 8:46 AM, Jeremy Hylton wrote:

> On 12/5/05, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>
>> Jeremy Hylton wrote:
>> I would expect that you allocate in the process memory that needs to
>> outlive the arena, e.g. python strings. The fundamental problem is  
>> that
>> the arena deallocation cannot know whether such memory exists, and  
>> what
>> to do with it.
>>
>
> I can see that problem occurring with an all-or-nothing solution, but
> not if you have the freedom to allocate from an arena or from some
> other mechanism.  If there are multiple ways to allocate memory, there
> is some increased programming burden (you have to remember how each
> pointer was allocated) but you gain flexibility.  The ast-arena branch
> allocates most memory from an arena, but allocates identifiers on the
> regular heap as PyObjects.  It does keep a list of these PyObjects so
> that it can DECREF them later.

ISTM that having to remember which pointers are arena-allocated and  
which are normally-refcounted-allocated removes the major gain that  
an arena method is supposed to bring: resistance to mistakes. I'd  
find having a single way to allocate and track memory easier than  
multiple. Then you just have to follow the single set of best  
practices for memory management, and you're all set. (and with  
PyObjects, the same practices the rest of python uses, another win.)

I'd also like to parrot the concern others have had that if the AST  
nodes are not made of PyObjects, then a mirror hierarchy of PyObject- 
ified AST nodes will have to be created, which seems like quite a  
wasteful duplication. If it is required that there be a collection of  
AST python objects (which I think it is), is there really any good  
reason to make the "real" AST objects not be the _only_ AST objects?  
I've not heard one.

James