[Python-Dev] Memory management in the AST parser & compiler

[Thomas Lee]

Nick Coghlan wrote:

>Greg Ewing wrote:
>  
>
>>Neal Norwitz wrote:
>>
>>    
>>
>>>I'm mostly convinced that using PyObjects would be a good thing. 
>>>However, making the change isn't free as all the types need to be
>>>created and this is likely quite a bit of code.
>>>      
>>>
>>Since they're all so similar, perhaps they could be
>>auto-generated by a fairly simple script?
>>
>>(I'm being very careful not to suggest using Pyrex
>>for this, as I can appreciate the desire not to make
>>such a fundamental part of the core dependent on it!)
>>    
>>
>
>The ast C structs are already auto-generated by a Python script (asdl_c.py, to 
>be precise). The trick is to make that script generate full PyObjects rather 
>than the simple C structures that it generates now.
>
>  
>
I was actually trying this approach last night. I'm back to it this 
evening, working with the ast-objects branch. I'll push a patch tonight 
with whatever I get done.

Quick semi-related question: where are the marshal_* functions called? 
They're all static in Python-ast.c and don't seem to be actually called 
anywhere. Can we ditch them?

>The second step is to then modify ast.c to use the new structures. A branch 
>probably wouldn't help much with initial development (this is a "break the 
>world, check in when stuff compiles again" kind of change, which is hard to 
>split amongst multiple people), but I think it would be of benefit when 
>reviewing the change before moving it back to the trunk.
>
>  
>
Based on my (limited) experience and your approach, compile.c may also 
need to be modified a little too (this should be pretty trivial).

Cheers,
Tom