[Python-Dev] C AST to Python discussion

Nick Coghlan ncoghlan at gmail.com
Wed Feb 15 10:28:36 CET 2006

Greg Ewing wrote:
> Brett Cannon wrote:
>> One protects us from ending up with an unusable AST since
>> the seralization can keep the original AST around and if the version
>> passed back in from Python code is junk it can be tossed and the
>> original version used.
> I don't understand why this is an issue. If Python code
> produces junk and tries to use it as an AST, then it's
> buggy and deserves what it gets. All the AST compiler
> should be responsible for is to try not to crash the
> interpreter under those conditions. But that's true
> whatever method is used for passing ASTs from Python
> to the compiler.

I'd prefer the AST node be real Python objects. The arena approach seems to be 
working reasonably well, but I still don't see a good reason for using a 
specialised memory allocation scheme when it really isn't necessary and we 
have a perfectly good memory management system for PyObject's.

On the 'unusable AST' front, if AST transformation code creates illegal 
output, then the main thing is to raise an exception complaining about what's 
wrong with it. I believe that may need a change to the compiler whether the 
modified AST was serialised or not.

In terms of reverting back to the untransformed AST if the transformation 
fails, then that option is up to the code doing the transformation. Instead of 
serialising all the time (even for cases where the AST is just being inspected 
instead of transformed), we can either let the AST objects support the 
copy/deepcopy protocol, or else provide a method to clone a tree before trying 
to transform it.

A unified representation means we only have one API to learn, that is 
accessible from both Python and C. It also eliminates any need to either 
implement features twice (once in Python and once in C) or else let the Python 
and C API's diverge to the point where what you can do with one differs from 
what you can do with the other.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list