[IronPython] Using IronPython AST

Dino Viehland dinov at microsoft.com
Wed Oct 8 23:26:19 CEST 2008

You're right on #1 - for other nodes you could also look at how the parser creates them (or ask if there's something particularly tricky).

For #2 there isn't a good way to do this right now.  You could cheat and do it with private reflection for the time being.  First you need to create a PythonAst object (no problem, it's public).  That will hold onto the tree you created and lets you control some knobs on how the code gen will proceed.  You just need to call PythonNameBinder.BindAst(ast, compilerContext) (which is internal so reflection use #1) and then ast.TransformToAst (also internal, reflection use #2) which will return you a DLR LambdaExpression.  From there you could go to the DLR and ask them to compile the lambda (Expression<T>.Compile) or you could produce a ScriptCode object for it which you can run.  The way we produce a ScriptCode once we have the lambda is:

            if ((pythonOptions.Module & ModuleOptions.Optimized) != 0) {
                return new OptimizedScriptCode(lambda, sourceUnit);
            } else {
                // TODO: fix generated DLR ASTs
                lambda = new GlobalLookupRewriter().RewriteLambda(lambda);
                return new ScriptCode(lambda, sourceUnit);

And you can basically do the same thing.  The 1st path will leak memory if you're doing it repeatedly so you probably want the re-write & normal ScriptCode.

We could probably do a little refactoring to expose this functionality onto PythonAst so that you can just new one of those up and get a ScriptCode back but it seems too much like a feature for it to make 2.0 at this point.

On #3 unfortunately I didn't get our CodeDom code generator into 2.0 in time so it'll wait for the next release - mainly because I didn't have a chance to port the v1.0 tests forward.  But it's pretty easy to grab it from the v1.1 sources and update it as our AST hasn't changed very much.

-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Dan Eloff
Sent: Wednesday, October 08, 2008 1:42 PM
To: Discussion of IronPython
Subject: [IronPython] Using IronPython AST

I'm looking at the ast stuff in IronPython.Compiler.Ast and seeing how
difficult it would be to write a python 2.5 _ast (and 2.6 ast) wrapper
over it.

Jython actually supports this now, and I don't wish to see IronPython
left behind in this area.

It looks like I should be able to handle a read-only ast without much
difficulty. The trouble comes with altering the ast. It seems from
looking at the code that everything is marked readonly, the ast cannot
be modified in place?

This is not a blocker, it's possible to handle all modifications in
the wrapper, and then generate a new ast at the end that can be

For example:

node = UnaryOp()
node.op = USub()
node.operand = Num()
node.operand.n = 5

1) How would you turn this into an ast? Something like this maybe?

node = new UnaryExpression(PythonOperator.Negate, new ConstantExpression(5))

2) I assume there is a mechanism for compiling and/or executing an
ast, please could someone show how to use it (you can build off the
script below)

3) Is there a mechanism for turning an ast back into source code? This
would help me with debugging.


Thanks to Dino for showing how to get an ast:

import clr
from IronPython.Compiler import Parser

from Microsoft.Scripting import ErrorSink
from Microsoft.Scripting.Runtime import CompilerContext
from Microsoft.Scripting.Hosting.Providers import HostingHelpers
from IronPython.Hosting import Python
py = Python.CreateEngine() # beta 5 and beyond

src = HostingHelpers.GetSourceUnit(py.CreateScriptSourceFromString('print
pylc = HostingHelpers.GetLanguageContext(py)

p = Parser.CreateParser(CompilerContext(src,
pylc.GetCompilerOptions(), ErrorSink.Default), pylc.Options)
ast = p.ParseFile(True)
Users mailing list
Users at lists.ironpython.com

More information about the Ironpython-users mailing list