[IronPython] Using IronPython AST

[Dan Eloff]

#1 - Great, I can manage that.

#2 - This is promising. Because I want this to work in silverlight as
well, I'll have to patch IronPython to give a public interface, but
that looks remarkably simple from what you've told me. That's fine for
enabling me to continue forward, but not for sharing my work on this
later. Any chance we can get a public interface? Ideally if I pull
this off, my work should find its way into IronPython, but I
understand that will have to wait until the legal hurdles with
accepting community contributions have been cleared.

#3 - Sounds good, it's C# so fortuantly the compiler will flag most of
the breakages for me.

Thanks,

-Dan

On Wed, Oct 8, 2008 at 4:26 PM, Dino Viehland <dinov at microsoft.com> wrote:
> You're right on #1 - for other nodes you could also look at how the parser creates them (or ask if there's something particularly tricky).
>
> For #2 there isn't a good way to do this right now.  You could cheat and do it with private reflection for the time being.  First you need to create a PythonAst object (no problem, it's public).  That will hold onto the tree you created and lets you control some knobs on how the code gen will proceed.  You just need to call PythonNameBinder.BindAst(ast, compilerContext) (which is internal so reflection use #1) and then ast.TransformToAst (also internal, reflection use #2) which will return you a DLR LambdaExpression.  From there you could go to the DLR and ask them to compile the lambda (Expression<T>.Compile) or you could produce a ScriptCode object for it which you can run.  The way we produce a ScriptCode once we have the lambda is:
>
>            if ((pythonOptions.Module & ModuleOptions.Optimized) != 0) {
>                return new OptimizedScriptCode(lambda, sourceUnit);
>            } else {
>                // TODO: fix generated DLR ASTs
>                lambda = new GlobalLookupRewriter().RewriteLambda(lambda);
>                return new ScriptCode(lambda, sourceUnit);
>            }
>
> And you can basically do the same thing.  The 1st path will leak memory if you're doing it repeatedly so you probably want the re-write & normal ScriptCode.
>
> We could probably do a little refactoring to expose this functionality onto PythonAst so that you can just new one of those up and get a ScriptCode back but it seems too much like a feature for it to make 2.0 at this point.
>
> On #3 unfortunately I didn't get our CodeDom code generator into 2.0 in time so it'll wait for the next release - mainly because I didn't have a chance to port the v1.0 tests forward.  But it's pretty easy to grab it from the v1.1 sources and update it as our AST hasn't changed very much.
>
> -----Original Message-----
> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Dan Eloff
> Sent: Wednesday, October 08, 2008 1:42 PM
> To: Discussion of IronPython
> Subject: [IronPython] Using IronPython AST
>
> I'm looking at the ast stuff in IronPython.Compiler.Ast and seeing how
> difficult it would be to write a python 2.5 _ast (and 2.6 ast) wrapper
> over it.
>
> Jython actually supports this now, and I don't wish to see IronPython
> left behind in this area.
>
> It looks like I should be able to handle a read-only ast without much
> difficulty. The trouble comes with altering the ast. It seems from
> looking at the code that everything is marked readonly, the ast cannot
> be modified in place?
>
> This is not a blocker, it's possible to handle all modifications in
> the wrapper, and then generate a new ast at the end that can be
> evaluated.
>
> For example:
>
> node = UnaryOp()
> node.op = USub()
> node.operand = Num()
> node.operand.n = 5
>
> 1) How would you turn this into an ast? Something like this maybe?
>
> node = new UnaryExpression(PythonOperator.Negate, new ConstantExpression(5))
>
> 2) I assume there is a mechanism for compiling and/or executing an
> ast, please could someone show how to use it (you can build off the
> script below)
>
> 3) Is there a mechanism for turning an ast back into source code? This
> would help me with debugging.
>
> Thanks,
> -Dan
>
> Thanks to Dino for showing how to get an ast:
>
> import clr
> clr.AddReference('IronPython')
> clr.AddReference('Microsoft.Scripting')
> clr.AddReference('Microsoft.Scripting.Core')
> from IronPython.Compiler import Parser
>
> from Microsoft.Scripting import ErrorSink
> from Microsoft.Scripting.Runtime import CompilerContext
> from Microsoft.Scripting.Hosting.Providers import HostingHelpers
> from IronPython.Hosting import Python
> py = Python.CreateEngine() # beta 5 and beyond
>
> src = HostingHelpers.GetSourceUnit(py.CreateScriptSourceFromString('print
> "hello"'))
> pylc = HostingHelpers.GetLanguageContext(py)
>
>
> p = Parser.CreateParser(CompilerContext(src,
> pylc.GetCompilerOptions(), ErrorSink.Default), pylc.Options)
> ast = p.ParseFile(True)
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>