[IronPython] Building via AST
Justin Chase
justin.m.chase at gmail.com
Thu Apr 15 20:25:09 CEST 2010
SUCCESS!
Here's the final code:
var runtime = Python.CreateRuntime();
var functionDefinition = new FunctionDefinition(
"Do",
new[] { new Parameter("self") },
new ReturnStatement(new ConstantExpression("hello python!")));
var classDefinition = new IronPython.Compiler.Ast.ClassDefinition(
"Example",
new Expression[] { },
functionDefinition);
dynamic python = runtime.Compile(classDefinition);
dynamic example = python.Example();
Console.WriteLine(example.Do());
Console.ReadKey(true);
public static class DLRHelper
{
public static dynamic Compile(this ScriptRuntime runtime, params Statement[]
statements)
{
var engine = runtime.GetEngineByFileExtension(".py");
var context = (PythonContext)HostingHelpers.GetLanguageContext(engine);
var globals = new PythonDictionary();
globals.Add("__name__", "<dynamic>");
var options = new PythonCompilerOptions(ModuleOptions.ExecOrEvalCode);
var unit = new SourceUnit(context, NullTextContentProvider.Null, "",
SourceCodeKind.AutoDetect);
var moduleContext = new ModuleContext(globals, context);
var codeContext = moduleContext.GlobalContext;
var pythonAst = new PythonAst(
new IronPython.Compiler.Ast.SuiteStatement(statements),
false,
ModuleOptions.ExecOrEvalCode,
false,
new CompilerContext(
unit,
options,
ErrorSink.Default));
pythonAst.Walk(new EnsureLocWalker());
pythonAst.Bind();
var lambda = (System.Linq.Expressions.LambdaExpression)pythonAst.Reduce();
var func = (Func<CodeContext, FunctionCode, object>)lambda.Compile();
var result = func(codeContext, null); // result is null
return HostingHelpers.CreateScriptScope(engine, codeContext.GlobalScope);
}
// I created a walker called EnsureLocWalker that overrides all Walk methods
and calls SetLoc to new SourceSpan(new SourceLocation(1, 1, 1), new
SourceLocation(2, 2, 2));
// and header when appropriate to new SourceLocation(1, 1, 1)
}
FYI your WithStatement, TryStatement and ForStatement are missing getters on
their header property. That's frustrating. I want to be able to do this:
if (node.Header == SourceLocation.Invalid)
node.Header = new SourceLocation(1, 1, 1);
Also it would be nice if the headers and locations were completely ignored
if they're invalid instead of throwing exceptions. Also it would be *reaaaally
*nice if you had a Walk(Node node) override on the PythonWalker. So you
could do something to all nodes. Also, why have a million overrides for
this? Can't you just make a generic PythonWalker? Then just have a single
Walk(T node) and AfterWalk(T node). That would be a lot less gross.
The reason why I want to do this is because I am working on a DSL tool (
http://metasharp.codeplex.com) where I'm compiling down into an AST.
Currently I have transforms to create CodeDom objects and linq Expression
objects but I lack the ability to dynamically create Types or directly
generate IL. Using this I think i can save a TON of work for myself.
Thanks for all the help!
On Wed, Apr 14, 2010 at 6:21 PM, Dino Viehland <dinov at microsoft.com> wrote:
> Try calling SetLoc on the FunctionDefinition and also try setting the
> Header property. I think if it’s some reasonable span it’ll work.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Justin Chase
> *Sent:* Wednesday, April 14, 2010 3:35 PM
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Building via AST
>
>
>
> Ok awesome!
>
>
>
> So now I have:
>
> dynamic example = globals["Example"];
>
> dynamic instance = example();
>
> Console.WriteLine(instance.Do());
>
>
>
> But I'm getting an exception about a line number. Any idea how I can get
> past that? I've tried a bunch of things but I'm not sure what's going on.
>
>
>
> ArgumentOutOfRangeException
>
> Specified argument was out of the range of valid values. Parameter name:
> line must be greater than or equal to 1
>
> at Microsoft.Scripting.SourceLocation.ValidateLocation(Int32 index,
> Int32 line, Int32 column)
>
> at Microsoft.Scripting.SourceLocation..ctor(Int32 index, Int32 line,
> Int32 column)
>
> at IronPython.Compiler.Ast.FunctionDefinition.CreateFunctionLambda()
>
> at IronPython.Compiler.Ast.FunctionDefinition.EnsureFunctionLambda()
>
> at IronPython.Compiler.Ast.FunctionDefinition.GetLambda()
>
> at IronPython.Runtime.FunctionCode.get_Code()
>
> at IronPython.Runtime.FunctionCode.GetGeneratorOrNormalLambda()
>
> at IronPython.Runtime.FunctionCode.UpdateDelegate(PythonContext context,
> Boolean forceCreation)
>
> at IronPython.Runtime.FunctionCode.LazyCompileFirstTarget(PythonFunction
> function)
>
> at
> IronPython.Compiler.PythonCallTargets.OriginalCallTarget1(PythonFunction
> function, Object arg0)
>
> at IronPython.Runtime.PythonFunction.FunctionCaller`1.Call1(CallSite
> site, CodeContext context, Object func, T0 arg0)
>
> at
> System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite
> site, T0 arg0, T1 arg1, T2 arg2)
>
> at CallSite.Target(Closure , CallSite , Object )
>
> at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite
> site, T0 arg0)
>
> at CallSite.Target(Closure , CallSite , Object )
>
> at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite
> site, T0 arg0)
>
>
>
>
>
> On Wed, Apr 14, 2010 at 3:32 PM, Dino Viehland <dinov at microsoft.com>
> wrote:
>
> The problem here is just that you actually have a nested code context when
> for the top-level code you just need a single global context. Replace the
> line:
>
>
>
> var codeContext = new CodeContext(
>
> new PythonDictionary(),
>
> moduleContext);
>
>
>
> with:
>
> var codeContext = moduleContext.GlobalContext;
>
>
>
> and it’ll work. The nested code context that you’re creating is only used
> for when we’re inside of a function definition or a class definition.
>
>
>
> Also you can avoid creating the script scope and just pull Example out of
> the globals dictionary as well.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Justin Chase
> *Sent:* Wednesday, April 14, 2010 11:24 AM
>
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Building via AST
>
>
>
> I feel like I'm sooooo close but something isn't quite working. Here is my
> current code:
>
>
>
> var runtime = Python.CreateRuntime();
>
>
> runtime.LoadAssembly(typeof(IExample).Assembly);
>
>
>
> var engine =
> runtime.GetEngineByFileExtension(".py");
>
> var context =
> (PythonContext)HostingHelpers.GetLanguageContext(engine);
>
>
>
> var globals = new PythonDictionary();
>
> globals.Add("__name__", "<test>"); // I
> get an exception without this.
>
>
>
> var options = new
> PythonCompilerOptions(ModuleOptions.ExecOrEvalCode |
> ModuleOptions.Initialize);
>
> var unit = new SourceUnit(context,
> NullTextContentProvider.Null, "", SourceCodeKind.Statements);
>
>
>
> var moduleContext = new
> ModuleContext(globals, context);
>
> var codeContext = new CodeContext(
>
> new PythonDictionary(),
>
> moduleContext);
>
>
>
> var classDefinition = new
> IronPython.Compiler.Ast.ClassDefinition(
>
> "Example",
>
> new Expression[] { },
>
> new FunctionDefinition(
>
> "Do",
>
> new[] { new
> Parameter("self") },
>
> new
> ReturnStatement(new ConstantExpression("hello python!"))));
>
>
>
> var pythonAst = new PythonAst(
>
> new
> IronPython.Compiler.Ast.SuiteStatement(new Statement[] { /* import, */
> classDefinition }),
>
> false,
>
> ModuleOptions.ExecOrEvalCode,
>
> false,
>
> new CompilerContext(
>
> unit,
>
> options,
>
> new
> ThrowsErrorSink()));
>
>
>
> pythonAst.Bind();
>
> var lambda =
> (System.Linq.Expressions.LambdaExpression)pythonAst.Reduce();
>
> var func = (Func<CodeContext,
> FunctionCode, object>)lambda.Compile();
>
> var result = func(codeContext, null); //
> result is null
>
>
>
> dynamic python =
> HostingHelpers.CreateScriptScope(engine, moduleContext.GlobalScope);
>
> *dynamic example = python.Example(); //
> fails! Example is not in scope.*
>
> Console.WriteLine(example.Do());
>
> Console.ReadKey(true);
>
>
>
>
>
> Can anyone tell me what I'm missing? This has got to be pretty close.
>
>
>
>
>
> On Mon, Apr 12, 2010 at 7:51 PM, Justin Chase <justin.m.chase at gmail.com>
> wrote:
>
> Awesome. I will thanks.
>
> On Apr 12, 2010 7:49 PM, "Dino Viehland" <dinov at microsoft.com> wrote:
>
> This might be possible. If you wrap this all up in a PythonAst object
> (calling the constructor which takes a CompilerContext), call Bind on it
> then you should get a LambdaExpression back out. You can Compile() on
> that.
>
>
>
> But it’s not like this is well traveled territory and this only applies to
> 2.6.1 (before that the trees weren’t DLR ASTs so they weren’t reducable).
> When we do this ourselves we also call the produced delegate and flow in
> some data. The delegate is going to want at least a FunctionCode object as
> an argument but I think you could get away with passing null (at least as
> long as no exceptions are thrown). The delegate might also want a
> CodeContext object as well depending on the compilation mode we end up using
> (which is based on the CompilerContext you give us). This you wouldn’t be
> able to get away w/ passing null. But you can get one by doing new
> ModuleContext(new PythonDictionary(), pythonContext).GlobalContext. The
> HostingHelpers class can give you a LanguageContext from the ScriptEngine
> for Python which you can cast to a PythonContext.
>
>
>
> Let me know if it works! J
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Justin Chase
> *Sent:* Monday, April 12, 2010 4:09 PM
>
> To: Discussion of IronPython Subject: Re: [IronPython] Building via AST
>
> Ok, so at risk of being a nuissance I have one last question because I
> feel like I'm half way t...
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> --
> Justin Chase
> http://www.justnbusiness.com
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> --
> Justin Chase
> http://www.justnbusiness.com
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
--
Justin Chase
http://www.justnbusiness.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20100415/14790ca7/attachment.html>
More information about the Ironpython-users
mailing list