SUCCESS!<div><br></div><div>Here's the final code:</div><div><div>var runtime = Python.CreateRuntime();</div><div><br></div><div>var functionDefinition = new FunctionDefinition(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Do",</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>new[] { new Parameter("self") },</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>new ReturnStatement(new ConstantExpression("hello python!")));</div>
<div><br></div><div>var classDefinition = new IronPython.Compiler.Ast.ClassDefinition(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Example",</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>new Expression[] { },</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>functionDefinition);</div><div><br></div><div>dynamic python = runtime.Compile(classDefinition);</div><div>dynamic example = python.Example();</div><div><br>
</div><div>Console.WriteLine(example.Do());</div><div>Console.ReadKey(true);</div><div><br></div><div><div>public static class DLRHelper</div><div>{</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>public static dynamic Compile(this ScriptRuntime runtime, params Statement[] statements)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>{</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var engine = runtime.GetEngineByFileExtension(".py");</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var context = (PythonContext)HostingHelpers.GetLanguageContext(engine);</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var globals = new PythonDictionary();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>globals.Add("__name__", "<dynamic>");</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var options = new PythonCompilerOptions(ModuleOptions.ExecOrEvalCode);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var unit = new SourceUnit(context, NullTextContentProvider.Null, "", SourceCodeKind.AutoDetect);</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var moduleContext = new ModuleContext(globals, context);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var codeContext = moduleContext.GlobalContext;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var pythonAst = new PythonAst(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>new IronPython.Compiler.Ast.SuiteStatement(statements),</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>false,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ModuleOptions.ExecOrEvalCode,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>false,</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>new CompilerContext(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>unit,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>options,</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>ErrorSink.Default));</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>pythonAst.Walk(new EnsureLocWalker());</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>pythonAst.Bind();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var lambda = (System.Linq.Expressions.LambdaExpression)pythonAst.Reduce();</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>var func = (Func<CodeContext, FunctionCode, object>)lambda.Compile();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>var result = func(codeContext, null); // result is null</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>return HostingHelpers.CreateScriptScope(engine, codeContext.GlobalScope);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div>
</div><div><br></div><div><span class="Apple-tab-span" style="white-space: pre; "> </span>// I created a walker called EnsureLocWalker that overrides all Walk methods and calls SetLoc to new SourceSpan(new SourceLocation(1, 1, 1), new SourceLocation(2, 2, 2));</div>
<div><span class="Apple-tab-span" style="white-space: pre; "> </span>// and header when appropriate to new SourceLocation(1, 1, 1)</div><div>}</div><div><br></div><div><br></div><div>FYI your WithStatement, TryStatement and ForStatement are missing getters on their header property. That's frustrating. I want to be able to do this:</div>
<div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>if (node.Header == SourceLocation.Invalid)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>node.Header = new SourceLocation(1, 1, 1);</div>
</div><div><br></div><div>Also it would be nice if the headers and locations were completely ignored if they're invalid instead of throwing exceptions. Also it would be <i>reaaaally </i>nice if you had a Walk(Node node) override on the PythonWalker. So you could do something to all nodes. Also, why have a million overrides for this? Can't you just make a generic PythonWalker? Then just have a single Walk(T node) and AfterWalk(T node). That would be a lot less gross.</div>
<div><br></div><div>The reason why I want to do this is because I am working on a DSL tool (<a href="http://metasharp.codeplex.com">http://metasharp.codeplex.com</a>) where I'm compiling down into an AST. Currently I have transforms to create CodeDom objects and linq Expression objects but I lack the ability to dynamically create Types or directly generate IL. Using this I think i can save a TON of work for myself. </div>
<div><br></div><div>Thanks for all the help!</div><div><br></div><br><div class="gmail_quote">On Wed, Apr 14, 2010 at 6:21 PM, Dino Viehland <span dir="ltr"><<a href="mailto:dinov@microsoft.com">dinov@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Try calling SetLoc on the FunctionDefinition and also try
setting the Header property. I think if it’s some reasonable span it’ll work.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt">
<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>] <b>On
Behalf Of </b>Justin Chase<br>
<b>Sent:</b> Wednesday, April 14, 2010 3:35 PM</span></p><div><div></div><div class="h5"><br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython] Building via AST</div></div><p></p>
</div>
</div><div><div></div><div class="h5">
<p class="MsoNormal"> </p>
<p class="MsoNormal">Ok awesome!</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">So now I have:</p>
<div>
<p class="MsoNormal">dynamic example = globals["Example"];</p>
</div>
<div>
<p class="MsoNormal">dynamic instance = example();</p>
</div>
<div>
<p class="MsoNormal">Console.WriteLine(instance.Do());</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">But I'm getting an exception about a line number. Any idea
how I can get past that? I've tried a bunch of things but I'm not sure what's
going on.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">ArgumentOutOfRangeException</p>
</div>
<div>
<div>
<p class="MsoNormal">Specified argument was out of the range of valid
values. Parameter name: line must be greater than or equal to 1</p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> at
Microsoft.Scripting.SourceLocation.ValidateLocation(Int32 index, Int32 line,
Int32 column)</p>
</div>
<div>
<p class="MsoNormal"> at
Microsoft.Scripting.SourceLocation..ctor(Int32 index, Int32 line, Int32 column)</p>
</div>
<div>
<p class="MsoNormal"> at IronPython.Compiler.Ast.FunctionDefinition.CreateFunctionLambda()</p>
</div>
<div>
<p class="MsoNormal"> at
IronPython.Compiler.Ast.FunctionDefinition.EnsureFunctionLambda()</p>
</div>
<div>
<p class="MsoNormal"> at
IronPython.Compiler.Ast.FunctionDefinition.GetLambda()</p>
</div>
<div>
<p class="MsoNormal"> at IronPython.Runtime.FunctionCode.get_Code()</p>
</div>
<div>
<p class="MsoNormal"> at IronPython.Runtime.FunctionCode.GetGeneratorOrNormalLambda()</p>
</div>
<div>
<p class="MsoNormal"> at
IronPython.Runtime.FunctionCode.UpdateDelegate(PythonContext context, Boolean
forceCreation)</p>
</div>
<div>
<p class="MsoNormal"> at
IronPython.Runtime.FunctionCode.LazyCompileFirstTarget(PythonFunction function)</p>
</div>
<div>
<p class="MsoNormal"> at IronPython.Compiler.PythonCallTargets.OriginalCallTarget1(PythonFunction
function, Object arg0)</p>
</div>
<div>
<p class="MsoNormal"> at
IronPython.Runtime.PythonFunction.FunctionCaller`1.Call1(CallSite site,
CodeContext context, Object func, T0 arg0)</p>
</div>
<div>
<p class="MsoNormal"> at
System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site,
T0 arg0, T1 arg1, T2 arg2)</p>
</div>
<div>
<p class="MsoNormal"> at CallSite.Target(Closure , CallSite , Object
)</p>
</div>
<div>
<p class="MsoNormal"> at
System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0
arg0)</p>
</div>
<div>
<p class="MsoNormal"> at CallSite.Target(Closure , CallSite , Object
)</p>
</div>
<div>
<p class="MsoNormal"> at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite
site, T0 arg0)</p>
</div>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Wed, Apr 14, 2010 at 3:32 PM, Dino Viehland <<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>> wrote:</p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">The problem here is just that you
actually have a nested code context when for the top-level code you just need a
single global context. Replace the line:</span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal">
var codeContext = new CodeContext(</p>
<p class="MsoNormal">
new PythonDictionary(),</p>
<p class="MsoNormal">
moduleContext);</p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">with:</span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;color:#1F497D">var codeContext
= moduleContext.GlobalContext;</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">and it’ll work. The nested code
context that you’re creating is only used for when we’re inside of a function
definition or a class definition.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Also you can avoid creating the script
scope and just pull Example out of the globals dictionary as well.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>
[mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Justin Chase<br>
<b>Sent:</b> Wednesday, April 14, 2010 11:24 AM</span></p>
<div>
<div>
<p class="MsoNormal"><br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython] Building via AST</p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I
feel like I'm sooooo close but something isn't quite working. Here is my
current code:</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<div>
<p class="MsoNormal">
var runtime = Python.CreateRuntime();</p>
</div>
<div>
<p class="MsoNormal">
runtime.LoadAssembly(typeof(IExample).Assembly);</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var engine = runtime.GetEngineByFileExtension(".py");</p>
</div>
<div>
<p class="MsoNormal">
var context = (PythonContext)HostingHelpers.GetLanguageContext(engine);</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var globals = new PythonDictionary();</p>
</div>
<div>
<p class="MsoNormal">
globals.Add("__name__", "<test>"); // I get an
exception without this.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var options = new PythonCompilerOptions(ModuleOptions.ExecOrEvalCode |
ModuleOptions.Initialize);</p>
</div>
<div>
<p class="MsoNormal">
var unit = new SourceUnit(context, NullTextContentProvider.Null, "",
SourceCodeKind.Statements);</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var moduleContext = new ModuleContext(globals, context);</p>
</div>
<div>
<p class="MsoNormal">
var codeContext = new CodeContext(</p>
</div>
<div>
<p class="MsoNormal">
new PythonDictionary(),</p>
</div>
<div>
<p class="MsoNormal">
moduleContext);</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var classDefinition = new IronPython.Compiler.Ast.ClassDefinition(</p>
</div>
<div>
<p class="MsoNormal">
"Example",</p>
</div>
<div>
<p class="MsoNormal">
new Expression[] { },</p>
</div>
<div>
<p class="MsoNormal">
new FunctionDefinition(</p>
</div>
<div>
<p class="MsoNormal">
"Do",</p>
</div>
<div>
<p class="MsoNormal">
new[] { new Parameter("self") },</p>
</div>
<div>
<p class="MsoNormal">
new ReturnStatement(new ConstantExpression("hello python!"))));</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
var pythonAst = new PythonAst(</p>
</div>
<div>
<p class="MsoNormal">
new IronPython.Compiler.Ast.SuiteStatement(new Statement[] { /* import, */
classDefinition }),</p>
</div>
<div>
<p class="MsoNormal">
false,</p>
</div>
<div>
<p class="MsoNormal">
ModuleOptions.ExecOrEvalCode,</p>
</div>
<div>
<p class="MsoNormal">
false,</p>
</div>
<div>
<p class="MsoNormal">
new CompilerContext(</p>
</div>
<div>
<p class="MsoNormal">
unit,</p>
</div>
<div>
<p class="MsoNormal">
options,</p>
</div>
<div>
<p class="MsoNormal">
new ThrowsErrorSink()));</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
pythonAst.Bind();</p>
</div>
<div>
<p class="MsoNormal">
var lambda = (System.Linq.Expressions.LambdaExpression)pythonAst.Reduce();</p>
</div>
<div>
<p class="MsoNormal">
var func = (Func<CodeContext, FunctionCode, object>)lambda.Compile();</p>
</div>
<div>
<p class="MsoNormal">
var result = func(codeContext, null); // result is null</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">
dynamic python = HostingHelpers.CreateScriptScope(engine,
moduleContext.GlobalScope);</p>
</div>
<div>
<p class="MsoNormal">
<b>dynamic example = python.Example(); // fails! Example is not in scope.</b></p>
</div>
<div>
<p class="MsoNormal">
Console.WriteLine(example.Do());</p>
</div>
<div>
<p class="MsoNormal">
Console.ReadKey(true);</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Can
anyone tell me what I'm missing? This has got to be pretty close.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On
Mon, Apr 12, 2010 at 7:51 PM, Justin Chase <<a href="mailto:justin.m.chase@gmail.com" target="_blank">justin.m.chase@gmail.com</a>>
wrote:</p>
<p>Awesome. I will thanks. </p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">On Apr
12, 2010 7:49 PM, "Dino Viehland" <<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>>
wrote:</p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">This might be possible. If you
wrap this all up in a PythonAst object (calling the constructor which takes a
CompilerContext), call Bind on it then you should get a LambdaExpression back
out. You can Compile() on that. </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">But it’s not like this is well traveled
territory and this only applies to 2.6.1 (before that the trees weren’t DLR
ASTs so they weren’t reducable). When we do this ourselves we also call
the produced delegate and flow in some data. The delegate is going to
want at least a FunctionCode object as an argument but I think you could get
away with passing null (at least as long as no exceptions are thrown).
The delegate might also want a CodeContext object as well depending on the
compilation mode we end up using (which is based on the CompilerContext you
give us). This you wouldn’t be able to get away w/ passing null.
But you can get one by doing new ModuleContext(new PythonDictionary(),
pythonContext).GlobalContext. The HostingHelpers class can give you a
LanguageContext from the ScriptEngine for Python which you can cast to a
PythonContext.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Let me know if it works! </span><span style="font-size:11.0pt;font-family:Wingdings;color:#1F497D">J</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
</div>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>
[mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Justin Chase<br>
<b>Sent:</b> Monday, April 12, 2010 4:09 PM</span></p>
<p><span style="color:#500050">To: Discussion of IronPython Subject: Re:
[IronPython] Building via AST</span></p>
</div>
</div>
</div>
<p><span style="color:#500050"> Ok, so at risk of being a nuissance I
have one last question because I feel like I'm half way t...</span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<br>
-- <br>
Justin Chase <br>
<a href="http://www.justnbusiness.com" target="_blank">http://www.justnbusiness.com</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<br>
-- <br>
Justin Chase <br>
<a href="http://www.justnbusiness.com" target="_blank">http://www.justnbusiness.com</a></p>
</div>
</div></div></div>
</div>
</div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Justin Chase <br><a href="http://www.justnbusiness.com">http://www.justnbusiness.com</a><br>
</div>