I have been trying to hire somebody to implement the greenlet extension module, which would enable modules like eventlet and gevent to work on IronPython. The person I currently have looking into it, however, isn't hopeful. A condensed version of his perspective is below. Perhaps the IronPython list (Dino?) has an idea for a solution?<div>
<div><div><br></div><div>== Response to Greenlets on IronPython == </div><div><br></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">I looked at various solutions but none of them seem to arrive at the correct solution. I did try generating the DLR using AST again to make the function stackless but the limitation is with python and .NET. </span></div>
<div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">Let me explain the issue. <br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">1) .NET lanugages (C#. <a href="http://VB.NET">VB.NET</a> etc.) are strongly typed. In other words, if I get a variable I know what its type is at compile time. IronPython actually compiles the functions into CLR and runs them on the fly. A function is compiled once and can be executed again and again. One must know that the function needs to be stackless at compile time and a function cannot be made stackless at run time.<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">2) Python is a typeless language. A variable can assume any type at any point in execution at run-time. Now consider the following code in Python using the greenlet API:<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">a = greenlet(foo)<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">def abc():<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
print "X"<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "> a.switch()<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
print "Y"<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
Although variable a is defined, it does not have a strong type and during execution, a can be assigned a different object which can also implement the switch function and the code will execute fine.<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">As mentioned above, IronPython needs to know at compile time that the function has to be made stackless. In python since a is not typed, a can be greenlet in one execution of abc and any other object in the second call. Also, there is no way to determine at compile that the function abc needs to be made stackless since it needs to switch since the type of a is unknown till it is actually called at runtime. <br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
<br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hence is it next to impossible to implement the switch because of the above implementation.</span></div>
<div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
CPython dos not compile the functions and does not use the yield keyword (which is not avilable in C/C++ anyways) to do it. It does this by using context switching which is built into python and so the API is fairly straightforward in CPython. CPython basically has the stack of all previous functions calls and saves the stack so that it can resume later from that point. It needs to store the context and uses the inherent context switching built into Python to do it.</span></div>
<div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">== End Response ==</span></div>
<div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">Ideas?</span></div>
<div><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></span></div><div>Tristan</div><div><br><div class="gmail_quote">On Mon, May 31, 2010 at 1:28 PM, Dino Viehland <span dir="ltr"><<a href="mailto:dinov@microsoft.com">dinov@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">You’ll want to first look at GeneratorRewriter.cs - it’s responsible for taking a DLR AST and re-writing it so that it can re-start at an arbitrary location
and so that it’s stackless. There’s one of these for debugging and one that’s used for IronPython’s generators, you can probably adapt either one. You can get the AST to re-write from the function code object but that’s not public right now (we’ll make it
public if anyone has a compelling scenario to do so – this could be it but you’ll probably need more public surface area as well).</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">From there I think you just need to worry about having to save multiple frames (which I get the impression is supported from the docs – eg f() calls g() and
then g switches). For that you’ll probably want to re-write all of the dynamic expressions in the generator (these show up in several forms though based upon how we’re generating code- there’s a ReduciableDynamicExpression and some PythonDynamicExpression’s
floating around now). If greenlets only works across calls then you just need to intercept the ones which are using invoke binders. If you can have a descriptor who’s __get__ or __set__ switches then you’d also need to handle gets/sets/etc… If you can do
something like import a module which hasn’t been loaded yet and then have the top-level module code switch then you’ll need to start getting really creative
</span><span style="font-size:11.0pt;font-family:Wingdings;color:#1F497D">J</span><span style="font-size:11.0pt;color:#1F497D"> You’ll then presumably need to change these into using some trampoline to do the actual invocation.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Hopefully that will get you started – I’m sure there’ll be lots of little issues so if you have questions feel free to ask. If you get it all working we
can figure out what exactly we need to make public and how we should expose that so you can do this as an extension – but I’m sure you’ll initially be using a bunch of internal APIs.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Tristan Zajonc<br>
<b>Sent:</b> Sunday, May 30, 2010 6:59 PM<br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> [IronPython] Implementing Greenlets API in IronPython</span></p>
</div>
</div><div><div></div><div class="h5">
<p class="MsoNormal"> </p>
<p class="MsoNormal">Hi -</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">The greenlets C-extension module (<a href="http://packages.python.org/greenlet/" target="_blank">http://packages.python.org/greenlet/</a>) provides a very basic api that is used by eventlet and gevent to provide asynchronous programming constructs. It
would be nice to have a reasonably performant version of the greenlet API in IronPython.</p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Before I start playing around with possibilities, is there an obvious approach to implement the greenlets API in IronPython?</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Thanks</p>
</div>
<div>
<p class="MsoNormal">Tristan</p>
</div>
</div>
</div>
</div></div></div>
</div>
</div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
<br></blockquote></div><br></div></div></div>