[IronPython] Implementing Greenlets API in IronPython

[Tristan Zajonc]

I have been trying to hire somebody to implement the greenlet extension
module, which would enable modules like eventlet and gevent to work on
IronPython.  The person I currently have looking into it, however, isn't
hopeful. A condensed version of his perspective is below.  Perhaps the
IronPython list (Dino?) has an idea for a solution?

== Response to Greenlets on IronPython ==

I looked at various solutions but none of them seem to arrive at the correct
solution. I did try generating the DLR using AST again to make the function
stackless but the limitation is with python and .NET.

Let me explain the issue.

1) .NET lanugages (C#. VB.NET etc.) are strongly typed. In other words, if I
get a variable I know what its type is at compile time. IronPython actually
compiles the functions into CLR and runs them on the fly. A function is
compiled once and can be executed again and again. One must know that the
function needs to be stackless at compile time and a function cannot be made
stackless at run time.

2) Python is a typeless language. A variable can assume any type at any
point in execution at run-time. Now consider the following code in Python
using the greenlet API:

a = greenlet(foo)

def abc():
    print "X"
    a.switch()
    print "Y"

Although variable a is defined, it does not have a strong type and during
execution, a can be assigned a different object which can also implement the
switch function and the code will execute fine.

As mentioned above, IronPython needs to know at compile time that the
function has to be made stackless. In python since a is not typed, a can be
greenlet in one execution of abc and any other object in the second call.
Also, there is no way to determine at compile that the function abc needs to
be made stackless since it needs to switch since the type of a is unknown
till it is actually called at runtime.

Hence is it next to impossible to implement the switch because of the above
implementation.

CPython dos not compile the functions and does not use the yield keyword
(which is not avilable in C/C++ anyways) to do it. It does this by
using context switching which is built into python and so the API is
fairly straightforward in CPython. CPython basically has the stack of all
previous functions calls and saves the stack so that it can resume later
from that point. It needs to store the context and uses the inherent context
switching built into Python to do it.

== End Response ==

Ideas?

Tristan

On Mon, May 31, 2010 at 1:28 PM, Dino Viehland <dinov at microsoft.com> wrote:

>  You’ll want to first look at GeneratorRewriter.cs - it’s responsible for
> taking a DLR AST and re-writing it so that it can re-start at an arbitrary
> location and so that it’s stackless.  There’s one of these for debugging and
> one that’s used for IronPython’s generators, you can probably adapt either
> one.  You can get the AST to re-write from the function code object but
> that’s not public right now (we’ll make it public if anyone has a compelling
> scenario to do so – this could be it but you’ll probably need more public
> surface area as well).
>
>
>
> From there I think you just need to worry about having to save multiple
> frames (which I get the impression is supported from the docs – eg f() calls
> g() and then g switches).  For that you’ll probably want to re-write all of
> the dynamic expressions in the generator (these show up in several forms
> though based upon how we’re generating code-  there’s a
> ReduciableDynamicExpression and some PythonDynamicExpression’s floating
> around now).  If greenlets only works across calls then you just need to
> intercept the ones which are using invoke binders.  If you can have a
> descriptor who’s __get__ or __set__ switches then you’d also need to handle
> gets/sets/etc…  If you can do something like import a module which hasn’t
> been loaded yet and then have the top-level module code switch then you’ll
> need to start getting really creative J  You’ll then presumably need to
> change these into using some trampoline to do the actual invocation.
>
>
>
> Hopefully that will get you started – I’m sure there’ll be lots of little
> issues so if you have questions feel free to ask.  If you get it all working
> we can figure out what exactly we need to make public and how we should
> expose that so you can do this as an extension – but I’m sure you’ll
> initially be using a bunch of internal APIs.
>
>
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Tristan Zajonc
> *Sent:* Sunday, May 30, 2010 6:59 PM
> *To:* Discussion of IronPython
> *Subject:* [IronPython] Implementing Greenlets API in IronPython
>
>
>
> Hi -
>
>
>
> The greenlets C-extension module (http://packages.python.org/greenlet/)
>  provides a very basic api that is used by eventlet and gevent to
> provide asynchronous programming constructs.  It would be nice to have a
> reasonably performant version of the greenlet API in IronPython.
>
>
>
> Before I start playing around with possibilities, is there an obvious
> approach to implement the greenlets API in IronPython?
>
>
>
> Thanks
>
> Tristan
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20100625/e0f58f4c/attachment.html>