At 09:24 PM 8/1/2010 -0700, Guido van Rossum wrote:

I don't understand all the details and corner cases (e.g. the concatenation of stacks

It's just to ensure that you never have From's iterating over other From's, vs. just iterating whatever's at the top of the stack.

which seems to have to do with the special-casing of From objects in __new__)

It isn't connected, actually except that it's another place where I'm keeping From's flat, instead of nested. (I hear that flat is better. ;-) )

I am curious whether, if you need a trampoline for async I/O anyway, there would be a swaying argument for integrating this functionality into the general trampoline (as in the PEP 342 example),

Originally, that was why I wasn't very enthusiastic about PEP 380; it didn't seem to me to be adding any new value over what you could do with existing, widely-used libraries. (Twisted's had its own *and* multiple third-party From-ish libraries supporting it for many years now.) After I wrote From(), however (which was originally intended to show why I thought 380 was unnecessary), I realized that having One Obvious Way to implement generator-based pseudothreads independent of an event loop, is actually useful precisely *because* it separates the pseudothreadedness from what you're using the pseudothreadedness for. Essentially, the PEP 380-ish bit is the hardest part of writing an actual pseudothread implementation; connecting that implementation to an I/O framework is actually the relatively simple part. You just write code that steps into the generator, and uses the yielded object to initiate an I/O operation and register a callback. (If you're using Twisted or something else that has "promise"-like deferred results, it's *really* easy, because you only have a couple of types of yielded objects to deal with, and a uniform callback signature.) Indeed, if you're using an existing async I/O framework, you don't even really *have* a "trampoline" as such -- you just have a bit of code that registers callbacks to itself, and the app's main event loop just calls back to that wrapper when the I/O is done. In effect, an I/O framework integration would just give you a single API like, "run(From(geniter))", which then performs one iteration, and then registers whatever callback it's told to by the yield, and the callback it registers would actually be a reinvocation of run() on the same From instance when the I/O is ready, but with a value to pass back into the send(), or an error to throw(). So, the I/O framework's event loop is half of the "trampoline", and the wrapper that sends or throws, then registers an I/O callback, is the other half. Something like: def run(coroutine, value=None, exc_info=()): if exc_info: action = coroutine.throw(*exc_info) else: action = coroutine.send(value) action.registerCallback(partial(run, coroutine)) Where 'action' is some I/O command object, and registerCallback() will call its argument back with a value or exc_info, after the I/O is done. Of course, a real framework integration might actually dispatch on type here rather than using special command objects like this, and there might be more glue code to deal with exceptions, but really, the heart of the thing is just going to look like that. (I just wrote it that way to show the basic structure.) Really, it's just a few functions, maybe a utility routine or two, and maybe a big if-then or dictionary dispatch on types if you just want to be able to 'yield' existing I/O objects provided by the frameworks. IOW, it's a *lot* simpler than actually rolling your own I/O or GUI framework like Twisted or Eventlet or wxPython or tk or some other such thing.

But it seems a bit of a waste to have two different trampolines, especially since the trampoline itself is so hard to understand (speaking for myself here :-). ISTM that the single combined trampoline is easier to understand than the From class.

Well, the PEP 342 example was made to look simple, because it doesn't have to actually DO anything (like I/O!) To work for real, it'd need some pluggability, and some things to help it interoperate with different GUI and I/O frameworks and event loops. (Using your own event loop "for real" isn't very useful in a lot of non-trivial applications.) Heck, after writing From(), it gave me an idea that I could just write a trampoline that *could* integrate with other event loops, with an idea to have it be a general-purpose companion to From. But, after several wasted hours, I realized that yes, it *could* be written (I still have the draft), but it was mostly just something that would save a little boilerplate in bolting From()'s onto an existing async I/O framework, and not really anything to write home about. So, I guess what I'm saying is, the benefit of separating the trampoline from control flow, is that people can then use them with their favorite event loop or framework, instead of the stdlib trying to compete with the experts on a slower release cycle. There's additional benefit here in that 1) getting pseudothread implementations correct can be difficult, but once done, they're extremely stable, and 2) having a blessed syntax for identifying pseudothreaded calls and returns is a boon to competition in I/O frameworks. So, instead of everybody having their own versions of From (and I've written more than a couple in my time), there's One Obvious Way To Do It. All that will differ between I/O libraries are the actual API calls for I/O and scheduling and starting up pseudothreads, so there won't be as big of a switching barrier between frameworks.