[Python-ideas] The async API of the future: yield-from

[Calvin Spealman]

On Tue, Oct 16, 2012 at 1:04 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> Well, how do you plan for that callback to execute Python code?
>
> IMO, that is the most important question in all of this discussion.
>
> With any I/O some waiting is required - there must be a point where the application is not doing anything other than waiting for the I/O to complete, regardless of whether a loop is used or not. (Ideally the I/O is already complete by the time we start waiting.) The callbacks in the particular examples require a thread to be in an alertable wait state, which is basically equivalent to select(), though a little less discriminatory (as in, ANY I/O callback can interrupt an alertable wait).
>
> In my view, these callbacks should be 'leaving a message' for the main program to run a particular function when it next has a chance. Like an interrupt handler, the aim is to do the minimum amount of work and then get out of the way.

I like this model as well.

However, I recognize some problems with it. If we don't kick whatever
handles the callback and result immediately, we are essentially
re-introducing pre-emptive scheduling. If TaskA is waiting on the
result of TaskB, and when TaskB finishes we say "OK, but we need to go
let TaskC do something before TaskA is given that result" then we
leave room for C to break things, modify state, and generally act in a
less-than-determinable way.

I really *like* this model better, I just don't know the best way to
reconcile this problem.

> Having a context (or event loop, message loop or whatever you want to call it) as I described in my last email lets us do the minimum amount of work. I posted our implementation of such a context earlier and Dino posted an example/recipe for using the concept with an existing event loop (Tcl).
>
> So while I said we don't _need_ an event loop, that relies on the asynchronous operations being on a separate thread or otherwise not requiring the current thread to pay any attention to them, AND assumes that the continuations are agile and can be run on any thread (or in any process, or whatever granularity you are working at). I believe some way of getting code running back where it started from is essential, and this is most easily done with a loop.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy