[Python-ideas] The async API of the future: yield-from

[Calvin Spealman]

On Tue, Oct 16, 2012 at 12:31 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> I'm sure doing concurrent I/O will require an event loop, unless you use threads under the hood...
>
> Polling I/O will require some sort of loop, yes, but I/O that triggers a callback at the OS level (such as ReadFileEx and WriteFileEx on Windows) doesn't need it.
>
> Of course, without an event loop you still need to wait on the future - for polling I/O you could return a subclassed future where waiting starts the polling loop if there isn't a better event loop available.

What if the event poll was just inside a task, not requiring any loop
in the scheduler, or even knowledge by the scheduler, in any way?

An extremely rudimentary version:

class Selector(object):
    def __init__(self):
        self.r = []
        self.w = []
        self.x = []
        self.futures = {}
    def add(self, t, fd, future):
        self.futures[fd] = future
        getattr(self, t).append(fd)
    def __iter__(self): return self
    def __next__(self):
        r = [fd for fd,future in self.r]
        w = [fd for fd,future in self.w]
        x = [fd for fd,future in self.x]
        r, w, x = select(r, w, x)
        for fd in chain(r, w, x):
            self.futures.pop(fd).set_result(fd)
        for fd in r: self.r.remove(fd)
        for fd in w: self.w.remove(fd)
        for fd in x: self.x.remove(fd)

This, if even to the scheduler, would handle polling completely
outside the scheduler, which makes it easier to mix and match event
loops you need to use in a single project.

I know I probably got details wrong.

> My view is that the most helpful thing to have in the standard is a way for any code to find and interact with an event loop - if we can discover a scheduler/context/loop/whatever and use its commands for "run this callable as soon as you can" and "run this callable when this condition is true" then we can have portable support for polling or event-based I/O (as well as being able to handle other thread-sensitive code such as in UIs).
>
> For optimal support, you'll need to have very close coupling between the scheduler and the asynchronous operations. This can be built on top of the portable support, but aiming for optimal support initially is a good way to make this API painful to use and more likely to be ignored.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy