[Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)
Guido van Rossum
guido at python.org
Tue May 12 00:05:36 CEST 2015
On Mon, May 11, 2015 at 1:37 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 6 May 2015 at 16:46, Guido van Rossum <guido at python.org> wrote:
> > This is actually a great idea, and I encourage you to go forward with it.
> > The biggest piece missing from your inventory is probably Task, which is
> > needed to wrap a Future around a coroutine.
> OK, I've been doing some work on this. You're right, the asyncio
> framework makes Future a key component.
> But I'm not 100% sure why Future (and Task) have to be so fundamental.
> Ignoring cancellation (see below!) I can build pretty much all of a
> basic event loop, plus equivalents of the asyncio locks and queues
> modules, without needing the concept of a Future at all. The
> create_task function becomes simply a function to add a coroutine to
> the ready queue, in this context. I can't return a Task (because I
> haven't implemented the Task or Future classes) but I don't actually
> know what significant functionality is lost as a result - is there a
> reasonably accessible example of where using the return value from
> create_task is important anywhere?
In asyncio the Task object is used to wait for the result. Of course if all
you need is to wait for the result you don't need to call create_task() --
so in your situation it's uninteresting. But Task is needed for
cancellation and Future is needed so I/O completion can be implemented
using callback functions.
> A slightly more complicated issue is with the run_until_complete
> function, which takes a Future, and hence is fundamentally tied to the
> Future API. However, it seems to me that a "minimal" implementation
> could work by having a run_until_complete() that just took an
> awaitable (i.e., anything that you can yield from). Again, is there a
> specific reason that you ended up going with run_until_complete taking
> a Future rather than just a coroutine?
Actually it takes a Future *or* a coroutine. (The docs or the arg name may
be confusing.) In asyncio, pretty much everything that takes one takes the
> I think (but haven't confirmed
> yet by implementing it) that it should be possible to create a
> coroutine that acts like a Future, in the sense that you can tell it
> from outside (via send()) that it's completed and set its return
> value. But this is all theory, and if you have any practical
> experience that shows I'm going down a dead end, I'd be glad to know.
I don't know -- I never explored that.
> I'm not sure how useful this line of attack will be - if the API isn't
> compatible with asyncio.BaseEventLoop, it's not very useful in
> practice. On the other hand, if I can build a loop without Future or
> Task classes, it may indicate that those classes aren't quite as
> fundamental as asyncio makes them (which may allow some
> simplifications or generalisations).
Have you tried to implement waiting for I/O yet?
OTOH you may look at micropython's uasyncio -- IIRC it doesn't have Futures
and it definitely has I/O waiting.
> > I expect you'll also want to build cancellation into your "base async
> > framework"; and the primitives to wait for multiple awaitables. The next
> > step would be some mechanism to implement call_later()/call_at() (but
> > needs to be pluggable since for a "real" event loop it needs to be
> > implemented by the basic I/O selector).
> These are where I suspect I'll have the most trouble if I haven't got
> a solid understanding of the role of the Future and Task classes (or
> alternatively, how to avoid them :-)) So I'm holding off on worrying
> about them for now. But certainly they need to be covered. In
> particular, call_later/call_at are the only "generic" example of any
> form of wait that actually *waits*, rather than returning immediately.
> So as you say, implementing them will show how the basic mechanism can
> be extended with a "real" selector (whether for I/O, or GUI events, or
> > If you can get this working it would be great to include this in the
> > as a separate "asynclib" library. The original asyncio library would
> then be
> > a specific implementation (using a subclass of asynclib.EventLoop) that
> > I/O, subprocesses, and integrates with the selectors module (or with
> > on Windows).
> One thing I've not really considered in the above, is how a
> refactoring like this would work. Ignoring the "let's try to remove
> the Future class" approach above, my "basic event loop" is mostly just
> an alternative implementation of an event loop (or maybe an
> alternative policy - I'm not sure I understand the need for policies
A policy is mostly a wrapper around an event loop factory plus state that
records the current event loop.
> So it may simply be a case of ripping coroutines.py, futures.py,
> locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a
> new equivalent of events.py with my "minimal" loop in it. (So far,
> when I've tried to do that I get hit with some form of circular import
> problem - I've not worked out why yet, or how asyncio avoids the same
That sounds like a surface problem. Keep on debugging. :-)
> That in itself would probably be a useful refactoring, splitting out
> the IO aspects of asyncio from the event loop / async aspects.
Well, if you can.
> > I don't see any particular hurry to get this in before 3.5; the
> > of asyncio can be done later, in a backward compatible way. It would be a
> > good way to test the architecture of asyncio!
> Agreed. It's also not at all clear to me how the new async/await
> syntax would fit in with this, so that probably needs some time to
> settle down. For example, in Python 3.5 would run_until_complete take
> an awaitable rather than a Future?
It doesn't need to change -- it already calls async() on its argument
before doing anything (though with PEP 492 that function will be renamed to
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev