PEP 3156: Clarifying the different components of the event loop API
PEP 3156 currently lists *29* proposed methods for the event loop API. These methods serve quite different purposes and I think a bit more structure in the overall API could help clarify that. First proposal: clearly split the abstract EventLoop API from concrete DescriptorEventLoop and IOCPEventLoop subclasses. The main benefit here is to help clarify that: 1. the additional methods defined on DescriptorEventLoop and IOCPEventLoop are not available on all event loop implementations, so any code using them is necessarily event loop specific 2. the goal of the transport abstraction is to mask the differences between these low level platform specific APIs 3. other event loops are free to use a completely different API between their low level transports and the event loop Second proposal: better separate the "event loop management", "event monitoring" and "do things" methods I don't have a clear idea of how to do this yet (beyond restructuring the documentation of the event loop API in the PEP), but I can at least describe the split I see (along with a few name changes that may be worth considering). Event loop management: - run_once() - run() # Perhaps "run_until_idle()"? - run_forever() # Perhaps "run_until_stop()"? - run_until_complete() - stop() - close() - set_default_executor() Event monitoring: - add_signal_handler() - remove_signal_handler() - start_serving() # (The "stop serving" API is TBD in the PEP) Do things (fire and forget): - call_soon() - call_soon_threadsafe() - call_later() - call_repeatedly() Do things (and get the result with "yield from"): - wrap_future() # Perhaps "wrap_executor_future"? - run_in_executor() - getaddrinfo() - getnameinfo() Low level transport creation: - create_connection() - create_pipe() # Once it exists in the PEP Cheers, Nick. P.S. Off-topic for the thread, but I think the existence of run_once vs run (or run_until_idle) validates the decision to stick with only running one generation of ready callbacks per iteration. I forgot about it when we were discussing that question. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
(I'm out of time to respond at length, but I think you have a good point here and I expect I will heed it. It may be a while before I have time for another sprint with the PEP and Tulip though.) On Sat, Jan 19, 2013 at 6:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On Sun, Jan 20, 2013 at 12:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Somewhere early in the PEP, there may need to be a concise description of the two APIs for waiting for an asynchronous Future: 1. "f.add_done_callback()" 2. "yield from f" in a coroutine (resumes the coroutine when the future completes, with either the result or exception as appropriate) At the moment, these are buried in amongst much larger APIs, yet they're key to understanding the way everything above the core event loop layer interacts. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jan 20, 2013 10:46 PM, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
That's actually the "start_serving" method up in the event monitoring section. While it does end up creating transports, the overall flow is rather different from the client side one. Cheers, Nick. -- Sent from my phone, thus the relative brevity :)
On Sun, 20 Jan 2013 23:10:56 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah, right. Well, in any case, the API is much too limited. It doesn't support SSL, and it doesn't support UDP. “TBD: Support SSL? I don't even know how to do that synchronously, and I suppose it needs a certificate.” See http://docs.python.org/dev/library/ssl.html#server-side-operation (and the non-blocking handshake part also applies) Regards Antoine.
On Sat, Jan 19, 2013 at 6:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I like the idea of splitting up the big interface, but could you clarify what would go into such subclasses? I.e. isn't the current EventLoop interface supposed to represent an interface all event loops will adhere to? And sorry if this was discussed before and I'm missing the context, but what kinds of EventLoop implementations are we expecting to see eventually? Is it only a matter of implementing the API per platform (akin to the current tulip.unix_events.UnixEventLoop) or a broader expectation of frameworks like Twisted to plug into the API by providing their own implementation (PEP 3156 mentions this somewhere).
Second proposal: better separate the "event loop management", "event monitoring" and "do things" methods
<snip>
+1 These certainly look somewhat out of place in the generic EventLoop API, but concretely - how do you propose to structure the split? Eli
The concrete event loop methods are already separated in the PEP - they're just flagged as optional methods rather than distinct subclasses. The rest I think actually do belong on the event loop, hence the suggestion to start just by rearranging them into those categories, without making the class hierarchy any more complicated. -- Sent from my phone, thus the relative brevity :) On Jan 21, 2013 12:18 AM, "Eli Bendersky" <eliben@gmail.com> wrote:
(I'm out of time to respond at length, but I think you have a good point here and I expect I will heed it. It may be a while before I have time for another sprint with the PEP and Tulip though.) On Sat, Jan 19, 2013 at 6:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On Sun, Jan 20, 2013 at 12:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Somewhere early in the PEP, there may need to be a concise description of the two APIs for waiting for an asynchronous Future: 1. "f.add_done_callback()" 2. "yield from f" in a coroutine (resumes the coroutine when the future completes, with either the result or exception as appropriate) At the moment, these are buried in amongst much larger APIs, yet they're key to understanding the way everything above the core event loop layer interacts. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jan 20, 2013 10:46 PM, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
That's actually the "start_serving" method up in the event monitoring section. While it does end up creating transports, the overall flow is rather different from the client side one. Cheers, Nick. -- Sent from my phone, thus the relative brevity :)
On Sun, 20 Jan 2013 23:10:56 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Ah, right. Well, in any case, the API is much too limited. It doesn't support SSL, and it doesn't support UDP. “TBD: Support SSL? I don't even know how to do that synchronously, and I suppose it needs a certificate.” See http://docs.python.org/dev/library/ssl.html#server-side-operation (and the non-blocking handshake part also applies) Regards Antoine.
On Sat, Jan 19, 2013 at 6:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I like the idea of splitting up the big interface, but could you clarify what would go into such subclasses? I.e. isn't the current EventLoop interface supposed to represent an interface all event loops will adhere to? And sorry if this was discussed before and I'm missing the context, but what kinds of EventLoop implementations are we expecting to see eventually? Is it only a matter of implementing the API per platform (akin to the current tulip.unix_events.UnixEventLoop) or a broader expectation of frameworks like Twisted to plug into the API by providing their own implementation (PEP 3156 mentions this somewhere).
Second proposal: better separate the "event loop management", "event monitoring" and "do things" methods
<snip>
+1 These certainly look somewhat out of place in the generic EventLoop API, but concretely - how do you propose to structure the split? Eli
The concrete event loop methods are already separated in the PEP - they're just flagged as optional methods rather than distinct subclasses. The rest I think actually do belong on the event loop, hence the suggestion to start just by rearranging them into those categories, without making the class hierarchy any more complicated. -- Sent from my phone, thus the relative brevity :) On Jan 21, 2013 12:18 AM, "Eli Bendersky" <eliben@gmail.com> wrote:
participants (4)
-
Antoine Pitrou -
Eli Bendersky -
Guido van Rossum -
Nick Coghlan