On 1 July 2015 at 14:25, Ron Adam <ron3200@gmail.com> wrote:
That's really very nice. Are there advantages to asyncio over the multiprocessing module?
I find it most useful to conceive of asyncio as an implementation of an "event driven programming" development paradigm. This means that after starting out with imperative procedural programming in Python, you can branch out into other more advanced complexity management models like object-oriented programming (class statements, dunder protocols, descriptors, metaclasses, type hinting), functional programming (comprehensions, generators, decorators, closures, functools, itertools), array oriented programming (memoryview, __matmul__, NumPy, SciPy), and event driven programming (asyncio, Twisted, Tornado). The stark difference between event driven programming and the first three alternate development models I noted is that you can readily implement the notions of "imperative shell, OO core", "imperative shell, functional core", and "imperative shell, array oriented core", where you expose a regular procedural API to other code, and implement it internally using whichever approach makes the most sense for your particular component. Even generators follow this same basic notion of having a clear "start of iteration" and "end of iteration". The concurrent execution model that most readily aligns with this "imperative shell" approach is concurrent.futures (https://docs.python.org/3/library/concurrent.futures.html) - it's designed to let you easily take particular input->output operations and dispatch them for execution in parallel in separate threads or processes. By contrast, event driven programming fundamentally changes your execution model from "I will accept inputs at the beginning of the program, and produce outputs at the end of the program" to "I will start waiting for events, responding to them as they arrive, until one of them indicates I should cease operation". "Waiting for an event" becomes a core development concept, as now indicated by the "await" keyword in PEP 492. The "async" keyword in that same PEP indicates that the marked construct may need to wait for events as part of its operation (async def, async for, async with), but exactly *where* those wait points are depends on the specific construct (await expressions in the function body for async def, protocol method invocations for async for and async with). For the design of asyncio (and similar frameworks) to make any sense at all, it's necessary to approach them with that "event driven programming" mindset - they seem entirely nonsensical when approached with an inputs -> outputs algorithmic mindset, but more logical when considered from a "receive request -> send other requests -> receive responses -> send response" perspective. For folks that primarily deal with algorithmic problems where inputs are converted to outputs, the event driven model addresses a kind of problem that *they don't have*, so it can seem entirely pointless. However, there really are significant categories of problems (such as network service development) where the event driven model is a genuinely superior design tool. Like array oriented programming (and even object-oriented and functional programming), the benefits can unfortunately be hard to explain to folks that haven't personally experienced the problems these tools address, so folks end up having to take it on faith that we're applying the "Complex is better than complicated" line from the Zen of Python when introducing new modelling techniques into the core language. Regards, Nick. P.S. It *is* possible to implement the "imperative shell, event driven core" model, but it requires a synchronous-to-asynchronous adapter like gevent, or an event-loop-per-thread model and extensive use of "run_until_complete()". It's much more complex than "just use concurrent.futures". P.P.S. Credit to Gary Bernhardt for the "imperative shell, <X> core" phrasing for low coupling component design where the external API design is distinct from the internal architectural design -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia