[Python-ideas] Async API

[Steve Dower]

Yury Selivanov wrote:
> It would be great if we can address those problems with the new
> async API.  I.e. we can use threadpools where necessary, but make
> the public API look fancy and yield-from-able.  Same approach that
> Joyent uses in their libuv.  And when OSes gain more advanced and
> wide non-blocking support we can decrease use of threads.

This certainly seems to be the plan, though I expect the details will be determined as libraries are updated to support the async API. As long as we ensure that the API itself can support event loops and operations using threads and other OS primitives, then we don't need to specify each and every one at this stage.

My design (which I'm writing up now) puts most of the responsibility on the active scheduler, which should make it much easier to have different default schedulers for each platform (and maybe specialised ones that are optimised for more limited situations) while the operations themselves can be built out of existing Python functions. I'll post more details soon, but it basically allows schedulers to optionally support some operations (such as select() or Condition.wait()) 'natively', with the operation only having to implement a fallback (presumably on a thread pool).

Cheers,
Steve