[Async-sig] async executors

[Nathaniel Smith]

On Jun 6, 2016 5:37 AM, "Christoph Groth" <christoph at grothesque.org> wrote:
>
[...]
> Async programming with executors seems to be a useful approach for
number-crunching problems where the main algorithm (the one launching the
futures) is quite complicated and possibly concurrent, but light enough to
run on a single core (the heavy lifting is done “inside” the futures).
>
> This use case is addressed by asyncio's run_in_executor that uses the
executors provided by the concurrent.futures package.  There is, however, a
lot of duplication of functionality between concurrent.futures and
asyncio.  (The main difference is that concurrent.futures relies on
threading and locks.)  If one is going to use concurrent.futures only from
coroutines that run under the control of the asyncio event loop, it should
be possible to replace the concurrent.futures executors by async versions
of them.

Sounds like an interesting project :-)

FYI, the latest versions of curio also have their own thread/process pool
implementations that replace concurrent.futures, I think for similar
reasons to the ones you allude to:

http://curio.readthedocs.io/en/latest/reference.html#performing-external-work

There are certainly reasons you might prefer to use asyncio instead of
curio, but if you're implementing such a thing on asyncio then curio's
implementation and semantics might be interesting for reference.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160606/799dce30/attachment.html>