Re: [Async-sig] async executors
On Jun 6, 2016 5:37 AM, "Christoph Groth" <christoph@grothesque.org> wrote:
Async programming with executors seems to be a useful approach for number-crunching problems where the main algorithm (the one launching the futures) is quite complicated and possibly concurrent, but light enough to run on a single core (the heavy lifting is done “inside” the futures).
This use case is addressed by asyncio's run_in_executor that uses the executors provided by the concurrent.futures package. There is, however, a lot of duplication of functionality between concurrent.futures and asyncio. (The main difference is that concurrent.futures relies on
[...] threading and locks.) If one is going to use concurrent.futures only from coroutines that run under the control of the asyncio event loop, it should be possible to replace the concurrent.futures executors by async versions of them. Sounds like an interesting project :-) FYI, the latest versions of curio also have their own thread/process pool implementations that replace concurrent.futures, I think for similar reasons to the ones you allude to: http://curio.readthedocs.io/en/latest/reference.html#performing-external-wor... There are certainly reasons you might prefer to use asyncio instead of curio, but if you're implementing such a thing on asyncio then curio's implementation and semantics might be interesting for reference. -n
participants (1)
-
Nathaniel Smith