[Async-sig] async executors
Christoph Groth
christoph at grothesque.org
Mon Jun 13 12:22:38 EDT 2016
Łukasz Langa wrote:
>> As far as I can see the original
>> concurrent.futures.ProcessPoolExecutor has no advantage when
>> used in asyncio-based programs except when some coroutine
>> blocks for long enough for the call queue to become empty.
>> (But hey, async programming is about non-blocking coroutines!)
>
> That’s incorrect. The ProcessPoolExecutor is a way for you to
> run CPU-intensive tasks without affecting the responsiveness of
> the main event loop in your program. As long as you keep your
> arguments small (pickling/unpickling big data structures is
> pretty detrimental to performance here) and the duration of the
> task is non-trivial, the child processes enable you to use more
> CPU cores, and since they have their own GILs, this doesn't
> affect your loop at all, unlike using a thread pool.
I was comparing the (possible) advantages of the traditional
concurrent.futures.ProcessPoolExecutor with the new
aexecutor.ProcessPoolExecutor. Both use processes for the
workers, so the GIL is not a problem for either.
Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160613/7cbf8c11/attachment.sig>
More information about the Async-sig
mailing list