asyncio, coroutines, etc. and simultaneous execution
4kir4.1i at gmail.com
Mon Aug 24 01:28:38 CEST 2015
Charles Hixson <charleshixsn at earthlink.net> writes:
> If I understand correctly asyncio, coroutines, etc. (and, of course,
> Threads) are not simultaneously executed, and that if one wants that
> one must still use multiprocessing. But I'm not sure. The note is
> still there at the start of threading, so I'm pretty sure about that
Due to GIL (used by CPython, Pypy, not used by Jython, IronPython,
pypy-stm) only one thread executes Python bytecode at a time. GIL can be
released on I/O or in a C extension such as numpy, lxml, regex. It is
true for any Python module or concept you use.
Unrelated: use concurrent and parallel execution instead of
Parallelism might make a program faster (it implies that hardware
Concurrency is a way to structure the code. The same concurrent program
can run in parallel and without parallelism.
Recommended: "Concurrency is not Parallelism
(it's better!)" talk by Rob Pike's talk
> The requirement that coroutines always be awaited seems to confirm
> this, but doesn't really say so explicitly. And the concurrent.futures
> can clearly be either, depending on your choices, but the chart in
> 22.214.171.124.3 Example: Chain coroutines is of a kind that I am more
> familiar with in the context of multiprocessing. (E.g., the only gap
> in the chart, which extends across all headings is when a result is
> being waited for during a sleep.) For threaded execution I would
> expect there to be a gap whenever processing is shifted from one
> column to another.
> If someone has authority to edit the documentation a comment like:
Anybody can suggest a patch
> If you want your application to make better use of the computational
> resources of multi-core machines, you are advised to use
> or concurrent.futures.ProcessPoolExecutor
> <https://docs.python.org/3.5/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor>. However,
> threading is still an appropriate model if you want to run multiple
> I/O-bound tasks simultaneously.
> (to quote from the threading documentation) would be helpful at or
> very near the top of each of the appropriate modules. It would also
> be useful if the modules that were definitely intended to result in
> simultaneous execution, when feasible, were so marked quite near the
It is not necessary that multiprocessing will make your code faster on a
multi-core machine. It may make it slower depending on the task.
Performance optimization is a vast topic. Short remarks such as you've
suggested are likely misleading.
> OTOH, I may be mistaken about coroutines. I haven't been able to tell.
Many cooperative multitasking implementations use a *single*
thread. There is a way to offload blocking code e.g.,
loop.run_in_executor() in asyncio.
More information about the Python-list