[Python-ideas] An alternate approach to async IO
sturla at molden.no
Wed Nov 28 01:25:10 CET 2012
Den 28. nov. 2012 kl. 01:01 skrev Trent Nelson <trent at snakebite.org>:
> Perhaps I was a little too eager to highlight the ability for these
> background IO threads to run without needing to acquire the GIL.
> A Python thread could indeed do the same job, however, you still
> wouldn't interact with it from Python code as if it were a normal
threading.Thread allows us to spawn a thread in a platform-independent way.
Here is a thread pool:
pool = [Thread(target=foobar) for i in range(n)]
These threads can release the GIL and call GetQueuedCompletionStatus. They can do any post-processing they want without the GIL. And they can return back to Python while holding the GIL.
Using a pool of non-Python threads inbetween would also take some of the scalability of IOCPs away. The thread that was the last to run is the first to be woken up on IO completion. That way the kernel wakes up a thread that is likely still in cache. But if you prevent the IOCP from waking up the thread that will call back to Python, then this scalability trick is of no value.
More information about the Python-ideas