[Python-ideas] An alternate approach to async IO

[Trent Nelson]

On Tue, Nov 27, 2012 at 04:25:10PM -0800, Sturla Molden wrote:
> 
> 
> Den 28. nov. 2012 kl. 01:01 skrev Trent Nelson <trent at snakebite.org>:
> 
> > 
> >    Perhaps I was a little too eager to highlight the ability for
> >    these background IO threads to run without needing to acquire the
> >    GIL.  A Python thread could indeed do the same job, however, you
> >    still wouldn't interact with it from Python code as if it were a
> >    normal threading.Thread.
> 
> threading.Thread allows us to spawn a thread in a platform-independent
> way.
> 
> Here is a thread pool: pool = [Thread(target=foobar) for i in
> range(n)]
> 
> These threads can release the GIL and call GetQueuedCompletionStatus.
> They can do any post-processing they want without the GIL. And they
> can return back to Python while holding the GIL.
> 
> Using a pool of non-Python threads inbetween would also take some of
> the scalability of IOCPs away. The thread that was the last to run is
> the first to be woken up on IO completion. That way the kernel wakes
> up a thread that is likely still in cache. But if you prevent the IOCP
> from waking up the thread that will call back to Python, then this
> scalability trick is of no value.

    I'm not sure where you're getting the idea that I'll be hampering
    the Windows IOCP optimizations that wake the last thread.  Nothing
    I've described would have any impact on that.

        Trent.