[Python-ideas] non-blocking buffered I/O

[Guido van Rossum]

On Tue, Oct 30, 2012 at 9:11 AM, Kristján Valur Jónsson
<kristjan at ccpgames.com> wrote:
> By the way:  We found that acquiring the GIL by a random external thread in response to the IOCP to wake up tasklets was incredibly expensive.  I spent a lot of effort figuring out why that is and found no real answer.  The mechanism we now use is to let the external worker thread schedule a "pending call" which is serviced by the main thread at the earliest opportunity.  Also, the main thread is interrupted if it is doing a sleep.  This is much more efficient.

In which Python version? The GIL has been redesigned at least once.
Also the latency (not necessarily cost) to acquire the GIL varies by
the sys.setswitchinterval setting. (Actually the more responsive you
make it, the more it will cost you in overall performance.)

I do think that using the pending call mechanism is the right solution here.

-- 
--Guido van Rossum (python.org/~guido)