-----Original Message----- From: Python-ideas [mailto:python-ideas- email@example.com] On Behalf Of Greg Ewing Sent: 30. október 2012 05:10 To: firstname.lastname@example.org Subject: Re: [Python-ideas] non-blocking buffered I/O
From my point of view, IOCP fits in very well provided the callbacks (which will run in the IOCP thread pool) are only used to unblock tasks.
Is it really necessary to have a separate thread just to handle unblocking tasks? That thread will have very little to do, so it could just as well run the tasks too, couldn't it?
StacklessIO (which is an IOCP implementation for stackless) uses callbacks on an arbitrary thread (in practice a worker thread from window's own threadpool that it keeps for such things) to unblock tasklets. You don't want to do any significant work on such a thread because it is used for other stuff by the system.
By the way: We found that acquiring the GIL by a random external thread in response to the IOCP to wake up tasklets was incredibly expensive. I spent a lot of effort figuring out why that is and found no real answer. The mechanism we now use is to let the external worker thread schedule a "pending call" which is serviced by the main thread at the earliest opportunity. Also, the main thread is interrupted if it is doing a sleep. This is much more efficient.