[Python-Dev] Twisted Isn't Specific (was Re: Trial balloon: microthreads library in stdlib)

["Martin v. Löwis"]

Larry Hastings schrieb:
> Bob Ippolito wrote:
>> There is no single PerfectReactor. There are several use cases where
>> you need to wait on >1 different event systems, which guarantees at
>> least two OS threads (and two event loops). In general it's nice to
>> have a single Python event loop ("the reactor") to act on said threads
>> (e.g. something just sitting on a mutex waiting for messages) but
>> waiting for IO to occur should *probably* happen on one or more
>> ancillary threads -- one per event system (e.g. select, GTK,
>> WaitForMultipleEvents, etc.)
> Why couldn't PerfectReactor be a reactor for other reactors?  A sort of 
> concentrator for these multiple event systems and multiple threads.

Because you can't write a single function that blocks (idle) until
an event in one of the systems occurs. None of the system is truly
asynchronous. All APIs, eventually, rely on synchronous, blocking,
operating system calls. In Unix, the call to make is typically
select or poll, and you know it is safe to make because you have
all even sources as file descriptors and you know nothing else
needs to be done before one of the events occurs.

Now, for these generalized event loops, it may not be possible anymore
to combine all event sources into a single blocking call. If you have
timeouts, you have to find out what the minimum timeout is (to make
it the timeout for the blocking call, assuming that supports a timeout).
If you have idle tasks, you have to invoke all idle tasks of all
reactors (which you may not be able to get because there is no API
to fetch them).

Even more difficult: if these use different OS event mechanisms
(on systems that have multiple such systems). E.g. try combinging
a Win32 GetMessage/PostMessage loop with a WaitForMultipleObjects
loop with a WinSock select call; then add the Win32 RPC libraries
to that (i.e. try to determine the named pipe handles you have to
pass to WaitForMultipleObjects).


> You ask to listen to sockets, so it instantiates a singleton 
> PerfectReactor which instantiates a select() reactor and listens to it 
> directly in a single-threaded manner.  If you then ask to listen to 
> Win32 messages, the PerfectReactor instantiates a GetMessage() reactor.  
> Then, realizing it has two "event systems", it spawns a thread for each 
> child reactor with a listener that serializes the incoming events into 
> the PerfectReactor's queue.


Ah, threads :-( It turns out that you need to invoke GetMessage in the
context of the thread in which the window was created. In a different
thread, you won't get any messages. (This is indeed one of the biggest
limitations of USER32, which was only overcome in Vista when the 
"native" GUI programming model is based on DirectX - the problem is
then of course solved only for applications that use the DirectX UI
API).

> Bingo, your application doesn't need to be 
> written thread-safe, PerfectReactor is platform-agnostic, and you don't 
> have to know in advance all the event types you might ever listen to.
> 
> Sorry if this is a dumb question (or if I'm mangling the terminology),

Integrating with threads might be a solution in some cases, and
a problem in others. You can't assume it is a universal solution.

Regards,
Martin