Stackless, thread paradigm and C extensions
frederic.giacometti at arakne.com
Wed Nov 14 16:13:06 CET 2001
"Michael Hudson" <mwh at python.net> wrote in message
news:u8zdaodu4.fsf at python.net...
> "Frederic Giacometti" <frederic.giacometti at arakne.com> writes:
> > "Michael Hudson" <mwh at python.net> wrote in message
> > news:uu1w0e0vo.fsf at python.net...
> > > I must be missing something. Can you sketch how, say, socket.send()
> > > would be implemented? Would you spawn a new OS thread for every C API
> > > call? I'm afraid I don't understand you to this point...
> > As I mentioned, two thread pools would be maintained. To the extent
> > that send() is reentrant on the underlying OS, it would beexecuted
> > in one of the threads of the reentrant thread pool. This is the
> > 'thread pool' pattern; where threads are kept up (i.e. active) from
> > one call to the next. It's a standard pattern and concurrent
> > programming algorithm.
> So when Python code executes
> the interpreter would take a thread form the pool, and in effect say
> "here, run this function". Then the interpreter thread would go off
> and execute pending Python threads, and when the sock.recv call
> returned, it would add the thread that called it back to the set of
> pending interpreter threads? (I think this would be easier to discuss
> with paper and pencil...)
> What does this buy us again? It still makes writing C code that calls
> Python code a bit of a pain, doesn't it? Oh, maybe not. More
> thinking required...
A possible implementation would consist in using the method flag to mark C
python methods/functions to run outside the interpreter thread. This way,
the entire function runs in a separate thread.
The programmer does not have to explicitely manipulate OS threads at any
point; the continuation/OS thread correspondance would be taken care of
automatically by the threaded stackless implementation.
One would have parallel threads of execution through just continuations.
C extensions would run entirely either in a parallel thread, or in the
interpreter thread. There would be no more interpreter lock; and when a C
thread needs some service by the interpreter, it would just post a
continuation to the interpreter scheduler.
This will simplify programming when using Python callbacks, too; since the
currently active Python continuation would be the continuation active in the
More information about the Python-list