[Python-Dev] Thread questionlet

[Tim Peters]

If you ask Guido, the only reason to use threads is to do overlapped I/O.
And if you come up with a good counter-example, he'll find a way to *call*
it overlapped I/O, so there's no opposing him on this <wink>.

That's clearly a huge reason in practice to use threads, and that reason
requires using platform threads (to get true overlap).  Another huge reason
is to play nice with threaded libraries written in other languages, and
again that requires playing along with platform threads.

So what most thread users want is what Python gives them:  a thin wrapper
around native threads, complete with platform quirks.

The threading.py module adds *some* sanity to that, providing portable APIs
for some important synch primitives, and uniform thread shutdown semantics
(as Guido pointed out, when you use threading.py's thread wrappers, when the
main thread exits it waits for all (non-daemon) threads to quit).

What people seem to ask for most often now is a way for one thread to tell
another thread to stop.  In practice I've always done this by having each
thread poll a "time to stop?" variable from time to time.  That's not what
people want, though.  They want a way to *force* another thread to stop,
even if (e.g.) the target thread is stuck in a blocking read, or in the
middle of doing an extraordinarily expensive regexp search.  There simply
isn't a portable way to do that.  Java initially spec'ed a way to do it in
its thread model, but declared that deprecated after obtaining experience
with it:  not only did it prove impossible to implement in all cases, but
even when it worked, the thread that got killed had no way to leave global
invariants in a sane state (e.g., the thread may have had any number of
synch gimmicks-- like locks --in various states, and global invariants for
synch gimmicks can't tolerate a participant vanishing without both extreme
care and a way for a thread to declare itself unstoppable at times).

So that's a mess, but that's still what people want.  OTOH, they won't want
it for long if they get it (just as Java ran screaming from it).

I'm not sure the audience for cororoutine-style threads even overlaps.  You
could try to marry both models, by running coroutine-style threads in a pool
of OS threads.  Then, e.g., provided you knew a potentially blocking I/O
call when you saw one, you could farm it out to one of the real threads.  If
you can't do that, then I doubt the "real thread" crowd will have any
interest in uthreads (or will, but treat them as an entirely distinct
facility -- which, for their purposes, they would be).

For purposes of computational parallelism (more my background than
Guido's -- the idea that you might want to use a thread to avoid blocking on
I/O was novel to me <wink>), the global interpreter lock renders Python
useless except for prototyping, so there's not much point digging into the
hundreds of higher-level parallelism models that have been developed.

IOW, uthreads are their own universe.  I haven't used them, so don't know
what would be useful.  What do the current uthread users ask for?  That's
where I'd start.