[Tutor] demon / app I/F better way ?

[Lloyd Kvam]

Eric Raymond in "The Art of Unix Programming" claims that UNIX
developers tend to dislike threads.  He cites the Ousterhout article:
http://www.cs.utah.edu/~regehr/research/ouster.pdf
as an argument against threads.

Martelli in "Python in a Nutshell" provides detailed threading
recommendations.  These parallel Kent's recommendations of using the
Queue module and if carefully followed should give you a reliable
program.

I have to confess that I tend to avoid threads myself.

On Sun, 2004-11-14 at 14:13, Kent Johnson wrote:
> I don't think any of the complications Alan suggests are needed. The 
> Queue module takes care of the necessary locking semantics. It is 
> specifically intended for inter-thread communication. From the docs:
> 
> The Queue module implements a multi-producer, multi-consumer FIFO queue. 
> It is especially useful in threads programming when information must be 
> exchanged safely between multiple threads. The Queue class in this 
> module implements all the required locking semantics.
> 
> On a side note...
> I'm starting to think there is a cultural bias against threads in the 
> python community. I came to Python from Java where threads are commonly 
> used. For example the Jetty web server is a threaded web server that is 
> used in many production environments. I'm not sure there are any 
> industrial-strength Python web servers that use a threading model - they 
> are either based on asyncore or forking.
> 
> This is the second time I have suggested a threaded solution to a 
> problem posted on this list. Each time someone has fired back a reply 
> about the difficulties of threading. The replies surprise me because in 
> my experience, simple uses of threads can be, well, simple. Yes, there 
> are dangers, and certainly it can be hard to wrap your head around a 
> threaded app, but with a little care and the help of some library 
> classes you can get a lot done easily.
> 
> I'm not sure where I'm going with this except maybe, "Hey, guys, lighten 
> up a little! Threads can be your friends!"
> 
> Kent
> 
> Alan Gauld wrote:
> >>Having decided on threading the demon, and using a fifo, the queue
> >>module looks cool, I have one last dilema. One that quite often
> > 
> > causes
> > 
> >>me problems, that scope, namespace thing ;-)
> > 
> > 
> > To be honest the risks of deadlocks and race conditions using
> > multiple threads accessing a common queue are such that I would
> > tend to have the data collection thread writing to a separate
> > queue (out of a pool of queues) from the processing thread.
> > I'd then switch queues on each data change leaving the processor
> > reading a queue which has no risk of being overwritten by the
> > collector thread. I'd keep a status map of the queues such that
> > the collector marks them as full and the processor marks them
> > as empty. This way they avoid trampling on each other and if
> > the pool gets to the state of all full you can flag an error
> > and stop processing or dump it to a file before restarting
> > or whatever...
> > 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
-- 
Lloyd Kvam
Venix Corp