[Tutor] demon / app I/F better way ?

[Kent Johnson]

I don't think any of the complications Alan suggests are needed. The 
Queue module takes care of the necessary locking semantics. It is 
specifically intended for inter-thread communication. From the docs:

The Queue module implements a multi-producer, multi-consumer FIFO queue. 
It is especially useful in threads programming when information must be 
exchanged safely between multiple threads. The Queue class in this 
module implements all the required locking semantics.

On a side note...
I'm starting to think there is a cultural bias against threads in the 
python community. I came to Python from Java where threads are commonly 
used. For example the Jetty web server is a threaded web server that is 
used in many production environments. I'm not sure there are any 
industrial-strength Python web servers that use a threading model - they 
are either based on asyncore or forking.

This is the second time I have suggested a threaded solution to a 
problem posted on this list. Each time someone has fired back a reply 
about the difficulties of threading. The replies surprise me because in 
my experience, simple uses of threads can be, well, simple. Yes, there 
are dangers, and certainly it can be hard to wrap your head around a 
threaded app, but with a little care and the help of some library 
classes you can get a lot done easily.

I'm not sure where I'm going with this except maybe, "Hey, guys, lighten 
up a little! Threads can be your friends!"

Kent

Alan Gauld wrote:
>>Having decided on threading the demon, and using a fifo, the queue
>>module looks cool, I have one last dilema. One that quite often
> 
> causes
> 
>>me problems, that scope, namespace thing ;-)
> 
> 
> To be honest the risks of deadlocks and race conditions using
> multiple threads accessing a common queue are such that I would
> tend to have the data collection thread writing to a separate
> queue (out of a pool of queues) from the processing thread.
> I'd then switch queues on each data change leaving the processor
> reading a queue which has no risk of being overwritten by the
> collector thread. I'd keep a status map of the queues such that
> the collector marks them as full and the processor marks them
> as empty. This way they avoid trampling on each other and if
> the pool gets to the state of all full you can flag an error
> and stop processing or dump it to a file before restarting
> or whatever...
>