threads or queue for this task
James J. Besemer
jb at cascade-sys.com
Mon Sep 16 14:50:16 EDT 2002
robin at execulink.com wrote:
>I may sometimes be clever, and can no doubt cobble together code from
>reading the docs (which I have been doing so far), but what I don't
>know are some of the trade-offs of using certain constructs, or the
>optimal way to do things. That is much more difficult to glean without
>actual code examples.
>For example, I really don't know what "proper design" I should use in
>this case. Any sample code?
The following illustrates my understanding of what you're trying to do.
# three queues connect four threads
CmdQ1 = Queue.Queue()
CmdQ2 = Queue.Queue()
CmdQ3 = Queue.Queue()
# Reader thread reads commands from stdin and
# dispatches to appropriate thread via queue
line = stdin.readline()
if not line:
if line == '1':
elif line == '2':
elif line == '3':
continue # or signal error
# Command threads process commands as they arrive via queue
cmd = CmdQ1.get()
process1( cmd )
cmd = CmdQ2.get()
process2( cmd )
cmd = CmdQ3.get()
process3( cmd )
thread.start_new_thread( Cmd1, ())
thread.start_new_thread( Cmd2, ())
thread.start_new_thread( Cmd3, ())
# reader thread runs in mainline
Frankly, it doesn't get any simpler than this.
With threads as nodes and Queues as pipes, you can create practically
any data flow topology you wish.
>The interface to Queue is deceptively simple. Are there constructs to
>avoid or performance trade-offs to different methods? Again, any
>explicit code (or at least explicit help as to which methods are
>preferable) would be appreciated.
It's not deceptively simple. It's merely simple. Sometimes a Queue is
simply a Queue.
>I am also put off by the description of the Queue Object, particularly
>methods like empty() which says: "Return 1 if the queue is empty, 0
>otherwise. Because of multithreading semantics, this is not reliable."
The answer is accurate when it is returned to you but due to the nature
of multithreading semantics, the value also is subject to change without
notice. So if you read it and see a 0 or a 1, this is a true historical
fact but you can't rely on the value being true at arbitrary points in
the future so you can't really base any decisions on it.
This is a function that was trivial and natural for the class designer
to implement. It probably IS used internal to the class in order to
decide whether or not to suspend or resume a thread. But is of little
practical value to the end user. Fageddabouit.
>How can one code if you don't
>reliably know when a Queue is empty or full?
Generally, Queues never get "full" and you generally don't care if
The import thing about Queues is that the reader thread will
automatically be suspended when reading from an empty queue and
furthermore it will automatically wake up when the queue becomes non empty.
All you need from Queues is to create them and to get and put data.
>What this boils down to is: I'm sure I can do the job with the great
>help I've already received, but do not have the confidence to know if
>I'm doing the job the *right* way.
If your code does what you want and doesn't take an unreasonable amount
of time to run on your computer then that's 80% of the "*right*" way.
I'll defer to others to argue the remaining 20%.
There really aren't as many alternatives here as you seem to suspect.
You probably should look for a book on concurrent sequential processes.
Most of the paradigms and pitfalls are pretty standard and would be
spelled out in the book. Unfortunately, I am at a loss to make a
specific recommendation. Perhaps somebody else has an idea.
James J. Besemer 503-280-0838 voice
2727 NE Skidmore St. 503-280-0375 fax
Portland, Oregon 97211-6557 mailto:jb at cascade-sys.com
More information about the Python-list