[Tutor] Threading...

[A.T.Hofkamp]

Kent Johnson wrote:
> On Mon, Apr 27, 2009 at 2:46 AM, A.T.Hofkamp <a.t.hofkamp at tue.nl> wrote:
>> you are making
>> a IO-bound app, so threading doesn't buy you anything in performance and
>> gives a big headache in data protection
> 
> Please explain. Presumably the single-threaded app is not IO-bound.
> Adding threads can push it to the point of being IO-bound. The

Euh, what?
Suppose I have a CPU-intensive task. Now by adding threads (ie more 
CPU-intensive tasks running at the same time) I can make it an IO-bound 
application?

Wow.
Let's call seti at home and climateprediction.net .



As far as I know, doing DB queries is an IO-bound process, since you must make 
a connection to the remote DB server, and send data back and forth to the 
server over the network (which is ssssllllooooowwww compared to a calculation).
With threading, you will have more network connections that you are waiting 
for. W.r.t. performance you don't gain anything, since the CPU is already 
mostly idle.

 > requirements for communicating between threads in this app are modest
 > and not likely to cause a big headache.

With respect to the data problem, I don't know what the OP wants, but maybe he 
wants to run different tests at different times (ie first 1 minute test A, 
then 1 minute test B etc) or different mixes of queries. He may also want to 
collect performance results (avg x queries/second).
Such data is shared between the threads, so it must be protected to prevent it 
from getting corrupted.
Maybe it is modest in requirements, but you must do it none the less.

Also note that the protection itself may also become a bottle neck in the 
program (you must constantly claim and release locks which adds overhead even 
when there is no other thread waiting).



With Twisted, you can have at least the same number of IO connections as with 
threading, while you don't get the data protection problems and overhead for 
the simple reason that it is still a single-threaded program.



Maybe threading is the correct solution here. However the OP also asked for 
"...or is there another way to go about it?".
Twisted is definitely another way to go about it, and nobody mentioned it in 
previous posts, so I thought it correct to mention its existence here.


Sincerely,
Albert