unexpected performance from threads and network client benchmarking

Project2501 project2501 at project2501.cor
Mon Apr 12 19:24:26 EDT 2004


i'm a little puzzled by the received wisdom of python threads.

the wisdom says that you will not see any performance increase from the
use of threads on the client side trying to it a server (high request rate).

this is explained in terms of the GIL..

but i do see a difference... on python 2.3 on freebsd4.9 and linux 2.6

here is  the pseudocode:

loop:
	create a thread which will query a server and wait for response
	sleep(for interval)

that's it. the interval in the sleep does change in steps over time. the
threads themselves are passes the current time.time() and they themslves
take the difference between this time and the time.time at which the
response was received. (with a timeout). threads are created up to a given
maximum (finished threads decrement a global counter).

ok - now the interesting bit.

with a maximum of one client thread at a time, the graphs responce time
against the time.time() (at thread creation) are largely flat but with
random jumps... 

as more client threads are allowed, the graphs become smoother with
generally higher peak responce times ... and show:

  * initial flat response time - good, server is able to cope nicely
  * increase (almost linear) as the rate increases - server is queueing
    requests? or is taking longer to schedule more requests. whatever.
  * again, a flat rate at high request rates. possible client limit.
    perhaps reaching time/scheduler resolution of the python runtime?

there is higher responce time from the server as more client threads
attack it - why? if they can only execute one at a time? isi t because the
switching between them is quicker than the creation of a new attack thread
when there a max of 1 (sequential client)?

the target server is single threaded (radiator radius server, is well
known as single threaded and sequential)- it handles requests
sequentially! the client and server machines are also uni-processor.

Does this mean that there is a benefit to python threads? i'm puzzled! why
the benefit in responce times?



More information about the Python-list mailing list