please critique my thread code
pfreixes at gmail.com
Thu Jun 19 10:14:15 CEST 2008
> MRAB wrote:
> Erm, shurely the bottleneck will be bandwidth not processor/memory?* If it
> isn't then - yes, you run the risk of actually DOSing their servers!
> Your mac will run thousands of threads comfortably but your router may not
> handle the thousands of TCP/IP connections you throw at it very well,
> especially if it is a domestic model, and sure as hell sourceforge aren't
> going to want more than a handfull of concurrent connections from you.
> Typical sourceforge page ~ 30K
> Project pages to read = 240000
> = ~6.8 Gigabytes
The best path for avoid congestion network problem and probably "DOS attack"
it's use a equilibrate number of parallel connection with a number of
threads. One router can handle hundred connections and Mac thread
implementation may be also - with a lot of memory or stack size profiling.
If you run hundred threads you will be fight with OS, Webserver, Router and
network starvation problems.
I believe use a 64 number of threads it's ok, but probably all 64 thread
connections will be performance for the same sourceforge web server -
balancing algorithm - and may be that this week up suspicion to the admin.
May be, you can use httplib  for do a persistent connection and reuse
same connection for same thread.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-list