[Chicago] threads and xmlrpc?
szybalski at gmail.com
Fri Jan 30 05:46:47 CET 2009
On Thu, Jan 29, 2009 at 9:02 AM, Tim Gebhardt <tim at gebhardtcomputing.com> wrote:
> If xmlrpc obeys the HTTP standard connection limit, you're limited to 2
> concurrent connections per host.
Could you point me to some docs on this. What I am comparing it to is
an apache server which can handle 100+ requests per second with no
problems. With Project Gutenberg we are talking about TB of data. With
Pypi we are talking about <kb per request and maybe about ~3kb per
second. So I think I should be able to achieve bandwidth of about
20kb/s minimum without anybody noticing any performance hits.
I've emailed pypi, but if there are other things to consider, or you
might know why such a low throughput on xmlrpc I would be interested
to know more.
> Hammering PyPi's server with more than 5 concurrent connections is also a
> little inconsiderate on your own part (please don't take this as an insult.
> Email is one-dimensional communication). I've mirrored Project Gutenberg
> before and they heavily rate limit people who do that: 1 connection at a
> time and a 2 second delay after every request.
> -Tim Gebhardt
> tim at gebhardtcomputing.com
> On Thu, Jan 29, 2009 at 8:51 AM, Lukasz Szybalski <szybalski at gmail.com>
>> I'm trying to download stuff from pypi via their xmlrpc. So what you
>> are saying is I should limit the number of threads to something less
>> then 5 and see what happens?
>> Chicago mailing list
>> Chicago at python.org
> Chicago mailing list
> Chicago at python.org
How to create python package?
Bazaar and Launchpad
More information about the Chicago