[Chicago] threads and xmlrpc?

[Lukasz Szybalski]

On Thu, Jan 29, 2009 at 9:02 AM, Tim Gebhardt <tim at gebhardtcomputing.com> wrote:
> If xmlrpc obeys the HTTP standard connection limit, you're limited to 2
> concurrent connections per host.

Could you point me to some docs on this. What I am comparing it to is
an apache  server which can handle 100+ requests per second with no
problems. With Project Gutenberg we are talking about TB of data. With
Pypi we are talking about <kb per request and maybe about ~3kb per
second. So I think I should be able to achieve bandwidth of about
20kb/s minimum without anybody noticing any performance hits.

I've emailed pypi, but if there are other things to consider, or you
might know why such a low throughput on xmlrpc I would be interested
to know more.

Thanks,
Lucas



> Hammering PyPi's server with more than 5 concurrent connections is also a
> little inconsiderate on your own part (please don't take this as an insult.
> Email is one-dimensional communication).  I've mirrored Project Gutenberg
> before and they heavily rate limit people who do that: 1 connection at a
> time and a 2 second delay after every request.
> -Tim Gebhardt
> tim at gebhardtcomputing.com
>
> On Thu, Jan 29, 2009 at 8:51 AM, Lukasz Szybalski <szybalski at gmail.com>
> wrote:
>>
>> I'm trying to download stuff from pypi via their xmlrpc. So what you
>> are saying is I should limit the number of threads to something less
>> then 5 and see what happens?
>>
>> Thanks,
>> Lucas
>> _______________________________________________
>> Chicago mailing list
>> Chicago at python.org
>> http://mail.python.org/mailman/listinfo/chicago
>
>
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>
>



-- 
How to create python package?
http://lucasmanual.com/mywiki/PythonPaste
Bazaar and Launchpad
http://lucasmanual.com/mywiki/Bazaar