On Sun, Oct 12, 2008 at 2:12 PM, "Martin v. Löwis" martin@v.loewis.de wrote:
could you explain why is that a problem ?
It produces significant load on the master. If you look at the web stats, e.g for September:
http://pypi.python.org/webstats/usage_200809.html
you see that there had been 5671455 hits, or 41%, of accesses through wget.
The problem with wget mirroring is that it needs to read *many* pages, to find out the *few* changes.
Sure,
FWIW, it's also the case that 4940769 hits originate from France. Could it be that you are alone responsible for 40% of the traffic on PyPI?
Yes, I am the only Python developer in France. That's me.
Just kidding :)
France has a lot of python/plone developers that triggers buildouts every day, so I am pretty sure the mirrors don't make the whole traffic in PyPI.
we could probably do things better though. Here's my proposal:
+ see if we can locate the mirrors, so for instance, if i register a "Paris mirror" people will eventually go there because it is the nearest location for them. (à la CPAN)
+ create a new user agent for mirroring tools
Regards Tarek
Regards, Martin