[Catalog-sig] PyPI and PEP 381

Tarek Ziadé ziade.tarek at gmail.com
Mon Jan 18 19:49:38 CET 2010

On Mon, Jan 18, 2010 at 7:29 PM, Jannis Leidel <jannis at leidel.info> wrote:
>> So in  that case pip will just have to check for the last modified
>> date, as explained in the PEP, to know how "fresh" a mirror is. The
>> strategy to take depending on this freshness it's up to the client
>> program.
> Really, pip gets to decide which package is fresh enough? Like a PIP_BEST_BEFORE setting var?

Pip gets to decide which *mirror* to use, given their last modified
dates. I am not sure what would be the best strategy here.

>>> Is this API going to be open for other non-official/non-open mirrors?
>> Which API ?
> The API of PyPI which would actively ping mirrors to update their package data.

We v'e discussed this last year, and came up with the conclusion that
asking PyPI to actively call each mirror was quite an intensive work
because it means it has to call each mirror for each update (there are
many updates per hour), and deal with a timeout for each request, etc.
IOW, work *all day long* just for that.

The other problem is that when a mirror is down or unreachable for a
while, it can't get that ping. So what happens is that the mirror
still needs to update itself in these situations. (because PyPI will
certainly not implement a replay-system when some ping fails.)

So why bother setting up two different update systems ? each mirror
can look at the CHANGELOG every minute or so and get updated on their


Tarek Ziadé | http://ziade.org

More information about the Catalog-SIG mailing list