[Catalog-sig] PyPI and PEP 381

David Lyon david.lyon at preisshare.net
Mon Jan 18 22:52:05 CET 2010


> On Mon, Jan 18, 2010 at 7:29 PM, Jannis Leidel <jannis at leidel.info> wrote:
> [..]
>>> So in  that case pip will just have to check for the last modified
>>> date, as explained in the PEP, to know how "fresh" a mirror is. The
>>> strategy to take depending on this freshness it's up to the client
>>> program.
>>
>> Really, pip gets to decide which package is fresh enough? Like a
>> PIP_BEST_BEFORE setting var?
>
> Pip gets to decide which *mirror* to use, given their last modified
> dates. I am not sure what would be the best strategy here.
>
>
>>>> Is this API going to be open for other non-official/non-open mirrors?
>>> Which API ?
>>
>> The API of PyPI which would actively ping mirrors to update their
>> package data.
>
> We v'e discussed this last year, and came up with the conclusion that
> asking PyPI to actively call each mirror was quite an intensive work
> because it means it has to call each mirror for each update (there are
> many updates per hour), and deal with a timeout for each request, etc.
> IOW, work *all day long* just for that.
>
> The other problem is that when a mirror is down or unreachable for a
> while, it can't get that ping. So what happens is that the mirror
> still needs to update itself in these situations. (because PyPI will
> certainly not implement a replay-system when some ping fails.)
>
> So why bother setting up two different update systems ? each mirror
> can look at the CHANGELOG every minute or so and get updated on their
> side.

You guys could make it a lot easier for yourselves and look
at twisted/jabber/rss.

They're industrial protocols for pushing out feed information
slowly. They take a few hours to set up and don't bog the
processors down in any way.

Jabber(twisted) and xmpp is very easy to push update information
messages out on.

David




More information about the Catalog-SIG mailing list