[Catalog-sig] PyPI and PEP 381

[Jannis Leidel]

Changing this subject..

Am 18.01.2010 um 19:03 schrieb Tarek Ziadé:

> On Mon, Jan 18, 2010 at 6:47 PM, Jannis Leidel <jannis at leidel.info> wrote:
> [..]
>> With "unsychronized" I mean indexes that are registered with PyPI to mirror it, but aren't up-to-date because of loss of connectivity/failure/interest.
> 
> So in  that case pip will just have to check for the last modified
> date, as explained in the PEP, to know how "fresh" a mirror is. The
> strategy to take depending on this freshness it's up to the client
> program.

Really, pip gets to decide which package is fresh enough? Like a PIP_BEST_BEFORE setting var?

>> Will the mirrors check periodically the main index for updates?
> Mirrors are dealing on their own to be up-to-date, by checking PyPI
> periodically.
>> Or will PyPI send out a message to the slaves, requesting an update, e.g. with WebHooks [1] or PubSubHubbub [2]?
> 
> No, the only event that will happen is the daily call the PyPI server
> will do on each mirror to get the stats. But that's unrelated to the
> mirror being updated.
>> Is this API going to be open for other non-official/non-open mirrors?
> Which API ?

The API of PyPI which would actively ping mirrors to update their package data. Those "webhooks" are already provided by project hosting sites like Google Code, Github and Bitbucket for post-commit/post-push situations. 

I don't see a reason why PyPI shouldn't use a similar technique to let mirrors know when an update happened. It lowers the number of periodic requests to PyPI from the mirrors, simplifies the setup of slave mirrors and -- as a bonus -- makes the life of continuous integration projects like pony-build easier.

Jannis