[Catalog-sig] PyPI and PEP 381

Jannis Leidel jannis at leidel.info
Tue Jan 19 00:06:32 CET 2010

Am 18.01.2010 um 23:20 schrieb Martin v. Löwis:

>>> Can you please propose a specific change to something?
>> I'm proposing:
>> Use push notifications (HTTP POST requests) to tell the mirrors about
>> the availability of updates on the main main index, instead of having
>> the mirrors pull that information periodically.
> Ok: what specific notifications (i.e.: what specific URL, what specific
> payload), and what specific servers to contact (i.e. where does the
> list of servers come from)?

Extra URLs of the mirror

1. /update

receives a payload with a timestamp to tell the mirror to update itself
the mirror should use that timestamp to compare it to its own "last-modified" timestamp.

2. /update/<project-name>/<version>

receives metadata of each project (same metadata the XML-RPC API uses) as well as a timestamp.

Extra URLs of PyPI


In case a mirror doesn't accept the update payload (e.g. due to temporary loss of connectivity) the mirror needs to be able to catch up at a later time.

Correct me if I'm wrong, but I believe the XML-RPC API is capable of providing that information already. Once the mirror gained the list of projects still to be updated, it would mirror them "manually".

List of servers

Actually this wouldn't differ from the PEP: starting with trusted mirrors. And I absolutely agree with what you earlier said, to slowly start with such a new feature.

On the long run -- but I keep that for a later proposal -- there would be also a way to register non-mirror consumers of the push notifications on the PyPI website.


More information about the Catalog-SIG mailing list