[Catalog-sig] pypi mirrors in bad shape

Jannis Leidel jannis at leidel.info
Thu Jan 24 15:28:18 CET 2013

On Wed, Jan 23, 2013 at 10:59 PM, Donald Stufft <donald.stufft at gmail.com>wrote:

> On Wednesday, January 23, 2013 at 4:52 PM, Christian Theune wrote:
> My proposal would be to sit down at PyCon to dig a bit more into the
> code to make it more robust. My feeling is that the current client
> tries way to hard on a very low-level API (httplib) for a lot of
> mechanics. I'd be happy to refactor and provide tests, I think.
> Also, a script to determine internal consistency and consistency
> compared to PyPI would be nice. Then again, the rsync idea might not be
> that far off regarding the amount of work and the problems we're
> dealing with …
> Crate.io uses it's own mirroring code that seems to handle things
> a bit better than pep381client does (infact I originally wrote it because
> of that). However it creates a full mirror with metadata and all and is
> currently being refactored to be a ton simpler.
> I still think for the "dumb" mirrors that setting up PyPI mirroring
> protocol to work via rsync is the right direction to go in. Rsync
> has way more manhours invested into it and a lot more testing
> for what pep381client essentially boils down to which is "keep
> these files and those files in sync".


But till that rsync API is set up, does anyone know what changed on PyPI
(or wherever) recently so that the mirrors so often go AWOL?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130124/00286db8/attachment.html>

More information about the Catalog-SIG mailing list