[Catalog-sig] High-availability for PyPI, mirroring infrastructures?

Tarek Ziade tarek.ziade at ingeniweb.com
Mon Aug 11 09:48:12 CEST 2008


2008/8/11 Andreas Jung <lists at zopyx.com>

> Hi there,
>
> Python eggs and zc.buildout are playing a major role in Zope world - both
> for development and deployment. PyPI right now is apparently a
> single-point-of-failure. Although the availability of PyPI become much
> better over time, the complete infrastructure is not highly available which
> is crucial when you are doing commercial development.
>
> What is the perspective for addressing this issue? I have seen that
> Ingeniweb maintains/maintained a PyPI mirror (does not seem to be up2date).
> What is the current recommended way for building a (private) mirror?


Hi Andreas

to do our mirror, we use iw.eggproxy. It is a simple web proxy that grabs
files over pypi upon requests.
every file is then kept locally for the next call.

I know Zope Corp has another script that generates a local copy by scanning
PyPI through XML-RPC
but it is a full copy.



> How big (in GB) is currently PI? If a company would be interested to
> provide a mirroring server, how much diskspace (and bandwidth) would be
> needed?


Around 5 gigas iirc

Regards

Tarek


>
>
> Regards,
> Andreas
>
>
> --
> ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
> Web: www.zopyx.com - Email: info at zopyx.com - Phone +49 - 7071 - 793376
> Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
> Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK
> ------------------------------------------------------------------------
> E-Publishing, Python, Zope & Plone development, Consulting
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>
>


-- 
Tarek Ziadé - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - Bâtiment D - 9ème étage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une société du groupe Alter Way
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080811/61492af4/attachment.htm>


More information about the Catalog-SIG mailing list