[Catalog-sig] High-availability for PyPI, mirroring infrastructures?

[Tarek Ziade]

2008/8/11 Jim Fulton <jim at zope.com>

>
> On Aug 11, 2008, at 2:56 AM, Andreas Jung wrote:
>
>  Hi there,
>>
>> Python eggs and zc.buildout are playing a major role in Zope world - both
>> for development and deployment. PyPI right now is apparently a
>> single-point-of-failure. Although the availability of PyPI become much
>> better over time, the complete infrastructure is not highly available which
>> is crucial when you are doing commercial development.
>>
>> What is the perspective for addressing this issue? I have seen that
>> Ingeniweb maintains/maintained a PyPI mirror (does not seem to be up2date).
>>
>
> There's also one at http://download.zope.org/simple/
>
>  What is the current recommended way for building a (private) mirror?
>>
>
> There's a project at http://svn.zope.org/zc.mirrorcheeseshopslashsimple/
> for mirroring the pypi index data.
>
> In terms of reducing risk, I think that zc.sourcerelease is also relevant,
> because it lets you make a self-contained snapshot of an application's
> source, with all of the eggs used.  If you need to deploy an application,
> you can use a source release, or a binary release built from it, without any
> need to talk to a package index.


Jim,
Since mirrors for PyPI and alternative egg servers are being , what about
making the "index" variable in
zc.buildout accept multiple values, like what find-links does ?

This could let us use these mirrors/alternative servers at the buildout
level like this:

  [buildout]

  index =
      http://my.mirror1/simple
      http://pypi.python.org/simple
      http://my.alternative.egg.server/simple

and let buildout look for packages in each Package index object,
sequentially

two use cases:

- a mirror or PyPI is down
- an egg is not present in every server

Tarek


>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
>
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>



-- 
Tarek Ziadé - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - Bâtiment D - 9ème étage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une société du groupe Alter Way
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080812/cd48a7d5/attachment-0001.htm>