[Catalog-sig] High-availability for PyPI, mirroring infrastructures?

Tarek Ziade tarek.ziade at ingeniweb.com
Mon Aug 11 13:49:24 CEST 2008


2008/8/11 Andreas Jung <lists at zopyx.com>

>
>
> --On 11. August 2008 12:03:32 +0200 Philipp von Weitershausen <
> philipp at weitershausen.de> wrote:
>
>
>>> I know Zope Corp has another script that generates a local copy by
>>> scanning PyPI through XML-RPC
>>> but it is a full copy.
>>>
>>
>> Yes, zc.mirrorpypislashsimple creates a full copy in the sense that it
>> mirrors the pages of *every* package. However, it does not mirror the
>> actual download archives. They are still retrieved from python.org.
>>
>>
>>
> zc.mirrorpypislashsimple is not much helpful right now since it does not
> mirror the real data (just checked the code and played a bit with it).
>
> A task for our Blackforest sprint would be:
>
> - all to pass-in a configuration file where you define with packages should
>  be mirrored where each line of the configuration would represent a
>  package pattern:
>
>  mirror.cfg:
>  zope.*
>  collective.*
>  z3c.*
>  lovely.*
>
> This would also fit in our haufe.eggserver infrastructure where we maintain
> a local egg repository on the filesystem.   zc.mirrorpypislashsimple could
> directly sync PyPI with our local egg server.


Our approach for this is to let the buildouts/easy_install scripts drive the
mirror, e.g. only the archives that are needed are pulled
on demand. The benefit of this approach is that you don't have to specify
which packages are mirrored in a configuration file.

Tarek


>
> Andreas




-- 
Tarek Ziadé - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - Bâtiment D - 9ème étage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une société du groupe Alter Way
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080811/762cf9c7/attachment.htm>


More information about the Catalog-SIG mailing list