[Catalog-sig] PyPI replication project

[Andreas Jung]

On 10.10.2008 8:38 Uhr, Tarek Ziade wrote:
>
>
> 2008/10/10 Andreas Jung <lists at zopyx.com <mailto:lists at zopyx.com>>
>
>     Am 09.10.2008 18:59 Uhr, Martin v. Löwis schrieb:
>
>             Mirrors help every other packaging system. So it stands to
>             reason
>             that it would help pypi too. I think since many zope people
>             have been
>             using mirrors instead of using pypi directly... pypi has
>             been more
>             available. It's running lots better for other reasons too...
>             but less
>             load is probably also nice for pypi :)
>
>
>         I'm fine with people operating their own mirrors. I just don't think
>         it can be made *invisible* to users that they use a mirror. In the
>         mirroring systems for Linux distributions, for example, people have
>         to explicitly select which mirror they want to use (and accept that
>         the mirror may lag behind by a day or so). It's also clear that
>         it is
>         a "mere" mirror.
>
>
>     Implict or explict mirror selection is not primary point in phase 1
>     of the project. The point is that we must have access to the
>     distribution packages and eggs at any time - independent of the
>     available of PyPI (either related to issues with the PyPI server or
>     caused by internet outages or routing problems).
>
>     An implicit selection of a mirror in case of an detected outtage
>     would be nice but this is possibly not the most important issue
>     right now.
>     We can always reconfigure out buildout configurations easily to a
>     new server or define a series of mirroring servers.
>
>
>
>         What Andreas was asking how a distributed PyPI installation
>         could work,
>         by which I assume he was asking for one that a) is invisible (of
>         called
>         misleadingly "transparent") to users, and b) allows updates to
>         replica.
>
>         I'm skeptical that such an system would work all that well, and can
>         be created in a reasonable amount of time
>
>
>     As stated earlier: we can already define multiple servers as part
>     of a buildout configuration. A better mirror selection algorithm
>     would be nice to have for the future but right now we don't actually
>     need it and can live with the current state.
>
>
> I think the key is to choose upon several indexes, not to use extra
> find-links.
>
> (I have submitted a patch for setuptools to handle several indexes for
> that: I think the key
> is to choose upon several indexes, not to used)
>
> Last, I think we should create a mirror registration system on PyPI, to
> ping mirrors when
> a new package is uploaded, so the sync is simpler.

I really want to defer this discussion for now since it is not relevant 
for phase 1 of the project.

Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 330 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20081010/3c2d3d43/attachment.vcf>