[Catalog-sig] Proposal: Move PyPI static data to the cloud for better availability
Tarek Ziadé
ziade.tarek at gmail.com
Tue Jun 15 22:46:46 CEST 2010
On Tue, Jun 15, 2010 at 10:14 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Tarek Ziadé wrote:
>> On Tue, Jun 15, 2010 at 7:34 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> [..]
>>>> So I think it would be better to focus on PEP 381, and make those
>>>> existing mirrors comply with it. And maybe work on the legal issues
>>>> you've mentioned
>>>
>>> That can all happen in parallel.
>>
>> I really doubt it.
>>
>> You have come with a cloud proposal and want it to be funded by the PSF.
>>
>> Your proposal is basically a proprietary mirroring system, and it competes
>> with the mirroring protocol we wanted to build, based on the existing
>> mirrors the community has.
>
> I'm not trying to compete with your mirror PEP, just trying
> to solve a problem.
We are trying to solve the same problem, aren't we ?
That is : avoiding any downtime when PyPI is used by setuptools and
derived tools.
So if you solve this problem by implementing a cloud system backed by
a PSF funding,
and managed by the PSF, and if you claim that there will be no more
downtime, then PEP 381
will be useless.
I am just arguing that I don't think it's the best solution, compared to what
was started e.g. a community network of mirrors.
>
>> So far I don't see any advantage in a cloud-based mirror managed by the PSF,
>> compared to a round of community mirrors.
>
> We can have it up and running in a few days and it doesn't
> require any changes to existing client tools, that's the main
> argument.
The global uptime of PyPI in this last year was probably around 99.9%,
so I don't think we are in such a rush to set up something in any case.
The problem occured in the past, and was fixed in a matter of hours.
every. time.
It's just that everytime it happens it makes us all want to improve the system.
So why don't we implement the best solution ? Maybe we could use a wiki page
and work on a synthetic overview of the pros and cons.
>
> The proposal solves a problem we have now and doesn't get in the
> way of PEP 381. Instead it buys it more time to get finalized,
> implemented and deployed on the client side.
>
> If you need funding for PEP 381, please write a proposal.
I won't.
I think we should decide here, all together, what is the best technical solution
to set up mirrors (e.g. cloud vs community)
Then, ask for its funding from the PSF.
> This would then also need to address the problem of added administration
> overhead (screening mirror server providers, getting them registered or
> removed, monitored and verified for correct operation, etc.).
This overhead is minimum compared to an in-house administration of a
full mirroring
system based on a cloud imho.
Regards
Tarek
--
Tarek Ziadé | http://ziade.org
More information about the Catalog-SIG
mailing list