<div dir="ltr">I'll get in touch with Martin.</div><div class="gmail_extra"><br><br><div class="gmail_quote">On 26 January 2014 11:40, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"><p dir="ltr"><br>
On 26 Jan 2014 09:51, "Richard Jones" <<a href="mailto:richard@python.org" target="_blank">richard@python.org</a>> wrote:<br>
><br>
> Thanks everyone who helped make this happen.</p>
</div><p dir="ltr">Indeed - fine work! :)</p><div class="im">
<p dir="ltr">> From my perspective* I believe the ssh upload mechanism was added to address security issues around the basic-auth-over-http method used historically. Now uploads *may* be done over https, and those using the ssh method can move over to using twine or pip upload, I think that it's reasonable to discontinue support for ssh uploads.</p>
</div><p dir="ltr">Yes, I agree that pointing the (very few) pypissh users towards twine as a replacement is the most reasonable option at this point - we should chat to MvL about putting a notice to that effect in the pypissh README (IIRC, MvL is the creator of that upload option). </p>
<p dir="ltr">Cheers,<br>
Nick.</p><div class="HOEnZb"><div class="h5">
<p dir="ltr">><br>
><br>
> Richard<br>
><br>
> * as the guy who will be hassled if its loss is noticed ;)<br>
><br>
><br>
> On 26 January 2014 10:38, Donald Stufft <<a href="mailto:donald@stufft.io" target="_blank">donald@stufft.io</a>> wrote:<br>
>><br>
>> Today (Sat Jan 25, 2014) the Infrastructure team has migrated PyPI to new<br>
>> infrastructure.<br>
>><br>
>> The old infrastructure was:<br>
>><br>
>> - a single database server managed by OSUOSL<br>
>> - a pair of load balancers shared by all of the <a href="http://python.org" target="_blank">python.org</a> services hosted on<br>
>> OSUOSL<br>
>> - a single backend VM that served as everything else for PyPI<br>
>><br>
>> The VM that was acting as the backend server from PyPI was partially hand<br>
>> configured and partially setup to be managed by chef. Additionally it had an<br>
>> issue that caused it to kernel panic every so often which had been the cause of<br>
>> a number of downtimes in the last few months. Because it was primarily<br>
>> configured and administered by hand and because the way it was set up it was<br>
>> not feasible to have any sort of failover or spare.<br>
>><br>
>> The new infrastructure is:<br>
>><br>
>> - 2 Web VMs<br>
>> - 2 Database servers in Master/Slave Configuration<br>
>> - 2 PgPool Servers pooling connections to the database servers and load<br>
>> balancing reads across them.<br>
>> - 2 GlusterFS servers backed by Cloud Block Storage acting as the file storage<br>
>> for package and package docs<br>
>> - 1 metrics server to handle updating the download counts as they come in from<br>
>> Fastly<br>
>><br>
>> All of the VMs are hosted on Rackspace’s Public Cloud and have their<br>
>> configuration completely controlled and managed using Salt. Going forward this<br>
>> will allow us to easily scale out as required or kill malfunctioning servers<br>
>> and spin up new ones easily. Additionally the setup has been setup so that<br>
>> where possible there is two servers performing the same role, ideally in an<br>
>> Active/Active configuration but at least in a Master/Slave configuration. This<br>
>> should allow PyPI to be far more stable moving forward and make downtimes much<br>
>> easier to recover from.<br>
>><br>
>> The services are still fronted by Fasty’s CDN and in the new infrastructure<br>
>> we’ve removed our load balancer and have replaced it by having Fastly handle<br>
>> the load balancing for us. Additionally we’ve recently setup a static mirror of<br>
>> PyPI that is updated once every minute. This is hosted on Rackspace cloud as<br>
>> well but in a separate data center from the rest of PyPI. Fastly is configured<br>
>> to fall back to this static mirror in the case that neither of the two web<br>
>> heads are functioning. This should ensure that even in the event of a<br>
>> catastrophic failure of the PyPI service that the bulk of package installations<br>
>> should hopefully remain working.<br>
>><br>
>> The bad news, (and the “Breakage” from the subject) is that while the new<br>
>> infrastructure was being planned out, built, and migrated to the “pypissh”<br>
>> package was forgotten. The pypissh package is an alternative way to upload<br>
>> packages to PyPI however it is very difficult, because of the way it works, to<br>
>> provide HA support for it as we’ve set up for everything else. We don’t have<br>
>> any numbers for how many people are actively using this package but looking<br>
>> at a roughly 2 week chunk of time in PyPI’s download history, the pypissh<br>
>> package was downloaded 7 times by a browser, and 7 times by pip. All other<br>
>> downloads were caused by the mirroring system.<br>
>><br>
>> As of right now pypissh is non functional and due to the difficulty in HAing<br>
>> and monitoring the current setup and because it is apparently has a very<br>
>> small set of users we would like to effectively kill off this particular<br>
>> service. Additionally the benefits of pypissh have been reduced now that PyPI<br>
>> is available over a TLS connection with a well trusted certificate. My question<br>
>> to you is, is this something that distutils-sig is willing to have happen? If<br>
>> we are to re-enable pypissh we’ll need to write a new solution to doing it that<br>
>> can be properly HA’d and we’d prefer to put our efforts into improving things<br>
>> for a much larger set of people.<br>
>><br>
>> So yea, PyPI should be loads more stable and more reliable now.<br>
>><br>
>> -----------------<br>
>> Donald Stufft<br>
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA<br>
>><br>
>><br>
>> _______________________________________________<br>
>> Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" target="_blank">Distutils-SIG@python.org</a><br>
>> <a href="https://mail.python.org/mailman/listinfo/distutils-sig" target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" target="_blank">Distutils-SIG@python.org</a><br>
> <a href="https://mail.python.org/mailman/listinfo/distutils-sig" target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br>
><br>
</p>
</div></div></blockquote></div><br></div>