[Distutils] PyPI Migrated to New Infrastructure with some Breakage

[Kyle Kelley]

s/states/salt states/


On Sat, Jan 25, 2014 at 9:15 PM, Kyle Kelley <rgbkrk at gmail.com> wrote:

> Congrats! Thanks for always making the PyPI infrastructure better and
> better.
>
> Where are the states stored?
>
>
> On Sat, Jan 25, 2014 at 5:18 PM, Donald Stufft <donald at stufft.io> wrote:
>
>>
>> On Jan 25, 2014, at 7:04 PM, Chris Jerdonek <chris.jerdonek at gmail.com>
>> wrote:
>>
>> > On Sat, Jan 25, 2014 at 3:38 PM, Donald Stufft <donald at stufft.io>
>> wrote:
>> >> Today (Sat Jan 25, 2014) the Infrastructure team has migrated PyPI to
>> new
>> >> infrastructure.
>> >>
>> >> The old infrastructure was:
>> >>
>> >> - a single database server managed by OSUOSL
>> >> - a pair of load balancers shared by all of the python.org services
>> hosted on
>> >>  OSUOSL
>> >> - a single backend VM that served as everything else for PyPI
>> >>
>> >> The VM that was acting as the backend server from PyPI was partially
>> hand
>> >> configured and partially setup to be managed by chef. Additionally it
>> had an
>> >> issue that caused it to kernel panic every so often which had been the
>> cause of
>> >> a number of downtimes in the last few months. Because it was primarily
>> >> configured and administered by hand and because the way it was set up
>> it was
>> >> not feasible to have any sort of failover or spare.
>> >>
>> >> The new infrastructure is:
>> >>
>> >> - 2 Web VMs
>> >> - 2 Database servers in Master/Slave Configuration
>> >> - 2 PgPool Servers pooling connections to the database servers and load
>> >>  balancing reads across them.
>> >> - 2 GlusterFS servers backed by Cloud Block Storage acting as the file
>> storage
>> >>  for package and package docs
>> >> - 1 metrics server to handle updating the download counts as they come
>> in from
>> >>  Fastly
>> >>
>> >> All of the VMs are hosted on Rackspace’s Public Cloud and have their
>> >> configuration completely controlled and managed using Salt. Going
>> forward this
>> >
>> > Can you say a little about the choice to use Salt instead of Chef?  I
>> > don't really care either way, but am just curious.  Is it because Salt
>> > is written in Python, or were there other reasons (functionality,
>> > etc)?
>> >
>> > --Chris
>>
>> I’d need to ask Ernest to be sure, but I believe it was mostly that he
>> was more familiar
>> with it. The fact that it was written in Python was a bonus as well ;) I
>> don’t think that
>> there was anything that Chef was missing or that Salt had over Chef, just
>> familiarity
>> of the person who did most of the work. I’ll double check with Ernest to
>> make sure there
>> wasn’t another reason :)
>>
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
>> DCFA
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140125/bbdccab6/attachment.html>