[Distutils] PyPI Migrated to New Infrastructure with some Breakage

Kyle Kelley rgbkrk at gmail.com
Sun Jan 26 05:15:15 CET 2014


Congrats! Thanks for always making the PyPI infrastructure better and
better.

Where are the states stored?


On Sat, Jan 25, 2014 at 5:18 PM, Donald Stufft <donald at stufft.io> wrote:

>
> On Jan 25, 2014, at 7:04 PM, Chris Jerdonek <chris.jerdonek at gmail.com>
> wrote:
>
> > On Sat, Jan 25, 2014 at 3:38 PM, Donald Stufft <donald at stufft.io> wrote:
> >> Today (Sat Jan 25, 2014) the Infrastructure team has migrated PyPI to
> new
> >> infrastructure.
> >>
> >> The old infrastructure was:
> >>
> >> - a single database server managed by OSUOSL
> >> - a pair of load balancers shared by all of the python.org services
> hosted on
> >>  OSUOSL
> >> - a single backend VM that served as everything else for PyPI
> >>
> >> The VM that was acting as the backend server from PyPI was partially
> hand
> >> configured and partially setup to be managed by chef. Additionally it
> had an
> >> issue that caused it to kernel panic every so often which had been the
> cause of
> >> a number of downtimes in the last few months. Because it was primarily
> >> configured and administered by hand and because the way it was set up
> it was
> >> not feasible to have any sort of failover or spare.
> >>
> >> The new infrastructure is:
> >>
> >> - 2 Web VMs
> >> - 2 Database servers in Master/Slave Configuration
> >> - 2 PgPool Servers pooling connections to the database servers and load
> >>  balancing reads across them.
> >> - 2 GlusterFS servers backed by Cloud Block Storage acting as the file
> storage
> >>  for package and package docs
> >> - 1 metrics server to handle updating the download counts as they come
> in from
> >>  Fastly
> >>
> >> All of the VMs are hosted on Rackspace’s Public Cloud and have their
> >> configuration completely controlled and managed using Salt. Going
> forward this
> >
> > Can you say a little about the choice to use Salt instead of Chef?  I
> > don't really care either way, but am just curious.  Is it because Salt
> > is written in Python, or were there other reasons (functionality,
> > etc)?
> >
> > --Chris
>
> I’d need to ask Ernest to be sure, but I believe it was mostly that he was
> more familiar
> with it. The fact that it was written in Python was a bonus as well ;) I
> don’t think that
> there was anything that Chef was missing or that Salt had over Chef, just
> familiarity
> of the person who did most of the work. I’ll double check with Ernest to
> make sure there
> wasn’t another reason :)
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140125/a154fd99/attachment.html>


More information about the Distutils-SIG mailing list