PyPI CDN Updates For Greater Availability

Several changes were just deployed to PyPI's CDN. The general theme behind the changes is making it so that PyPI appears as functional as possible through a failure of the server hosting it. This should increase the availability of PyPI and enable things such as installation and browsing the site to continue to work through a catastrophic host failure on the PSF infrastructure. The details of what changes are: - Anonymous users will find that /pypi* pages are now cached for a short amount of time (currently 60s). - Objects will be stored in the cache for some time past their expiration date. They will not be used except in two circumstances: - A request is taking longer than 15s to complete, a "stale" object will be returned to prevent a pile up from occurring. - The backend[1] has been deemed unhealthy, in which case stale objects will be served in order to allow some level of functionality until the backend has been restored. - In the advent of an unhealthy backend all requests will be forced to be anonymous, making them eligible for the stale objects that have been cached. - The /mirrors and /security pages will be cached for a week, allowing them to likely be available through a backend failure making it easy to locate mirrors[2] or report a security issue. - Miscellaneous changes to normalize various things so that a single item in the cache will be able to be used for more requests, making it more likely that any particular request will be served from the Cache. [1] Backend in this context means the server hosting PyPI itself, what the CDN itself connects too. [2] Using the mirrors is done so at your own risk. None of the tools currently verify the downloads and they are downloaded over HTTP. This makes it trivial for an attacker to execute arbitrary code on your machine via a MITM. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
participants (1)
-
Donald Stufft