[Catalog-sig] Pypi cdn for hosted packages

[Jim Fulton]

On Thu, Feb 28, 2013 at 5:39 AM, Jesse Noller <jnoller at gmail.com> wrote:
> Thread fork.
>
> Anyway. I know we have at least 1 major rep of a cloud provider on the list, and I have at least one off in my pocket.
>
> I'd like to start discussing (completely ignoring past efforts and discussion which got bogged down) how we can start distributing the package data we host via CDN rather than the mirroring system.

Woohoo!!!

One issue is how to keep the relatively dynamic stuff up to date.
One option is to bypass the CDN for dynamic content, but a better
approach might be to use CDN-provided invalidation APIs.

I'm mainly thinking of the simple index pages, which are relatively
static, but, when they change, you want to get the change quickly.
Ideally, when there's an update to package, PyPI would
invalidate the index page.  Everything should have a really long
cache interval.

BTW, we use CloudFront fairly extensively for media.  We get a lot
more origin requests than we expected, given the cache intervals
we'd set.  (It was still a big win for us.)

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm