[Catalog-sig] Rewrite PyPI for App Engine?

[Noah Kantrowitz]

On Jun 25, 2010, at 12:53 AM, Antoine Pitrou wrote:

> Noah Kantrowitz <noah <at> coderanger.net> writes:
>> 
>> GAE provides a professionally managed, "infinitely" scalable (or at least a
>> heck of a lot more scalable
>> than any other single server is likely to be, still not a substitute for
>> mirrors), battle tested platform.
> 
> Infinite scalability is the new fashionable thing. But most websites can run on
> a single server fine, and PyPI seems to be one of those.
> 
> As for "battle tested", the most popular frameworks are, as is SQLAlchemy, as is
> Apache, as is PostgreSQL... I don't get what GAE buys in this area.
> 
>> There are already implementations of the GAE APIs that can be run
>> independently, so I don't think it is
>> quite as proprietary as you might think
> 
> Isn't it like chasing a moving target, though?
> For an analogy, there are independent implementations of the Win32 APIs, but I'm
> not sure anyone would trust Wine for running production services.
> 
>> Also, while Google is real
>> company and has its own business to attend to, they have almost always been
>> an ally and partner to the Python
>> community and would likely be willing to work with us more
>> so than, say, Amazon Web Services (Rackspace is also a big Python proponent >
> though, and has cloud offerings
>> similar to AWS).
> 
> But the point in this discussion is not to try to pit the various service
> providers one against another. It's to choose whether we want to rely on a
> proprietary platform (modulo alternate implementations, see above), or on a
> similarly battle-tested "standard" FLOSS-based stack.
> 
> And, assuming Google would like to provide servers and hosting, why wouldn't
> they simply provide Linux servers on which to run Apache and anything else we
> need to?

Its mostly a question of ongoing management. Apache+Linux+$SQLSERVER+etc can certainly handle our needs (which, lets face it, aren't really that complex), but we don't have a full-time management staff for our server. By leaning on Google (or Amazon, Rackspace, etc) we don't have to worry about the day-to-day details of running the site. How many of the recent PyPI downtimes have just required bouncing Apache? Wouldn't it have been nice if a site engineer got paged within 60 seconds and had it dealt with soon after instead of having to wait for one of the PyPI volunteers to notice and get to a computer? It isn't a question of capability, it is just where are our man-hours best spent: simple maintenance or actually improving the site?

--Noah