[Catalog-sig] PyPI slowdowns
"Martin v. Löwis"
martin at v.loewis.de
Fri Aug 31 14:25:15 CEST 2007
> In the interest of giving people running PyPI data on problem
> periods, PyPI struggled on several occasions over the past few days:
> Aug 25, 16:01-16:07
> Aug 26, 9:00- 9:21
> Aug 26, 14:56-15:00
> Aug 27, 03:56-04:06
> All of these times are UTC.
> I haven't otherwise noticed problems like this for quite a while.
Thanks. I couldn't quite match all these incidences with the log files,
but apparently, what happens is this:
- some application gets overloaded (probably the Wiki, but I'm not
certain), for some reason
- FastCGI finds that the application does not respond quickly enough,
and kills it
- it does that a number of times, and then decides to back-off
- as a consequence, all Apache process start blocking for that
application. This can be seen at
when there are 256 Apache processes.
- as a consequence, the entire web server is unaccessible, as
the MaxClients limit is exhausted.
I don't know how detect this problem before it happens. I have
added response-time measuring to MoinMoin; if a response takes more
than 10s, it will refuse all requests with a QUERY_STRING, for
120s. As the expensive MoinMoin requests are those with query
parameters, I hope that this will cause fast processing of any
backlog that may have been built up.
More information about the Catalog-SIG