On Thu, Dec 30, 2004 at 03:44:32PM +0100, Antoine Pitrou wrote: [...]
Looking at the Twisted code, I see that very long timeouts have been defined for both protocols.http.HTTPFactory and web.server.Site (60*60*12, that is 12 hours!). If I override the "timeout" parameter when constructing the Site receiving XMLRPC connections, then the problem disappears: reactor.listenTCP(xmlrpc_port, server.Site(self, timeout=30), interface=xmlrpc_host)
I think the default values in Twisted are quite bogus and should be changed to more sensible ones. 30 or 60 seconds is ok in the context of an HTTP connection. Very long timeouts on the other hand make the server very vulnerable.
One person's "reasonable" timeout is another person's "ludicrous"... this sort of thing will need tuning per-application. 12 hours is conservative, but highly unlikely to interfere with what anyone with pre-timeout Twisted expects, while solving the problem twistedmatrix.com's web site was having with the occasional connection not being closed. And having long-running connections mysteriously die due to a timeout setting you didn't even know existed can be very surprising and frustrating to debug.
That said, the HTTP code is now quite careful with timeouts: it disables them while sending the response (or more accurately, after receiving a complete request, and then reenables them after it finishes sending the response), so the only time a timeout can occur is while waiting for a request to be sent -- i.e. it won't time out someone that's downloading an ISO over a dial-up link. So there probably is room to reduce it.
However, for very large POST requests over a slow link (ever tried to attach a large file to your webmail over dial-up?), 30 or 60 seconds is still too slow, but I could see an argument for, say, 30 minutes being the default. Part of the issue here is backwards compatibility -- there may be deployments of old versions of Twisted out there that depends on the conservative (or no!) timeouts, and we don't want to gratuitously break them. If we do decide to be more aggressive by default, we need to make this very clear in the documentation for the next version.
Regardless, if a particular application has specific needs, they can always change the setting, just like you did. It's not a major issue.
-Andrew.