[Twisted-Python] web site error
![](https://secure.gravatar.com/avatar/5dffa63f0d07e6ff08f8abf5fff832fd.jpg?s=120&d=mm&r=g)
Hi, I'm trying to navigate the site but many (almost all) of the pages show an error page of the form: web.Server Traceback exceptions.IOError: [Errno 24] Too many open files: '/var/www/WebSite/twistedmatrix.com/products/twisted' ... just thought you might like to know. Gil
![](https://secure.gravatar.com/avatar/3a7e70f3ef2ad1539da42afc85c8d09d.jpg?s=120&d=mm&r=g)
On Wed, Oct 30, 2002 at 10:48:25AM -0500, Gil Hauer wrote:
Fixed, for now. Glyph mentioned this bug last night, and now we see the results of it ... The web server is leaking sockets. Anyone have a clue? -- Christopher Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Tue, 29 Oct 2002 17:10:11 +0100, Christopher Armstrong <radix@twistedmatrix.com> wrote:
Well, it's not really leaking them. As far as I can tell, the other end of the socket isn't closing the connection, so Twisted just keeps it open 'cause there's no timeout. If there were a timeout, that would mean that proxying certain things through HTTP might break. What I'm wondering is why _other_ webservers don't suffer from this problem. Do they have a timeout? Is there something else happening in Twisted that I'm missing? This is certainly a very frustrating bug to test for. (After all, it's possible that this is some impossibly obscure errno that we're not accounting for, too...) -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/fe24473d748a78f2ae29cdbc11c293a4.jpg?s=120&d=mm&r=g)
Glyph Lefkowitz wrote:
For whatever little it is worth, I've had troubles like this in the past with our own homebrew webserver in our games at work. There are some buggy web caches and other things out there that keep connections open. We had implemented a connection timeout to work around the problem. - Bruce
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002 16:47:53 -0700, Bruce Mitchener <bruce@cubik.org> wrote:
For whatever little it is worth, I've had troubles like this in the past with our own homebrew webserver in our games at work.
Actually, this is really useful info for me. It means that I can implement a timeout with impunity and not worry that I'm doing bad things :). Now just have to write the code that determines if a connection has been idle... -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/d50173484c13acf1179d42faa660a76b.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002, Glyph Lefkowitz wrote:
Generally, one would rarely want to have sockets that don't timeout. To have sockets without timeout opens a server up to easy DoS attacks. I can't imagine a time when you wouldn't want to time out sockets, even if you're trying to tunnel something over HTTP. If you pull out an ethernet cable or a modem phone line, connections don't get closed either. Any assumption that connections will always close nicely is flawed. Though, maybe I'm misunderstanding what you mean by 'no timeout' Personally, when I was running high volume web servers, we'd tune down the keepalive timers to a fairly short interval to keep sockets from getting tied up too long, to prevent resource availability issues that we were seeing. J
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002 23:54:46 +0000 (GMT), Jeremy Noetzelman <jjn@kriln.com> wrote:
Just out of curiosity, how does one configure this timeout with apache? -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/41e7d91156750be9c240a2597eadbb94.jpg?s=120&d=mm&r=g)
Glyph Lefkowitz <glyph@twistedmatrix.com> writes:
Look at the Timeout and KeepAlive directives in httpd.conf or apache docs. Basically either timeout or max # of requests, and connection will be cut. As for 2.4 kernels, check Documentation/networking/ip-sysctl.txt and tcp_keepalive_* there. Note: if you drop this to short value at tm.com, ssh users will hate you :) --j
![](https://secure.gravatar.com/avatar/3a7e70f3ef2ad1539da42afc85c8d09d.jpg?s=120&d=mm&r=g)
On Wed, Oct 30, 2002 at 10:48:25AM -0500, Gil Hauer wrote:
Fixed, for now. Glyph mentioned this bug last night, and now we see the results of it ... The web server is leaking sockets. Anyone have a clue? -- Christopher Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Tue, 29 Oct 2002 17:10:11 +0100, Christopher Armstrong <radix@twistedmatrix.com> wrote:
Well, it's not really leaking them. As far as I can tell, the other end of the socket isn't closing the connection, so Twisted just keeps it open 'cause there's no timeout. If there were a timeout, that would mean that proxying certain things through HTTP might break. What I'm wondering is why _other_ webservers don't suffer from this problem. Do they have a timeout? Is there something else happening in Twisted that I'm missing? This is certainly a very frustrating bug to test for. (After all, it's possible that this is some impossibly obscure errno that we're not accounting for, too...) -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/fe24473d748a78f2ae29cdbc11c293a4.jpg?s=120&d=mm&r=g)
Glyph Lefkowitz wrote:
For whatever little it is worth, I've had troubles like this in the past with our own homebrew webserver in our games at work. There are some buggy web caches and other things out there that keep connections open. We had implemented a connection timeout to work around the problem. - Bruce
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002 16:47:53 -0700, Bruce Mitchener <bruce@cubik.org> wrote:
For whatever little it is worth, I've had troubles like this in the past with our own homebrew webserver in our games at work.
Actually, this is really useful info for me. It means that I can implement a timeout with impunity and not worry that I'm doing bad things :). Now just have to write the code that determines if a connection has been idle... -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/d50173484c13acf1179d42faa660a76b.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002, Glyph Lefkowitz wrote:
Generally, one would rarely want to have sockets that don't timeout. To have sockets without timeout opens a server up to easy DoS attacks. I can't imagine a time when you wouldn't want to time out sockets, even if you're trying to tunnel something over HTTP. If you pull out an ethernet cable or a modem phone line, connections don't get closed either. Any assumption that connections will always close nicely is flawed. Though, maybe I'm misunderstanding what you mean by 'no timeout' Personally, when I was running high volume web servers, we'd tune down the keepalive timers to a fairly short interval to keep sockets from getting tied up too long, to prevent resource availability issues that we were seeing. J
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Wed, 30 Oct 2002 23:54:46 +0000 (GMT), Jeremy Noetzelman <jjn@kriln.com> wrote:
Just out of curiosity, how does one configure this timeout with apache? -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
![](https://secure.gravatar.com/avatar/41e7d91156750be9c240a2597eadbb94.jpg?s=120&d=mm&r=g)
Glyph Lefkowitz <glyph@twistedmatrix.com> writes:
Look at the Timeout and KeepAlive directives in httpd.conf or apache docs. Basically either timeout or max # of requests, and connection will be cut. As for 2.4 kernels, check Documentation/networking/ip-sysctl.txt and tcp_keepalive_* there. Note: if you drop this to short value at tm.com, ssh users will hate you :) --j
participants (7)
-
Bruce Mitchener
-
Christopher Armstrong
-
Gil Hauer
-
Glyph Lefkowitz
-
Jaakko Niemi
-
Jeremy Noetzelman
-
Moshe Zadka