[Twisted-Python] CLOSE_WAIT (debian)

Hi, I have a fairly simple twisted daemon based on the starpy protocol, which I am running in combination with the asterisk pbx. I sometimes this daemon accumulates CLOSE_WAIT connections in a rather unpredictable fashion, which when stacked up eventually cause the daemon to stop responding to new connections. My understanding is that there is no default timeout for the CLOSE_WAIT socket state, as per the relevant rfc standard, and that in any netstat declares that CLOSE_WAIT indicates that the remote end of the connection has shut down, and that it is now up to the local application to finish closing the socket. The implication is that there is something wrong with my application code, however before posting the whole thing I'd like to know if there are any cannonical reasons for this kind of issue being raised in twisted applications. For instance is this likely to occur when one uses defertothread for blocking third party code, and the child process spawned to handle this returns to find the parent process already gone? ( e.g. my twisted app spawns for a new call from asterisk, but the call instance is destroyed for some reason before the defertothread that it executes returns?) I'm having trouble reliably causing the issue, but it continues to pop up, so any general information about regarding debugging twisted applications would be of help. Thanks!

Hi, I've been digging around a bit more and it seems like it might be advisable for me to use the d.setTimeout(TIMEOUT) function for my deferToThread objects. I see from the docstrings that this is deprecated, and that I should rely on the timeout mechanism supplied by the API I'm using, however that doesn't seem to suggest an immediate solution. I dug up the following reference, but is it really such a bad idea to use d.setTimout(TIMEOUT)? http://twistedmatrix.com/pipermail/twisted-python/2004-April/007536.html Thanks! 2008/6/5 Josef Novak <josef.robert.novak@gmail.com>:
Hi, I have a fairly simple twisted daemon based on the starpy protocol, which I am running in combination with the asterisk pbx. I sometimes this daemon accumulates CLOSE_WAIT connections in a rather unpredictable fashion, which when stacked up eventually cause the daemon to stop responding to new connections.
My understanding is that there is no default timeout for the CLOSE_WAIT socket state, as per the relevant rfc standard, and that in any netstat declares that CLOSE_WAIT indicates that the remote end of the connection has shut down, and that it is now up to the local application to finish closing the socket.
The implication is that there is something wrong with my application code, however before posting the whole thing I'd like to know if there are any cannonical reasons for this kind of issue being raised in twisted applications. For instance is this likely to occur when one uses defertothread for blocking third party code, and the child process spawned to handle this returns to find the parent process already gone? ( e.g. my twisted app spawns for a new call from asterisk, but the call instance is destroyed for some reason before the defertothread that it executes returns?)
I'm having trouble reliably causing the issue, but it continues to pop up, so any general information about regarding debugging twisted applications would be of help.
Thanks!

On 08:25 am, josef.robert.novak@gmail.com wrote:
Hi, I've been digging around a bit more and it seems like it might be advisable for me to use the d.setTimeout(TIMEOUT) function for my deferToThread objects. I see from the docstrings that this is deprecated, and that I should rely on the timeout mechanism supplied by the API I'm using, however that doesn't seem to suggest an immediate solution. I dug up the following reference, but is it really such a bad idea to use d.setTimout(TIMEOUT)? http://twistedmatrix.com/pipermail/twisted- python/2004-April/007536.html
Yes. Double-yes for Deferreds returned from deferToThread. So, first: if you have deferred a function to a thread, there is nothing generic you can do to stop it. If that blocking function specifically provides an API to stop it from other threads, then you can stop it. Second, setTimeout was *never* a good idea. It's been deprecated for long enough that it may be removed at any time; and probably will be, soon. As it is implemented, you will pretty much always see a traceback if the timeout actually fires. Unless the API is specifically written to accommodate this style of timeouts (and none in Twisted are) the API which is expecting to callback or errback a Deferred will instead encounter an `AlreadyCalledError`, since the timeout has invoked it already. Your issue with CLOSE_WAIT sockets could be due to registering producers which have no further data to produce. Does that sound like something you, or starpy, might be doing?

On Thu, 2008-06-05 at 17:25 +0900, Josef Novak wrote:
Hi, I've been digging around a bit more and it seems like it might be advisable for me to use the d.setTimeout(TIMEOUT) function for my deferToThread objects.
What code are you running in the thread, BTW?
participants (3)
-
glyph@divmod.com
-
Itamar Shtull-Trauring
-
Josef Novak