[Twisted-Python] Error: filedescriptor out of range in select

Don Schoeman wrote:
Hi guys, I have started having this problem a few weeks ago and it happens about once a week after which I have to restart my Twisted based server to function again. It seems to be happening when I make RPC calls using twisted.web.xmlrpc.Proxy. I have reason to believe that I am either running out of file handles or connection limits. I have up-sized my connection limits and ulimit -n gives me 9000.
Note though that select() has a builtin limit, which varies by platform but is probably 1024 for you. So perhaps try the poll or epoll reactor instead.
I receive less than 30 connections though so there must be some kind of leak. The error I'm getting is the following:
30 is much less than 1024, though, so a leak does sound probable. [...]
Now I know this is a very generic error and it could mean a lot of things, but how would I even start tracking the leak down? Is there a way I can try and track the number of file descriptors?
There's always strace, or looking in /proc/PID/fd.
I am using Twisted version 8.2.0 on Ubuntu Server Edition 9.10
Maybe try upgrading Twisted, preferably to 10.0.0? I'm not sure if a bug related to your problems was fixed since 8.2.0, but a lot of bugs have been fixed in that ~2 years. Hopefully there's a PPA somewhere with a newer Twisted for your version of Ubuntu. -Andrew.

I think you will find that gprs devices will drop off the network but the router will still keep the coonection open. The gprs device will then reconnect on a different connection and port. So you will need to clean up old connections used by that device. Hope that helps John Aherne On Fri, Mar 19, 2010 at 2:53 PM, Don Schoeman <don@delphexonline.com> wrote:
Hi Andrew, thanks for taking the time to comment on my email.
I'm going to have to study your proposal to switch to poll/epoll anyway since I am expecting more and more connections to be made over time. This particular application is used to communicate with a bunch of GPRS devices and I'm starting to think that these devices might not be closing a connection properly which could perhaps lead to this particular problem. I am going to discuss it with the manufacturers of these devices as well.
Thanks again for your help.
Kind Regards, Don
Andrew Bennetts wrote:
Don Schoeman wrote:
Hi guys, I have started having this problem a few weeks ago and it happens about once a week after which I have to restart my Twisted based server to function again. It seems to be happening when I make RPC calls using twisted.web.xmlrpc.Proxy. I have reason to believe that I am either running out of file handles or connection limits. I have up-sized my connection limits and ulimit -n gives me 9000.
Note though that select() has a builtin limit, which varies by platform but is probably 1024 for you. So perhaps try the poll or epoll reactor instead.
I receive less than 30 connections though so there must be some kind of leak. The error I'm getting is the following:
30 is much less than 1024, though, so a leak does sound probable.
[...]
Now I know this is a very generic error and it could mean a lot of things, but how would I even start tracking the leak down? Is there a way I can try and track the number of file descriptors?
There's always strace, or looking in /proc/PID/fd.
I am using Twisted version 8.2.0 on Ubuntu Server Edition 9.10
Maybe try upgrading Twisted, preferably to 10.0.0? I'm not sure if a bug related to your problems was fixed since 8.2.0, but a lot of bugs have been fixed in that ~2 years. Hopefully there's a PPA somewhere with a newer Twisted for your version of Ubuntu.
-Andrew.
_______________________________________________ Twisted-Python mailing listTwisted-Python@twistedmatrix.comhttp://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (3)
-
Andrew Bennetts
-
Don Schoeman
-
John Aherne