About handling thousands of sockets connections simultaneously
i use twisted as TCP server, when handling thousands of sockets connections simultaneously, i have found that the maximum TCP connections limited about 500. What's the matter? some people say it is because that the number of a process opening file descriptors is limited,is't right? how can i solve the problem? Arvin
On 09:25 am, hustxnwsg@gmail.com wrote:
i use twisted as TCP server, when handling thousands of sockets connections simultaneously, i have found that the maximum TCP connections limited about 500. What's the matter?
Try switching to a different reactor. For example, the select reactor is limited to FD_SETSIZE open sockets. This is usually around 1000, but could be 500 on some systems. Poll reactor is limited to the "open files" rlimit which can be raised by superusers. IOCP reactor has no arbitrary limits, I think. http://twistedmatrix.com/documents/current/core/howto/choosing- reactor.html Jean-Paul
thank you for your advice. i have chose the epoll reactor,and the requirement of my project is to be able to support 60,000 TCP-connects simultaneously. it seems that just choosing different reactor couldn't solve the problem, can you give some more suggestions? On Tue, Sep 7, 2010 at 8:50 PM, <exarkun@twistedmatrix.com> wrote:
On 09:25 am, hustxnwsg@gmail.com wrote:
i use twisted as TCP server, when handling thousands of sockets connections simultaneously, i have found that the maximum TCP connections limited about 500. What's the matter?
Try switching to a different reactor. For example, the select reactor is limited to FD_SETSIZE open sockets. This is usually around 1000, but could be 500 on some systems. Poll reactor is limited to the "open files" rlimit which can be raised by superusers. IOCP reactor has no arbitrary limits, I think.
http://twistedmatrix.com/documents/current/core/howto/choosing- reactor.html
Jean-Paul
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
-- Services Computing Technology and System Lab && Cluster and Grid Computing Lab. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan,430074,China.
On Tue, Sep 7, 2010 at 3:35 PM, Arvin Wang <hustxnwsg@gmail.com> wrote:
thank you for your advice. i have chose the epoll reactor,and the requirement of my project is to be able to support 60,000 TCP-connects simultaneously. it seems that just choosing different reactor couldn't solve the problem, can you give some more suggestions?
60000? I'm very curious to know how it's possible to achieve this with twisted. http://www.kegel.com/c10k.html -- Cristiano GPG Key: 4096R/C17E53C6 2010-02-22 Fingerprint = 4575 4FB5 DC8E 7641 D3D8 8EBE DF59 B4E9 C17E 53C6
On 01:35 pm, hustxnwsg@gmail.com wrote:
thank you for your advice. i have chose the epoll reactor,and the requirement of my project is to be able to support 60,000 TCP-connects simultaneously. it seems that just choosing different reactor couldn't solve the problem, can you give some more suggestions?
If you are more precise about the problem you're having, perhaps. Jean-Paul
2010/9/7 Arvin Wang <hustxnwsg@gmail.com>:
thank you for your advice. i have chose the epoll reactor,and the requirement of my project is to be able to support 60,000 TCP-connects simultaneously. it seems that just choosing different reactor couldn't solve the problem, can you give some more suggestions?
Discard that you're exhausting ephemeral ports launching a lot of client with same client host. Monitor server process (memory, cpu, logs) to see possible server problems or bottlenecks.
On Sep 7, 2010, at 9:35 AM, Arvin Wang wrote:
it seems that just choosing different reactor couldn't solve the problem, can you give some more suggestions?
Check "ulimit -n": some systems set it to be very low by default. E.g. it's set to 1024 on my system. That means I can't use more than 1024 file descriptors in a process. Check if yours is similarly low. James
participants (5)
-
Arvin Wang -
Cristiano Paris -
exarkun＠twistedmatrix.com -
James Y Knight -
lasizoillo