I am doing load testing on my twisted SSL server. I spawn threads(~2000) from the application and each thread will act as a SSL client and connect to the server.
Client code is straight forward. It creates a socket, does SSL wrap and connects to the server. Once all threads connect to the server, every thread closes its socket.
During my initial tests, I found that after 245 successful connections, all remaining threads started to fail with this error on the client side.
SSLError: [Errno 8] _ssl.c:503: EOF occurred in violation of protocol
Then I figured out that its an issue with number of open files which was by default - 256. I changed the limit for server and client to 10000. It successfully completed the 500 connections test.
Now it fails after 1010 successful connections. I checked the number of open files of the server process it remains at 1010 and there are no stale open files.
Client shows the same error
SSLError: [Errno 8] _ssl.c:503: EOF occurred in violation of protocol
Twisted logs on the server shows that it connects first and then disconnects with this error.
<127.0.0.1:55249> lost connection: [('SSL routines', 'SSL23_READ', 'ssl handshake failure')]
How can I fix this ? How should I approach to debug ?