
"Jason J. W. Williams" <jasonjwwilliams@gmail.com> writes:
Actually, I think the TIME_WAIT is the problem. It's what I see in netstat, and the Agent requests are fired sequentially via yield inside a for loop (inlineCallbacks). So they shouldn't be running in parallel.
`yield` returns before TIME_WAIT expires otherwise it would require ~1 minute per request.
The use case here is loading a Riak server with keys to prepare for a test. There's not a real way to get around sending one POST per key.
How would I set the timeout value in Twisted? Or do I have to modify the timeout/keepalive systemwide in /proc?
In addition to net.ipv4.tcp_fin_timeout you could increase the ephemeral port range (net.ipv4.ip_local_port_range sysctl parameter). Each connection can be identified using 4-tuple (server IP, server port, client IP, client port) Everything except client port is fixed in your case so there could be at most ~ net.ipv4.ip_local_port_range/net.ipv4.tcp_fin_timeout connections per second (even less in practice due to other applications and other settings taking preference such as fs.file-max). For example: net.ipv4.ip_local_port_range = 32768 61000 net.ipv4.tcp_fin_timeout = 30 There could be ~900 connections per second that might be good enough. Reusing a local port via SO_REUSEADDR or better yet reusing a tcp connection via HTTP keep-alive aren't available with twisted as I understand it. -- akira