Scenario: TCP Receive Buffer on twisted HTTP server using the twisted application framework. And its behavior when set up as producer/consumer.
(1) Would like to set the receive buffer size on socket. One way to do this would be to create a derived class of TCPServer (or SSLServer) and set the buffer size and use derived class server in the .tac file. Would like to know if there is any sample code for such usage? For example, in which method of the derived class would one set the receive buffer size?
You don't need to subclass these classes; and in any event, it wouldn't help, TCPServer and SSLServer don't have connections of their own, they're services that hold listening sockets, not connected sockets. If you want to set an option unsupported by Twisted, "transport.getHandle()" will give you the Python socket object (on those reactors which use socket objects internally, which is most of them). You can just set SO_RCVBUF on that socket from your Protocol class.
(2) When an incoming http POST request is acting as a producer of data, which is tied to a consumer resource (some other connection), how can I control the incoming tcp window size, if the consumer has paused consuming? I presume the incoming network data will keep piling up in the 'huge' tcp buffer eventually advertising a 'tcp zero window' to the network peer of the data producer, AND the server ends up using up a large amount of memory for the paused connection. Is there an alternative? I realize that the TCP protocol inhibits 'reducing of an already advertised receive window', but I am wondering if pauseProducing() on an http channel could do something to at least prevent the tcp window size from increasing any further?
I don't know that much about window scaling, but won't the fact that the application isn't receiving any data from the kernel automatically prevent the window from scaling any bigger?