I'm writing a simple JSONRPC server that is intended to have a number of
longpolling http connections.
-On the server side, I'm using txJSON-RPC and utilizing returning deferred
(which translate into NOT_DONE_YET at the next layer).
-On a clientside, I'm using JQuery to kick off a longpolling ajax POST to
the rpc interface.
Upon the requests arrival to the RPC server I returned a Deferred, adding
it to my list. The POST should only return data (finish the request) once
the server side has some changed state. It works for basic tests.
However, once I increase the number of outstanding connections, I don't see
any more connections accepted from my browser. I seem to be capped at 6
open connections. If I start a new browser, ie: chrome, or even change the
URL from localhost to 127.0.0.1 I'm able to open another 6.
I've never done much server programming so I'm a bit confused as to where
to look to fix this. I've import pdb and set_trace() all the way down to
the tcp.py part and I still don't see any activity coming in.
What steps should I take to identify and fix this limitation? Any help
would be appreciated!
Thanks, Laurens. I will have a try with your advices.
Date: Thu, 21 Mar 2013 07:32:18 -0700
From: Laurens Van Houtven <_(a)lvh.cc>
Subject: Re: [Twisted-web] A Question, TIA!
To: Twisted Web World <twisted-web(a)twistedmatrix.com>
Content-Type: text/plain; charset="utf-8"
Sounds like you want twisted.web.client.Agent; when you make a request you
get an IResponse object that has a deliverBody method that takes a
protocol. That protocol can receive chunks (dataReceived), or you could use
e.g. a LineReceiver if you know they're delimited chunks.
My program(Cumulus) receives http put(file by chunk).And then resend the file to another process(Hadoop).
The trouble is that i need get the file object in my program(not stored on disk then read from).
So where should i get it? rawDataReceived or allContentReceived?
Scenario: TCP Receive Buffer on twisted HTTP server using the twisted application framework. And its behavior when set up as producer/consumer.
(1) Would like to set the receive buffer size on socket. One way to do this would be to create a derived class of TCPServer (or SSLServer) and set the buffer size and use derived class server in the .tac file. Would like to know if there is any sample code for such usage? For example, in which method of the derived class would one set the receive buffer size?
(2) When an incoming http POST request is acting as a producer of data, which is tied to a consumer resource (some other connection), how can I control the incoming tcp window size, if the consumer has paused consuming? I presume the incoming network data will keep piling up in the 'huge' tcp buffer eventually advertising a 'tcp zero window' to the network peer of the data producer, AND the server ends up using up a large amount of memory for the paused connection. Is there an alternative? I realize that the TCP protocol inhibits 'reducing of an already advertised receive window', but I am wondering if pauseProducing() on an http channel could do something to at least prevent the tcp window size from increasing any further?