![](https://secure.gravatar.com/avatar/7304771f4f4f40784b90d6d4f0855654.jpg?s=120&d=mm&r=g)
I've run into another problem in my application that uses the dreid/ web2-client branch. This one I'm not sure how to solve, or even kludge around for the moment. My application makes a bunch of HTTP requests in serial. It needs to see the previous response to make the next request. When I handle a HTTP response, I originally wrote it like this (simplified): def _makeRequest(self, ...): req = client.Request(...) req.addCallback(self._handleResponse) def _handleResponse(self, response): stream.readStream(response.stream, self._handleResponseBody) def _handleResponseBody(self, body): self._makeRequest(...) This worked fine - proper HTTP Keep-Alives and whatnot - until I got a large response back. Then I realized that stream.readStream gives me chunks and I should instead be using this (if I want to handle the whole response at once): def _handleResponse(self, response): self.body = '' stream.readStream(response.stream, self._handleResponseChunk ).addCallback(self._handleResponseBody) def _handleResponseChunk(self, chunk): self.body += chunk def _handleResponseBody(self): self._makeRequest(...) self.body = None This works with the chunking, but I am now making an HTTP connection for each request. HTTPClient._endResponse() calls self.response.stream.finish() and then tears down the connection if no request is pending. But stream.finish() doesn't call my deferred right away; it happens later from the reactor. So though I want to make another request, I haven't had my chance yet. My connection is torn down, then I make a new one and do an expensive SSL handshake, which is all wasted round trips, bandwidth, and CPU time. I can think of two ways to solve this: (1) make stream.ProducerStream.finish() call _handleResponseBody right away. But there's a "reactor.callLater(0, self._read)" in stream._StreamReader that looks to be there for a reason. I haven't thought it through, but I suspect there are unsolvable recursion problems with this approach. (2) make HTTPClient not check if the connection should be torn down until after my deferred has had a chance to run. Would just a "reactor.callLater(0, self._endResponse2)" be enough? Are callLaters guaranteed to be run in order? -- Scott Lamb <http://www.slamb.org/>
![](https://secure.gravatar.com/avatar/7304771f4f4f40784b90d6d4f0855654.jpg?s=120&d=mm&r=g)
On 21 Oct 2005, at 21:51, Scott Lamb wrote:
(2) make HTTPClient not check if the connection should be torn down until after my deferred has had a chance to run. Would just a "reactor.callLater(0, self._endResponse2)" be enough? Are callLaters guaranteed to be run in order?
I just tried this and it seems to work. It'd be reassuring to hear that it's guaranteed, though. -- Scott Lamb <http://www.slamb.org/>
participants (1)
-
Scott Lamb