On Wed, Oct 24, 2012, at 09:16, Glyph wrote:
On Oct 23, 2012, at 8:10 AM, Peter Westlake <peter.westlake@pobox.com> wrote:
...
The problem I'm having is that flatten() returns immediately if given a string or anything else without an unfired Deferred, and that sends Client._continue into an unbounded recursion. Is there a general good way to handle this kind of problem? Somehow I need to return control to the reactor long enough for Client._request to return.
That sounds like a bug, although it's hard to say without seeing the exact code that you're talking about. Can you send a representative example?
Here it is: from benchlib import driver, Client from twisted.web.template import flatten from twisted.web.server import Request from twisted.web.http import HTTPChannel class Client(Client): channel = HTTPChannel() request = Request(channel, False) def _request(self): d = flatten(self.request, 'hello', lambda _: None) d.addCallback(self._continue) ### Infinite recursion happens here d.addErrback(self._stop) def main(reactor, duration): concurrency = 1 client = Client(reactor) d = client.run(concurrency, duration) return d if __name__ == '__main__': import sys import flatten_string driver(flatten_string.main, sys.argv) Because flatten does not have to wait for anything, it returns a Deferred that has already fired. The d.addCallback sees this and calls the callback immediately. Client._continue calls the next iteration of the test by calling self.request again, and the stack blows up. This is perfectly reasonable and standard behaviour for Deferreds, so I should be doing the iteration in some other way, probably not using Client at all. What I was hoping for was a pattern for how to transform the code to avoid the problem; I suspect the answer is to use iteration instead of recursion. It might even be that none of benchlib.py is usable directly. Or maybe putting the flatten() calls into a thread would work? But that runs the risk of race conditions, if it finishes before the callback is added. Peter.