[Twisted-Python] If lineReceived returns deferred, then connection is dropped and this deferred given as error message
I am observing the twisted behaviour which seems rather strange to me. In short: if lineReceived returns some deferred, then the whole connection is aborted, moreover, this deferred is returned as error message. Longer story (the code pieces extracted from quite large app, I have not yet tried writing small example) ### Crucial code fragments class SomeProtocol(basic.LineReceiver): (...) def lineReceived(self, line): (...) return defer.succeed(997) # magic value I can recognize easily class SomeFactory(protocol.ReconnectingClientFactory): (...) def clientConnectionLost(self, connector, reason): logger.warn("Connection lost. Reason: %s\nTraceback: %s" % ( reason.getErrorMessage(), reason.getTraceback())) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason) reactor.connectTCP(HOST, PORT, SomeFactory(...)) reactor.run() ### Things seen in log file myapp : WARNING Connection lost. Reason: <Deferred at 0xB551E1ECL current result: 997> Traceback: Traceback (most recent call last): Failure: <type 'instance'>: <Deferred at 0xB551E1ECL current result: 997> (you recognized this 997, didn't you?) ### Extra info 1. Removing return defer.succeed() from lineReceived helps. 2. I wanted to return deferreds from lineReceived because I wanted to write unit tests testing my factory object (and in some cases to check whether all worked correctly required waiting for some spawned deferred calls).
One more observation after looking at twisted code: twisted/protocols/basic.py funtion LineReceiver.dataReceived contains sth like that: why = self.lineReceived(line) if why or self.transport and self.transport.disconnecting: return why As this is WITHIN the loop, we see why returning deferred (or anything true) from lineReceived breaks processing. I did not search further but it seems somebody is using convention that if lineReceived (and maybe dataReceived) returns something, then this is an error. Why not exception??? And, is it possible to patch it somehow so lineReceived function could be unit-tested?
On 10/31/06, Marcin Kasperski <Marcin.Kasperski@softax.com.pl> wrote:
One more observation after looking at twisted code:
twisted/protocols/basic.py funtion LineReceiver.dataReceived contains sth like that:
why = self.lineReceived(line) if why or self.transport and self.transport.disconnecting: return why
As this is WITHIN the loop, we see why returning deferred (or anything true) from lineReceived breaks processing.
I did not search further but it seems somebody is using convention that if lineReceived (and maybe dataReceived) returns something, then this is an error. Why not exception??? And, is it possible to patch it somehow so lineReceived function could be unit-tested?
While Twisted's handling of the result of lineReceived is perhaps confusing, I find your reasons for wanting it changed dubious: your unit tests should be testing that lineReceived has the appropriate *effects*, not that arbitrary points in the code are reached by testing its return value (which would have no effect in the actual running of your program). -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/
While Twisted's handling of the result of lineReceived is perhaps confusing, I find your reasons for wanting it changed dubious: your unit tests should be testing that lineReceived has the appropriate *effects*,
And this is exactly what do I want: lineReceived should cause some effect (in my case, for instance some database update). So I call lineReceived with the intended string, and want - in unit test - to verify whether the update was actually made (it is done by deferred subroutine). But, to check for it, I must wait for it somehow....
And, is it possible to patch it somehow so lineReceived function could be unit-tested?
After short reflection I found the solution def myLineReceived(self, line): # ... # can return deferred def lineReceived(self, line): self.myLineReceived(line) and unit tests just use myLineReceived. A bit clumsy, but works. The main question (why lineReceived breaks connection on true reply) of course remains valid.
As it seems nobody has anything to add, I filed a bug http://twistedmatrix.com/trac/ticket/2215
participants (2)
-
Christopher Armstrong -
Marcin Kasperski