[Twisted-Python] Catching error in protocol.ReconnectingClientFactory
Hi there, I've been looking around for a solution for several hours now, and maybe I'm just missing something.. I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ... Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) I really don't care what the solution is, I already have the following method that logs everything for me def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) just annoying that it's still finds the need to print it out at the end. Anyway, I don't claim to be anywhere near proficient in Twisted, so any help would be greatly appreciated. Thanks a lot in advance. Martin
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..)
These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python. I can't say more without a code example, preferably a http://sscce.org/. -glyph
Hi, thanks for your reply. Indeed, the ReconnectingClientFactory is being sent the Failure object to def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason) where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled? Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d? In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run() Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Thu, Jun 14, 2012 at 7:38 PM, Martin
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably a<http://sscce.org/ .
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hi Martin d refers to a defer object.
On 14/06/12 12:38, Martin wrote:
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
No suggestions as to how I could get rid of this? :(
On 25/06/2012, at 10:16 PM, Martin
On 14/06/12 12:38, Martin wrote:
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
No suggestions as to how I could get rid of this? :(
I believe glyph gave you the answer to that already. In your example above of d.addErrBack, d is the deferred object. Each of those needs to have an errback handler attached to it. If you don't want to see anything out of them, simply don't print anything.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 25/06/12 22:27, Adrian Overbury wrote:
On 25/06/2012, at 10:16 PM, Martin
wrote: On 14/06/12 12:38, Martin wrote:
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
No suggestions as to how I could get rid of this? :(
I believe glyph gave you the answer to that already. In your example above of d.addErrBack, d is the deferred object. Each of those needs to have an errback handler attached to it. If you don't want to see anything out of them, simply don't print anything.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Well, kind of - I know that I need to do addErrBack on the deferred objects, I kind of knew that when I asked the original question having read the documentation on deferreds. What I still don't know though is how am I supposed to catch these deferreds - that is not being explained anywhere in the documentation I'm afraid.
On 26/06/2012, at 7:47 AM, Martin
On 25/06/12 22:27, Adrian Overbury wrote:
On 25/06/2012, at 10:16 PM, Martin
wrote: On 14/06/12 12:38, Martin wrote:
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so that my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
No suggestions as to how I could get rid of this? :(
I believe glyph gave you the answer to that already. In your example above of d.addErrBack, d is the deferred object. Each of those needs to have an errback handler attached to it. If you don't want to see anything out of them, simply don't print anything.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Well, kind of - I know that I need to do addErrBack on the deferred objects, I kind of knew that when I asked the original question having read the documentation on deferreds. What I still don't know though is how am I supposed to catch these deferreds - that is not being explained anywhere in the documentation I'm afraid.
The page and bookmark http://twistedmatrix.com/documents/current/core/howto/defer.html#auto4 explains it pretty well. When there's an error your errback function is called and a failure object is passed to it. You can handle it, drop it or pass it on as you please or seem appropriate. Hope that helps.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Tue, Jun 26, 2012 at 5:47 AM, Martin
On 25/06/2012, at 10:16 PM, Martin
wrote: On 14/06/12 12:38, Martin wrote:
Hi, thanks for your reply.
Indeed, the ReconnectingClientFactory is being sent the Failure object to
def clientConnectionLost(self, connector, reason): log.msg('Lost connection. Reason: %s' % reason) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
where "reason" is the Failure object that is "unhandled". is there anything I could do to it there to make it handled?
Also, the documentation in http://twistedmatrix.com/documents/current/core/howto/defer.html says that one can do d.addErrback(err.log) to deal with unhandled errors, but what is d?
In general, I'm not doing anything special.. I'm creating a ReconnectingClientFactory and connecting it to a server.. f = MyReconnectingClientFactory(..) reactor.connectSSL(url, port, f, ssl.ClientContextFactory()) reactor.run()
Basically it. Any thoughts? Thanks a lot!
On Jun 11, 2012, at 8:01 AM, Martin wrote:
Hi there,
I've been looking around for a solution for several hours now, and maybe I'm just missing something..
I've a factory that implements protocol.ReconnectingClientFactory which works just fine, however, whenever the program ends running, I get a bunch of Unhandled error in Deferred messages in my log for each disconnection that occurred during runtime, such as Failure: twisted.internet.error.ConnectionLost: ...
Is there any way I could somehow "handle" these during runtime, so
On 25/06/12 22:27, Adrian Overbury wrote: that
my log wouldn't be spammed with this at the end.. (or avoid it being spammed somehow..) These messages are coming from some code which is issuing Deferreds, not from ReconnectingClientFactory. You need to add an errback to those Deferreds in your application code. There's no way to write a catch-all, just like there's no way to write a catch-all that can stop exceptions from being raised in basic Python.
I can't say more without a code example, preferably ahttp://sscce.org/.
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
No suggestions as to how I could get rid of this? :(
I believe glyph gave you the answer to that already. In your example above of d.addErrBack, d is the deferred object. Each of those needs to have an errback handler attached to it. If you don't want to see anything out of them, simply don't print anything.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Well, kind of - I know that I need to do addErrBack on the deferred objects, I kind of knew that when I asked the original question having read the documentation on deferreds. What I still don't know though is how am I supposed to catch these deferreds - that is not being explained anywhere in the documentation I'm afraid.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hi Usually you don't need to catch a deferred object..adding some callbacks is enough.. I admit it might not be intuitive to understand defer concept at your first trial. But it is powerful & elegant. Regards gelin yan
On 06/25/2012 10:47 PM, Martin wrote:
Well, kind of - I know that I need to do addErrBack on the deferred objects, I kind of knew that when I asked the original question having read the documentation on deferreds. What I still don't know though is how am I supposed to catch these deferreds - that is not being explained anywhere in the documentation I'm afraid.
What do you mean by "catch these deferreds"? A deferred is just a return value of a function, either in the Twisted API or your own code. For example: from twisted.web import client def ignore_err(failure): pass def work(): a_deferred = client.getPage(...) a_deferred.addErrback(ignore_err) If you are seeing unhandled errors in deferreds and you don't know which deferreds, you're probably throwing away a return value somewhere. The traceback should point you in the right directon.
On 25 Jun, 12:16 pm, martin@webscio.net wrote:
No suggestions as to how I could get rid of this? :(
I put together a minimal example of ReconnectingClientFactory for the
connection refused case for you. It's attached. I don't see any errors
logged. If your program differs from this, please modify the example
until it produces the problem. Without this, it's very unlikely that
anyone will be able to determine the problem or recommend a solution.
For future questions, if you could put together this minimal example
yourself, you'll probably find you get help more quickly.
Here's the output I see when I run the program:
2012-06-26 10:07:26-0400 [-] Log opened.
2012-06-26 10:07:26-0400 [-] Starting factory
On 26/06/12 15:12, exarkun@twistedmatrix.com wrote:
On 25 Jun, 12:16 pm, martin@webscio.net wrote:
No suggestions as to how I could get rid of this? :(
I put together a minimal example of ReconnectingClientFactory for the connection refused case for you. It's attached. I don't see any errors logged. If your program differs from this, please modify the example until it produces the problem. Without this, it's very unlikely that anyone will be able to determine the problem or recommend a solution.
For future questions, if you could put together this minimal example yourself, you'll probably find you get help more quickly.
Here's the output I see when I run the program:
2012-06-26 10:07:26-0400 [-] Log opened. 2012-06-26 10:07:26-0400 [-] Starting factory
2012-06-26 10:07:26-0400 [Uninitialized] will retry in 2 seconds 2012-06-26 10:07:26-0400 [Uninitialized] Stopping factory 2012-06-26 10:07:28-0400 [-] Starting factory 2012-06-26 10:07:28-0400 [Uninitialized] will retry in 4 seconds 2012-06-26 10:07:28-0400 [Uninitialized] Stopping factory ^C2012-06-26 10:07:30-0400 [-] Received SIGINT, shutting down. 2012-06-26 10:07:30-0400 [-] Main loop terminated. Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thanks a lot for the suggestions! I tried plugging in a fake address (like you did) in my code and indeed I'm not getting the deferred message at the end. So apparently it only happens when a connection is actually established first and then lost. I'll try to figure out a minimal example where/why it starts happening (I'll probably know how to fix it then too.. I just thought there'd be an easier way).
On 28/06/12 12:46, Martin wrote:
On 26/06/12 15:12, exarkun@twistedmatrix.com wrote:
On 25 Jun, 12:16 pm, martin@webscio.net wrote:
No suggestions as to how I could get rid of this? :(
I put together a minimal example of ReconnectingClientFactory for the connection refused case for you. It's attached. I don't see any errors logged. If your program differs from this, please modify the example until it produces the problem. Without this, it's very unlikely that anyone will be able to determine the problem or recommend a solution.
For future questions, if you could put together this minimal example yourself, you'll probably find you get help more quickly.
Here's the output I see when I run the program:
2012-06-26 10:07:26-0400 [-] Log opened. 2012-06-26 10:07:26-0400 [-] Starting factory
2012-06-26 10:07:26-0400 [Uninitialized] will retry in 2 seconds 2012-06-26 10:07:26-0400 [Uninitialized] Stopping factory 2012-06-26 10:07:28-0400 [-] Starting factory 2012-06-26 10:07:28-0400 [Uninitialized] will retry in 4 seconds 2012-06-26 10:07:28-0400 [Uninitialized] Stopping factory ^C2012-06-26 10:07:30-0400 [-] Received SIGINT, shutting down. 2012-06-26 10:07:30-0400 [-] Main loop terminated. Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thanks a lot for the suggestions! I tried plugging in a fake address (like you did) in my code and indeed I'm not getting the deferred message at the end. So apparently it only happens when a connection is actually established first and then lost. I'll try to figure out a minimal example where/why it starts happening (I'll probably know how to fix it then too.. I just thought there'd be an easier way).
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
And found it.. the ReconnectingClientFactory was using an instance of a LengthDelimitedStream as the protocol and that's where the deferred was coming from. Sorry for all the nagging, I should have figured it out myself.. but tbh getting the hang of Twisted is not so easy :) Cheers!
participants (6)
-
Adrian Overbury
-
exarkun@twistedmatrix.com
-
gelin yan
-
Glyph
-
Martin
-
Phil Mayers