[Twisted-Python] Is this possible to catch this error? (returned by protocols/memcached.py)
Hi, Here's a sample application, which gets a key from memcached: from twisted.internet import reactor, protocol, defer from twisted.application import internet, service from twisted.protocols.memcache import MemCacheProtocol, DEFAULT_PORT mc = None class MemCacheFactory(protocol.ReconnectingClientFactory): protocol = MemCacheProtocol maxDelay = 5 # wait no more than 5 seconds to retry def buildProtocol(self, addr): p = self.protocol(self) p.factory = self p.persistentTimeOut = p.timeOut = 5 global mc mc = p return p def memc(): factory = MemCacheFactory() reactor.connectTCP('localhost', 11211, factory) reactor.callLater(1,query) @defer.inlineCallbacks def query(): res = yield mc.get('test') print "XXX",res application = service.Application('memc') serviceCollection = service.IServiceCollection(application) reactor.callLater(0,memc) If memcached is running on localhost:11211, it writes the "test" key's value: $ twistd -noy memct.py 2011-06-03 14:48:27+0200 [-] Log opened. 2011-06-03 14:48:27+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:48:27+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:48:27+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:48:28+0200 [MemCacheProtocol,client] XXX (0, None) Now I want to test the code for timeouts, where the TCP connection to memcached is open, but there is no answer. So instead of memcached, I start a netcat on port 11211: $ nc -l 11211 And start the above program. Now I get: $ twistd -noy memct.py 2011-06-03 14:50:08+0200 [-] Log opened. 2011-06-03 14:50:08+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:50:08+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:50:08+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:50:14+0200 [-] Unhandled error in Deferred: 2011-06-03 14:50:14+0200 [-] Unhandled Error Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: Connection timeout 2011-06-03 14:50:14+0200 [MemCacheProtocol,client] <twisted.internet.tcp.Connector instance at 0x804027dd0> will retry in 2 seconds 2011-06-03 14:50:14+0200 [MemCacheProtocol,client] Stopping factory <__builtin__.MemCacheFactory instance at 0x802a38710> As you can see, the connection is timed out, and protocols/memcached.py lost its connection. But how could I catch this around the "mc.get" call? I guess this error should raise an exception, so enclosing mc.get into a try-except would make possible to catch this. But currently this is not the case? How can this be solved? Thanks,
On 12:54 pm, bra@fsn.hu wrote:
Hi,
Here's a sample application, which gets a key from memcached: from twisted.internet import reactor, protocol, defer from twisted.application import internet, service from twisted.protocols.memcache import MemCacheProtocol, DEFAULT_PORT
mc = None
class MemCacheFactory(protocol.ReconnectingClientFactory): protocol = MemCacheProtocol maxDelay = 5 # wait no more than 5 seconds to retry
def buildProtocol(self, addr): p = self.protocol(self) p.factory = self p.persistentTimeOut = p.timeOut = 5 global mc mc = p
Using a global protocol instance like this is error prone.
return p
def memc(): factory = MemCacheFactory() reactor.connectTCP('localhost', 11211, factory) reactor.callLater(1,query)
What if the connection isn't set up within 1 second?
@defer.inlineCallbacks def query(): res = yield mc.get('test') print "XXX",res
application = service.Application('memc') serviceCollection = service.IServiceCollection(application) reactor.callLater(0,memc)
If memcached is running on localhost:11211, it writes the "test" key's value: $ twistd -noy memct.py 2011-06-03 14:48:27+0200 [-] Log opened. 2011-06-03 14:48:27+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:48:27+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:48:27+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:48:28+0200 [MemCacheProtocol,client] XXX (0, None)
Now I want to test the code for timeouts, where the TCP connection to memcached is open, but there is no answer. So instead of memcached, I start a netcat on port 11211: $ nc -l 11211 And start the above program. Now I get: $ twistd -noy memct.py 2011-06-03 14:50:08+0200 [-] Log opened. 2011-06-03 14:50:08+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:50:08+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:50:08+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:50:14+0200 [-] Unhandled error in Deferred: 2011-06-03 14:50:14+0200 [-] Unhandled Error Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: Connection timeout
2011-06-03 14:50:14+0200 [MemCacheProtocol,client] <twisted.internet.tcp.Connector instance at 0x804027dd0> will retry in 2 seconds 2011-06-03 14:50:14+0200 [MemCacheProtocol,client] Stopping factory <__builtin__.MemCacheFactory instance at 0x802a38710>
As you can see, the connection is timed out, and protocols/memcached.py lost its connection. But how could I catch this around the "mc.get" call? I guess this error should raise an exception, so enclosing mc.get into a try-except would make possible to catch this. But currently this is not the case?
What do you think it is not the case? That is how inlineCallbacks makes Failures available to you. Jean-Paul
Hi, On 06/03/11 15:57, exarkun@twistedmatrix.com wrote:
Using a global protocol instance like this is error prone.
return p
def memc(): factory = MemCacheFactory() reactor.connectTCP('localhost', 11211, factory) reactor.callLater(1,query)
What if the connection isn't set up within 1 second?
This is just an example, for demonstrating my problem.
@defer.inlineCallbacks def query(): res = yield mc.get('test') print "XXX",res
application = service.Application('memc') serviceCollection = service.IServiceCollection(application) reactor.callLater(0,memc)
If memcached is running on localhost:11211, it writes the "test" key's value: $ twistd -noy memct.py 2011-06-03 14:48:27+0200 [-] Log opened. 2011-06-03 14:48:27+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:48:27+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:48:27+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:48:28+0200 [MemCacheProtocol,client] XXX (0, None)
Now I want to test the code for timeouts, where the TCP connection to memcached is open, but there is no answer. So instead of memcached, I start a netcat on port 11211: $ nc -l 11211 And start the above program. Now I get: $ twistd -noy memct.py 2011-06-03 14:50:08+0200 [-] Log opened. 2011-06-03 14:50:08+0200 [-] twistd 10.1.0 (/usr/local/bin/python2.7 2.7.1) starting up. 2011-06-03 14:50:08+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 14:50:08+0200 [-] Starting factory <__builtin__.MemCacheFactory instance at 0x802a38710> 2011-06-03 14:50:14+0200 [-] Unhandled error in Deferred: 2011-06-03 14:50:14+0200 [-] Unhandled Error Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: Connection timeout
2011-06-03 14:50:14+0200 [MemCacheProtocol,client] <twisted.internet.tcp.Connector instance at 0x804027dd0> will retry in 2 seconds 2011-06-03 14:50:14+0200 [MemCacheProtocol,client] Stopping factory <__builtin__.MemCacheFactory instance at 0x802a38710>
As you can see, the connection is timed out, and protocols/memcached.py lost its connection. But how could I catch this around the "mc.get" call? I guess this error should raise an exception, so enclosing mc.get into a try-except would make possible to catch this. But currently this is not the case?
What do you think it is not the case? That is how inlineCallbacks makes Failures available to you. Because if I wrap mc.get into a try-except clause, it's not triggered. I guess if it would, the above error would show exact line numbers too.
On 02:04 pm, bra@fsn.hu wrote:
Hi,
On 06/03/11 15:57, exarkun@twistedmatrix.com wrote:
As you can see, the connection is timed out, and protocols/memcached.py lost its connection. But how could I catch this around the "mc.get" call? I guess this error should raise an exception, so enclosing mc.get into a try-except would make possible to catch this. But currently this is not the case?
What do you think it is not the case? That is how inlineCallbacks makes Failures available to you.
Because if I wrap mc.get into a try-except clause, it's not triggered. I guess if it would, the above error would show exact line numbers too.
I can't reproduce this. I changed the code to: @defer.inlineCallbacks def query(): try: res = yield mc.get('test') except Exception, e: print "get failed:", e else: print "XXX",res When I run this against the nc server, the "get failed:" print happens. Jean-Paul
participants (2)
-
Attila Nagy -
exarkun＠twistedmatrix.com