[Twisted-Python] Problem with Deferreds
Hi all, I recently got over myself and forced myself to read about Deferreds rather than using threading opperations. I've used them successfully in a couple of places, but this one has me flummoxed: Here's the code for the Deferred and it's sub-commands: def do_command(self, cmd, **kwargs): """Process a command in true Deferred style.""" if cmd.permissions(self.connection): cmd(self.connection, **kwargs) else: logger.warning('Blocked from running command %s which is secured with %s.', cmd.__name__, cmd.permissions.__name__) raise CommandError('You have insufficient privileges to perform this action. The staff have been notified.') def handle_error(self, err): """Handle an error from do_command.""" if isinstance(err, CommandError): return self.send_error(e.message, disconnect = e.disconnect) else: self.log(e, level = 'exception') self.send_error('Sorry, but a problem with the server means your command was not executed. The staff have been notified.') def lineReceived(self, line): """Parse an incoming command.""" global lines lines += 1 # Increment the line count. data = line.decode(settings.ENCODING) try: command, kwargs = json.loads(data) if not isinstance(command, string_types) or not isinstance(kwargs, dict): raise TypeError('Expecting [str, dict]. Got [%s, %s] instead.' % (type(command), type(kwargs))) except (TypeError, ValueError) as e: self.log('Invalid command string: %s', data, level = 'error') self.log(e, level = 'exception') return self.send_error('Invalid command.', disconnect = True) cmd = commands.commands.get(command, None) if cmd is None: self.log('Unrecognised command: %s.', command, level = 'warning') elif self.connection.player or not cmd.login_required: d = defer.Deferred() print('Adding callback.') d.addCallback(self.do_command, **kwargs) print('Adding errback.') d.addErrback(self.handle_error) print('Calling callback.') d.callback(cmd) print('Called.') # Never gets this far. return d else: return self.send_error('Not authenticated.', disconnect = True) Here's the traceback I get when the callback gets called: Unhandled Error Traceback (most recent call last): File "server/functions.py", line 88, in server_start reactor.run() File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/internet/base.py", line 1194, in run self.mainLoop() File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/internet/base.py", line 1206, in mainLoop self.doIteration(t) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/internet/epollreactor.py", line 396, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/python/log.py", line 101, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/python/log.py", line 84, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/internet/posixbase.py", line 610, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/internet/posixbase.py", line 258, in _disconnectSelectable selectable.connectionLost(failure.Failure(why)) File "/usr/local/lib/python3.5/site-packages/Twisted-16.0.0-py3.5.egg/twisted/python/failure.py", line 232, in __init__ tb = self.value.__traceback__ builtins.AttributeError: 'Deferred' object has no attribute '__traceback__' Anyone have any ideas? Cheers, Chris
Hi Chris, tl;dr: Returning a value from 'dataReceived', or any of its extensions such as 'lineReceived' in the 'LineReceiver' Protocol subclass, triggers a disconnect and uses the returned value as the 'reason'. A 'reason' must be an Exception or t.p.Failure object as other values will trigger this error. Are you quite certain that your last line is not getting printed? I'm not sure exactly where this feature is documented, but returning any non-None value from a Protocol's 'dataReceived' method can result in this behaviour. The t.protocols.basic.LineReceiver calls 'lineReceived' from 'dataReceived' and returns any value it gets from your implementation. The value returned from 'dataReceived' is passed along to the transport's 'doRead' which, again, returns it to the portion of the reactor handling selectables. The reactor assumes that anything returned from a transport during a read or write operation is a bad thing and disconnects the transport. During the disconnect process the reactor is generating a t.p.failure.Failure object and passing in your returned value as the 'why' which is expected to be an Exception or Failure and not a Deferred. Try returning None instead of your Deferred. That should resolve this particular issue. On Sat, Apr 2, 2016 at 4:39 AM Chris Norman <chris.norman2@googlemail.com> wrote:
Hi Chris, tl;dr: Returning a value from 'dataReceived', or any of its extensions such as 'lineReceived' in the 'LineReceiver' Protocol subclass, triggers a disconnect and uses the returned value as the 'reason'. A 'reason' must be an Exception or t.p.Failure object as other values will trigger this error. Are you quite certain that your last line is not getting printed? I'm not sure exactly where this feature is documented, but returning any non-None value from a Protocol's 'dataReceived' method can result in this behaviour. The t.protocols.basic.LineReceiver calls 'lineReceived' from 'dataReceived' and returns any value it gets from your implementation. The value returned from 'dataReceived' is passed along to the transport's 'doRead' which, again, returns it to the portion of the reactor handling selectables. The reactor assumes that anything returned from a transport during a read or write operation is a bad thing and disconnects the transport. During the disconnect process the reactor is generating a t.p.failure.Failure object and passing in your returned value as the 'why' which is expected to be an Exception or Failure and not a Deferred. Try returning None instead of your Deferred. That should resolve this particular issue. On Sat, Apr 2, 2016 at 4:39 AM Chris Norman <chris.norman2@googlemail.com> wrote:
participants (2)
-
Chris Norman -
Kevin Conway