Re: [Twisted-Python] How to debug an AMP connection?

On Fri, 24 Jul 2009 16:12:28 +0100, Peter Westlake wrote:

On Fri, 24 Jul 2009 10:44 -0400, "Drew Smathers" wrote:

...

On Fri, Jul 24, 2009 at 9:35 AM, Peter Westlake wrote:

...

Hello,

I'm having trouble with an AMP connection that doesn't fire the Deferred returned by callRemote. The AMP command copies files from the client (called a "worker" in the code below) to a server ("controller"). It sends a chunk of text at a time - 16K originally, but I tried smaller amounts too. The server appends the text to a queue to be written; it shouldn't take long, because it doesn't do the I/O synchronously, just starts a thread.

"justs starts a thread"? Where are you starting a thread in the code example you've posted?

...

Please post a complete example; I don't see methods on client code below such as send_next() or next_or_retry(). It's hard to help without a complete example.

Sorry - I was trying to avoid clogging everyone's inboxes with too much code. So far I haven't been able to reproduce the bug in a simple way. In the meantime, here's the code that starts the thread. It's one of the very first bits of code I wrote with either Twisted or Python, so it may do things in quite the wrong way. It serialises writes to a file by storing requests in a queue, and it gets called by the line

ac.instances[number].logfile[stream].write(data)

in the command handler.

from twisted.internet import defer, threads import os, errno

class FileMan(object): def __init__(self, filename): self.filename = filename self.queue = [] self.started = False

def check_file(self): self.started = os.path.exists(self.filename) return self.started

def makedir(self): dirname = os.path.dirname(self.filename) if not os.path.isdir(dirname): os.makedirs(dirname)

def add(self, op, *a, **kw): def getnext(result): del(self.queue[0]) if len(self.queue) > 0: self.queue[0].callback('unused') return result

d = defer.Deferred() d.addCallback(op, *a, **kw) d.addBoth(getnext) self.queue.append(d) if len(self.queue) == 1: d.callback('unused') return d

def write(self, data, mode='a'): def do_write(result, filename, data): def write_in_thread(filename, data): f = open(filename, mode) try: f.write(data) finally: f.close() self.started = True d = threads.deferToThread(write_in_thread, filename, data) return d return self.add(do_write, self.filename, data)

...

In general, though, if you're trying to debug Deferreds you might try setting debugging with defer.setDebugging(1).

What kind of output does that produce? I did try it, but didn't see anything out of the ordinary.

It makes Deferreds keep track of the call stack when they are created and when they are invoked. This can be helpful tracking down Deferred failures that don't otherwise identify themselves very well. If you're not seeing any failures being logged, then this probably won't help much, since there won't be anything to increase the verbosity of. :) Another thing to check out would be a network capture, to see if the server is actually sending back a response. If not, then you know you should look at the server code more carefully. If so, then the client code is probably at fault. Jean-Paul