[Twisted-Python] (no subject)
Hi Frank, Best that I can tell, it's not your various classes that need to be converted to deferreds, just your program that calls class1, or even class1 itself. The bottom line is that _something_ higher up the calling stack needs to be aware that this is an asynchronous process - and that a deferred gets generated along the way. I believe it would be helpful if the originating function is coded for this. I'm not a Twisted guru, so there may be an easier way to do this - but as I see it... I would be writing the classes to look more like the following: class1(): def check1(data): perform test if test failed: return defer.succeed(False) return class2.check2(data): class2(): def check2(data): perform test if test failed: return defer.succeed(False) return class3.check3(data): class3(): def check3(data): perform test if test failed: return defer.succeed(False) aDeferred = avatar.callRemote(check4,data) return aDeferred Then, my originating class (and possibly becomming a wrapper class if this is used in many places). class0(): def check0(data): class1.check1(data).addCallback(afterCheck) def afterCheck(result): return result This way, all the functions return a deferred back up to the calling program - but the only one that causes a delay is the call to check4. Ken
Ken wrote:
Hi Frank,
Best that I can tell, it's not your various classes that need to be converted to deferreds, just your program that calls class1, or even class1 itself.
Thanks for the reply.
The bottom line is that _something_ higher up the calling stack needs to be aware that this is an asynchronous process - and that a deferred gets generated along the way. I believe it would be helpful if the originating function is coded for this.
I understand what you are getting at, but I have spent a few hours trying to squeeze this concept into my program, and I am not succeeding. There are two main reasons that I can see. Firstly, I oversimplified my pseudo code. You suggested changing this class1(): def check1(data): perform test if test failed: return False if not class2.check2(data): return False return True to this class1(): def check1(data): perform test if test failed: return defer.succeed(False) return class2.check2(data) I understand that, but I should have used this as an example - class1(): def check1(data): perform test if test failed: return False if not class2.check2(data): return False do some extra stuff here # this is what I left out return True This 'extra stuff' can occur at any of the levels in the stack. I cannot see where I could put this using your version. Secondly, under some circumstances I loop over a number of objects and validate them one after the other - class1(): def check1(data): for obj in objList: if not check2(obj): return False perform test if test failed: return False if not check2(data): return False do some extra stuff here return True As you can see, it is getting even more complicated to do this using deferreds. The irony is that I actually want the client program to block until a response is received, so maybe I should not be using Twisted on the client at all. I could write a normal socket client, and use a ServerFactory on the server. I am very new to Twisted, so it is likely that my problem is that I have not learned to think in a Twisted manner yet. Any advice to put me straight will be much appreciated. Thanks Frank
Hi Ken Many thanks for your reply. Sorry for not replying earlier - I seem to have lost some mail due to ISP problems, and I just noticed your message in the archive. I have not included your response, as I want to come at this from another angle. I confess that I am under pressure to 'get something working', even if it is not the optimal solution, and this is all getting a bit complicated. I have been experimenting with using a socket client instead of Twisted, and I got something up and running quite quickly. When I take a step back and look at how I got it working, I feel that I should be able to apply the same technique to Twisted, but I cannot get it to work. I will show both methods, and perhaps someone can point me in the right direction. The trick is that I have two threads running concurrently - a wxPython main loop, and a subthread that monitors the socket. The socket loop looks like this (simplified), subclassed from threading.Thread - def run(self): readable = [s.fileno()] error = [] self.sendData = [] while 1: if self.sendData: writable = [s.fileno()] else: writable = [] r,w,e = select.select(readable,writable,error,0.01) if r: self.recvData = s.recv(1024) if w: s.send(self.sendData.pop(0)) def checkData(self,item,value): # this is called from the *wx* thread self.recvData = None self.sendData.append(cPickle.dumps((CHECK,item,value))) while self.recvData is None: sleep(0.01) return self.recvData As you can see, checkData() blocks the wxPython thread until it receives a response from the socket thread. This is how I tried with Twisted - def checkData(self,item,value): # this is called from the *wx* thread self.recvData = None self.callCheckData(item,value) while self.recvData is None: sleep(0.01) return self.recvData def callCheckData(self,item,value): self.avatar.callRemote('checkData',item,value).addCallback(self.dataChecked) def dataChecked(self,answer): self.recvData = answer It looks as if it should work, but the server method 'perspective_checkData' never gets called. Is it possible to do anything like this? Thanks Frank Millman
Frank Millman wrote:
I have been experimenting with using a socket client instead of Twisted, and I got something up and running quite quickly. When I take a step back and look at how I got it working, I feel that I should be able to apply the same technique to Twisted, but I cannot get it to work. I will show both methods, and perhaps someone can point me in the right direction.
No, of course you can't apply the same technique. A socket client is a blocking, synchronous bit of code. The ENTIRE POINT of Twisted is that all operations are non-blocking, and asynchronous. Basically you're going to find it hard going until that's clear (I know I did). And sadly you're using wx, which is REALLY BAD and does not interact well with Twisted until recent developments (see below).
The trick is that I have two threads running concurrently - a wxPython main loop, and a subthread that monitors the socket. The socket loop looks like this (simplified), subclassed from threading.Thread -
def run(self): readable = [s.fileno()] error = [] self.sendData = [] while 1: if self.sendData: writable = [s.fileno()] else: writable = [] r,w,e = select.select(readable,writable,error,0.01) if r: self.recvData = s.recv(1024) if w: s.send(self.sendData.pop(0))
def checkData(self,item,value): # this is called from the *wx* thread self.recvData = None self.sendData.append(cPickle.dumps((CHECK,item,value))) while self.recvData is None: sleep(0.01) return self.recvData
As you can see, checkData() blocks the wxPython thread until it receives a response from the socket thread.
Fine. Odd, but fine (I'd use a semaphore as opposed to while True: sleep but whatever floats your boat)
This is how I tried with Twisted -
def checkData(self,item,value): # this is called from the *wx* thread self.recvData = None self.callCheckData(item,value) while self.recvData is None: sleep(0.01) return self.recvData
Bzzt! Error. You cannot call twisted functions from a thread. See extensive (and repetitive) posts in the archives about "threadselectreactor AND wx" for about a million posts describing this. Other than that, and the fact you've got no locking around mutable data structures, that code would probably work.
def callCheckData(self,item,value):
self.avatar.callRemote('checkData',item,value).addCallback(self.dataChecked)
def dataChecked(self,answer): self.recvData = answer
It looks as if it should work, but the server method 'perspective_checkData' never gets called.
It should not and does not work for the reason above. I believe "threadselectreactor" is what you want for wx integation. Sadly, I've never used threadselectreactor or wx, so can't be more useful to you. See: http://bob.pythonmac.org/archives/2005/04/17/twisted-and-foreign-event-loops... http://twistedmatrix.com/pipermail/twisted-python/2005-April/thread.html#101...
participants (3)
-
Frank Millman -
kwhitesell＠adelphia.net -
Phil Mayers