[Twisted-Python] Serious performance problem of twisted

a very simple server and multi-process code, I found there's about 500 milliseconds delay for every msg I sent to client I just don't why, did I made something wrong ? the code below using a process to handle the data and in the shared taskList, after done put the data to the shared msgList, a thread in main process to read the msg and send to the clients. the msg delay is significant, you can even see it with your eyes in telnet client I checked the time in the client, it's approximate to 0.5 secs first I tested in java using the same logic with MINA, BlockingQueue stuff, the msg traveled from client to server and back is 0, so I pretty much sure the problem here is in the server side then I printed time in the server code, I found everything is as fast as it suppose to be, until the c.transport.write. After the write operation the client need 0.5secs to get the msg the code is below #!/usr/bin/env python #coding=utf-8 from twisted.internet.protocol import Factory,Protocol from twisted.internet import reactor from twisted.internet import error import cPickle import time from processing import Manager,Process import thread from time import time """flash policy file request""" policyRequest = "<policy-file-request/>" + chr(0) policyReturn = """<cross-domain-policy> <allow-access-from domain="*" to-ports="*" /> </cross-domain-policy>""" + chr(0) def handler(taskList,msgList): while 1: item = cPickle.loads(taskList.get()) print time() print 'item before handle ', item item['msg'] += ' hanlded done \r\n' msgList.put(cPickle.dumps(item)) def sender(): global msgList while 1: item = cPickle.loads(msgList.get()) print time() c = clients[item['cid']] c.transport.write(item['msg'] ) class DTProtocol(Protocol): global clients def connectionMade(self): clients.append(self) print "a new customer!" def dataReceived(self, data): global clients global taskList print 'msg received ',data if data == policyRequest: self.transport.write(policyReturn) self.transport.loseConnection() else: d = {'cid':clients.index(self),'msg':data,'done':False,'sent':False} taskList.put(cPickle.dumps(d)) print time() #self.transport.write(data) def connectionLost(self, reason): print reason if self in clients: clients.remove(self) print "bye!" def message(self,msg): self.transport.write(msg) class DTFactory(Factory): protocol = DTProtocol if __name__ == '__main__': port = 80 reactor.listenTCP(80,DTFactory()) clients = [] manager = Manager() taskList = manager.Queue() msgList = manager.Queue() h = Process(target=handler,args=(taskList,msgList)) h.start() thread.start_new_thread(sender,()) print 'server is runing at ',port reactor.run()

Le dimanche 19 octobre 2008 à 19:38 +0800, davy zhang a écrit :
Short answer: http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whydoesittakealo... A bit more elaborated: yes, you did something wrong. The Twisted APIs are not threadsafe, thus you must not call any Twisted method from a thread, except reactor.callFromThread. See http://twistedmatrix.com/projects/core/documentation/howto/threading.html for more information. -- Thomas

Le dimanche 19 octobre 2008 à 19:38 +0800, davy zhang a écrit :
Short answer: http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whydoesittakealo... A bit more elaborated: yes, you did something wrong. The Twisted APIs are not threadsafe, thus you must not call any Twisted method from a thread, except reactor.callFromThread. See http://twistedmatrix.com/projects/core/documentation/howto/threading.html for more information. -- Thomas
participants (3)
-
davy zhang
-
Itamar Shtull-Trauring
-
Thomas Hervé