[Twisted-Python] problem with Twisted and Processing
I used Processing package for python 2.5 which became an official package of python 2.6 named multipleprocess. 'Cause the twisted for python 2.6 is not ready, I'm still using python 2.5 on windows Here's the problem the code below is pretty simple and works fine until I tried to get access of the first element in the list, I can even print the whole list, but i just can not get the elements inside. I guess there's something get conflicted in these two packages, anyone knows why? To reproduce the error, run the code below with twisted and processing package and python 2.5, open a telnet client to open the localhost 9999,send the message ,hit carriage return. thanks a lot for any hints from twisted.protocols import basic from twisted.internet import reactor import time from processing import Process,Manager def sender(list): while 1: if len(list) > 0: print list[0] time.sleep(1) class MyChat(basic.LineReceiver): def connectionMade(self): print "Got new client!" self.factory.clients.append(self) def connectionLost(self, reason): print "Lost a client!" self.factory.clients.remove(self) def lineReceived(self, line): print "received", repr(line) for c in self.factory.clients: c.message(line) d = {'client':self,'msg':line} msgList.append(d) def message(self, message): self.transport.write(message + '\n') from twisted.internet import protocol from twisted.application import service, internet if __name__ == '__main__': factory = protocol.ServerFactory() factory.protocol = MyChat factory.clients = [] reactor.listenTCP(9999,factory) manager = Manager() msgList = manager.list() p = Process(target=sender,args=(msgList,)) p.start() reactor.run()
On 2008.10.16 21:21:35 +0800, davy zhang wrote:
I used Processing package for python 2.5 which became an official package of python 2.6 named multipleprocess.
multiprocessing
This is not a Twisted problem. You're just using Processing wrong. You're trying to shove a MyChat instance inside your Manager list. But Manager only supports a few types by default. So you either need a BaseManager subclass with a handler for MyChat, or you need to convert your MyChat objects to a supported type (for example, by calling str() on them) before passing them across shared memory. -- David Ripton dripton@ripton.net
davy zhang wrote:
AFAICT the "processing" module does things like "fork"ing a running python interpreter and then not immediately exec()ing. I'm pretty sure that'll cause Twisted (and maybe lots of other apps) to explode. However I note you're on Windows (which lacks fork()) so I'm not sure what it does there. Also, the Processing module communicates over pipes/fifos which will conflict with Twisted and block the reactor (or the reactor will block Processing). In short - nice though it looks, I don't think you can use Processing with Twisted, or will ever be able to. It's a shame more attention wasn't paid to this during development. You could look at: http://pypi.python.org/pypi/ampoule http://foss.eepatents.com/trac/AsynQueue Having either of these implementations inside Twisted would be excellent IMHO.
On Fri, Oct 17, 2008 at 1:19 AM, Phil Mayers <p.mayers@imperial.ac.uk> wrote:
thanks so much for the advise! I just want one process to recv the message from the clinents ,and one process to send message back both processes shared a same queue seems to work, but.... is there any idea of that? if in other language thread will make that possible, but in python thread can not get benefit from multi-core cpus so I turn to processes
On 2008.10.16 21:21:35 +0800, davy zhang wrote:
I used Processing package for python 2.5 which became an official package of python 2.6 named multipleprocess.
multiprocessing
This is not a Twisted problem. You're just using Processing wrong. You're trying to shove a MyChat instance inside your Manager list. But Manager only supports a few types by default. So you either need a BaseManager subclass with a handler for MyChat, or you need to convert your MyChat objects to a supported type (for example, by calling str() on them) before passing them across shared memory. -- David Ripton dripton@ripton.net
davy zhang wrote:
AFAICT the "processing" module does things like "fork"ing a running python interpreter and then not immediately exec()ing. I'm pretty sure that'll cause Twisted (and maybe lots of other apps) to explode. However I note you're on Windows (which lacks fork()) so I'm not sure what it does there. Also, the Processing module communicates over pipes/fifos which will conflict with Twisted and block the reactor (or the reactor will block Processing). In short - nice though it looks, I don't think you can use Processing with Twisted, or will ever be able to. It's a shame more attention wasn't paid to this during development. You could look at: http://pypi.python.org/pypi/ampoule http://foss.eepatents.com/trac/AsynQueue Having either of these implementations inside Twisted would be excellent IMHO.
On Fri, Oct 17, 2008 at 1:19 AM, Phil Mayers <p.mayers@imperial.ac.uk> wrote:
thanks so much for the advise! I just want one process to recv the message from the clinents ,and one process to send message back both processes shared a same queue seems to work, but.... is there any idea of that? if in other language thread will make that possible, but in python thread can not get benefit from multi-core cpus so I turn to processes
participants (3)
-
David Ripton -
davy zhang -
Phil Mayers