[Twisted-Python] Launching Twisted client using Process call
Hi , I have a twisted client server model running. My requirement is to launch my client from another program based on some event received by the master program. I try to launch it as a Process: foo = Process( target=twisted_client.main, args=(q,uid,)) foo.start() foo.join() status = q.get() Launching the client in this manner, is not showing reliable results. Frequently , I get this error: "*Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.TimeoutError'>: User timeout caused connection failure.* " when on the other side using: os.system("python twisted_client.py") runs perfectly, connecting everytime to the server. I found relevant discussion on the following thread but could not figure out a solution. http://twistedmatrix.com/pipermail/twisted-python/2007-December/016430.html Thanks for any help! Regards Naman
On 2009.11.17 07:07:50 +0100, naman jain wrote:
Launching the client in this manner, is not showing reliable results.
You didn't give a long enough example to be certain, but that looks like multiprocessing.Process syntax. The multiprocessing module does not work reliably with Twisted. Neither does the subprocess module. You should use the equivalent Twisted process functionality, shown at http://twistedmatrix.com/projects/core/documentation/howto/process.html There's also https://launchpad.net/ampoule, but for the small example you show, the basic Twisted process stuff should be fine. -- David Ripton dripton@ripton.net
Hi, Thanks for the reply. I actually came across Process documentation for twisted; but in my case my the script which calls the twisted client is not implemented in Twisted It is just a controller program which instantiates a central repository and spawns off 2 threads and then the threads update to that repository periodically (so I did not feel the need to run a reactor loop as in Twisted ). Is there any way to call a Twisted client in a Process or a thread without using the reactor? To explain better: master.py spawns 2 threads to update a central repository. build_worker_t : launches a Twisted client to connect to a Twisted server running on a build machine test_worker_t : launches a Twisted client to connect to a Twisted server running on a Test Machine master.py: ___________________________________________________________________ // make a central repository for the threads to update class Globals: repo = [] lock = threading.Lock() def main(): build_worker_t = threading.Thread( target=dispatch_build_worker) build_worker_t.start() test_worker_t = threading.Thread( target=dispatch_test_worker) test_worker_t.start() build_worker_t.join() test_worker_t.join() def dispatch_build_worker(): # build_worker.main is a twisted client foo = Process(target=build_worker.main, args=(build_q,uid,)) foo.start() foo.join() //wait for the process(ie. client-server communication to finish) //get status from build_q and update repo def dispatch_test_worker(): Regards Naman On Tue, Nov 17, 2009 at 3:30 PM, David Ripton <dripton@ripton.net> wrote:
Just to add to the previous post:
foo = Process(target=build_worker.main, args=(build_q,uid,))
is the statement I want to replace (because it is unreliable), with something other that reactor.spawnprocess as there is no reactor in the calling script. Thanks Naman On Tue, Nov 17, 2009 at 3:30 PM, David Ripton <dripton@ripton.net> wrote:
On 10:59 am, namanvit@gmail.com wrote:
Please could I get some help on this issue, if any one has some ideas.
Can you produce a short, self-contained, correct example (<http://sscce.org/>) of the problem you're having? Jean-Paul
On 2009.11.17 07:07:50 +0100, naman jain wrote:
Launching the client in this manner, is not showing reliable results.
You didn't give a long enough example to be certain, but that looks like multiprocessing.Process syntax. The multiprocessing module does not work reliably with Twisted. Neither does the subprocess module. You should use the equivalent Twisted process functionality, shown at http://twistedmatrix.com/projects/core/documentation/howto/process.html There's also https://launchpad.net/ampoule, but for the small example you show, the basic Twisted process stuff should be fine. -- David Ripton dripton@ripton.net
Hi, Thanks for the reply. I actually came across Process documentation for twisted; but in my case my the script which calls the twisted client is not implemented in Twisted It is just a controller program which instantiates a central repository and spawns off 2 threads and then the threads update to that repository periodically (so I did not feel the need to run a reactor loop as in Twisted ). Is there any way to call a Twisted client in a Process or a thread without using the reactor? To explain better: master.py spawns 2 threads to update a central repository. build_worker_t : launches a Twisted client to connect to a Twisted server running on a build machine test_worker_t : launches a Twisted client to connect to a Twisted server running on a Test Machine master.py: ___________________________________________________________________ // make a central repository for the threads to update class Globals: repo = [] lock = threading.Lock() def main(): build_worker_t = threading.Thread( target=dispatch_build_worker) build_worker_t.start() test_worker_t = threading.Thread( target=dispatch_test_worker) test_worker_t.start() build_worker_t.join() test_worker_t.join() def dispatch_build_worker(): # build_worker.main is a twisted client foo = Process(target=build_worker.main, args=(build_q,uid,)) foo.start() foo.join() //wait for the process(ie. client-server communication to finish) //get status from build_q and update repo def dispatch_test_worker(): Regards Naman On Tue, Nov 17, 2009 at 3:30 PM, David Ripton <dripton@ripton.net> wrote:
Just to add to the previous post:
foo = Process(target=build_worker.main, args=(build_q,uid,))
is the statement I want to replace (because it is unreliable), with something other that reactor.spawnprocess as there is no reactor in the calling script. Thanks Naman On Tue, Nov 17, 2009 at 3:30 PM, David Ripton <dripton@ripton.net> wrote:
On 10:59 am, namanvit@gmail.com wrote:
Please could I get some help on this issue, if any one has some ideas.
Can you produce a short, self-contained, correct example (<http://sscce.org/>) of the problem you're having? Jean-Paul
participants (3)
-
David Ripton -
exarkun＠twistedmatrix.com -
naman jain