I tried to implement this: parent_conn, child_conn = Pipe() f = defer.Deferred() f = threads.deferToThread(start_test.main_func, SCRIPT_PATH, TEMP_OUTPUT_PATH, self.output_name, child_conn) response = parent_conn.recv() print response //prints like: initialization done self.transport.write(response) response = parent_conn.recv() print response // configuration done self.transport.write(response) But get the same output ie all status messages concatenated and sent at once.! [Do we need to put some deffered/callback mechanism on parent_conn.recv() ?? as that is the call blocking the thread waiting for reading.] Any ideas ? How is such stuff done in twisted servers?? Naman On Tue, Oct 27, 2009 at 7:03 AM, Nathan wrote:

On Mon, Oct 26, 2009 at 11:44 PM, naman jain wrote:

...

Hi,

I have a client server model in twisted, where the server spawns a thread ( basically a test script in python that runs for about 20 mins) I want to track the progress of the thread, and send the progress to the client back

So, I write something like this in my server:

parent_conn, child_conn = Pipe() thread = Process(target = start_test.main_func, args=(SCRIPT_PATH, TEMP_OUTPUT_PATH, self.output_name, child_conn)) thread.start()

response = parent_conn.recv() print response //prints like: initialization done self.transport.write(response)

response = parent_conn.recv() print response // configuration done self.transport.write(response)

. . .

thread.join()

But the transport.write calls don't send at the same time. instead they wait for the thread to finish (coz of thread.join) and then append all the response and send it back; like "initialization doneconfiguration done...done" thereby defeating the purpose of creating a thread.

How do I give the control to the reactor to write the data back, and still keep the thread running? or is there any other way these kinda of progress can be tracked ? I am kinda stuck with this :(

I'm no expert with threads, but I don't see any deferToThread in there, which is what people on this list tend to mention whenever threading comes up. Perhaps you need to switch to the "twisted way" to create your thread?

http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.threads.html

~ Nathan

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Naman, Another problem you might (possibly) be encountering - is the way your server and/or client protocol is implemented. If you just inherit from "plain" twisted.internet.protocol.Protocol - you are automatically using write () and dataReceived() methods that do not implement buffering (it is left up to developer). To enforce that line (string) oriented communication works correctly - you might want to refactor to use twisted.protocols.basic.LineReceiver instead. Kind regards, Valeriy Pogrebitskiy Email: vpogrebi@iname.com -----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of naman jain Sent: Tuesday, October 27, 2009 6:36 AM To: Twisted general discussion Subject: Re: [Twisted-Python] Regarding Twisted Matrix I tried to implement this: parent_conn, child_conn = Pipe() f = defer.Deferred() f = threads.deferToThread(start_test.main_func, SCRIPT_PATH, TEMP_OUTPUT_PATH, self.output_name, child_conn) response = parent_conn.recv() print response //prints like: initialization done self.transport.write(response) response = parent_conn.recv() print response // configuration done self.transport.write(response) But get the same output ie all status messages concatenated and sent at once.! [Do we need to put some deffered/callback mechanism on parent_conn.recv() ?? as that is the call blocking the thread waiting for reading.] Any ideas ? How is such stuff done in twisted servers?? Naman On Tue, Oct 27, 2009 at 7:03 AM, Nathan wrote: On Mon, Oct 26, 2009 at 11:44 PM, naman jain wrote:

Hi,

I have a client server model in twisted, where the server spawns a thread ( basically a test script in python that runs for about 20 mins) I want to track the progress of the thread, and send the progress to the client back

So, I write something like this in my server:

parent_conn, child_conn = Pipe() thread = Process(target = start_test.main_func, args=(SCRIPT_PATH, TEMP_OUTPUT_PATH, self.output_name, child_conn)) thread.start()

response = parent_conn.recv() print response //prints like: initialization done self.transport.write(response)

response = parent_conn.recv() print response // configuration done self.transport.write(response)

. . .

thread.join()

But the transport.write calls don't send at the same time. instead they wait for the thread to finish (coz of thread.join) and then append all the response and send it back; like "initialization doneconfiguration done...done" thereby defeating the purpose of creating a thread.

How do I give the control to the reactor to write the data back, and still keep the thread running? or is there any other way these kinda of progress can be tracked ? I am kinda stuck with this :(

I'm no expert with threads, but I don't see any deferToThread in there, which is what people on this list tend to mention whenever threading comes up. Perhaps you need to switch to the "twisted way" to create your thread? http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.threads.html ~ Nathan _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python