[Twisted-Python] Fast RPC
It looks like this has come up once or twice on the list under the guyse of "how do I make synchronous calls to PB from another application" with no clear answer. It seems to be hard to do because of the design of PB. Use XMLRPC is the usual response. Here's the situation. I have an embedded python interpreter running inside of Yate. Because of the way Yate deals with threads and signals, it seems to be hard to get a twisted reactor to run inside the embedded interpreter. I also have a twisted process that wraps an API around a database and exposes it over XMLRPC and PB. The embedded interpreter needs to get at various bits of data by making RPC calls to this API. Using XMLRPC works. But it is very slow. Especially since xmlrpclib wants to make a new TCP connection for each call. Using a test PB client, PB calls are much faster, since there is a persistant TCP connection and simpler encoding. I have toyed with the idea of using Sun RPC or something similar to have the embedded interpreter talk to a local twisted process that then would talk over the network, and perhaps this is not a bad option. Any suggestions for a way to make RPC calls over the network with the speed of the PB and the synchronicity of XMLRPC? Cheers, -w
On Jul 2, 2005, at 9:37 AM, William Waites wrote:
It looks like this has come up once or twice on the list under the guyse of "how do I make synchronous calls to PB from another application" with no clear answer. It seems to be hard to do because of the design of PB. Use XMLRPC is the usual response.
Here's the situation. I have an embedded python interpreter running inside of Yate. Because of the way Yate deals with threads and signals, it seems to be hard to get a twisted reactor to run inside the embedded interpreter. I also have a twisted process that wraps an API around a database and exposes it over XMLRPC and PB. The embedded interpreter needs to get at various bits of data by making RPC calls to this API.
threadedselectreactor (in svn, will be in 2.1) runs the blocking part of the reactor in a separate thread, so you could do blocking PB- based RPC with that.. I just checked in an example of how you might be able to use it: Twisted/trunk/doc/core/examples/threadedselect/blockingdemo.py Not really tested or thought out very well, but it should work. -bob
On Sat, Jul 02, 2005 at 10:27:28AM -1000, Bob Ippolito wrote:
threadedselectreactor (in svn, will be in 2.1) runs the blocking part of the reactor in a separate thread, so you could do blocking PB- based RPC with that..
This seems to work, even just taking that file and adding it by hand to a 2.0.1 install. Thanks! -w
On Sun, Jul 03, 2005 at 12:49:20AM -0400, Itamar Shtull-Trauring wrote:
On Sat, 2005-07-02 at 15:37 -0400, William Waites wrote:
Because of the way Yate deals with threads and signals, it seems to be hard to get a twisted reactor to run inside the embedded interpreter.
reactor.run(installSignalHandlers=False) may fix that.
Hmmm. I tried that, but then the reactor stalls. No reactor.callLater's actually seem to get called... -w
On Tue, 2005-07-05 at 23:17 -0400, William Waites wrote:
reactor.run(installSignalHandlers=False) may fix that.
Hmmm. I tried that, but then the reactor stalls. No reactor.callLater's actually seem to get called...
That's how it's supposed to work: reactor.run() doesn't exit until you stop the reactor. You need to setup scheduled calls before running the reactor.
On Wed, Jul 06, 2005 at 11:51:37AM -0400, Itamar Shtull-Trauring wrote:
On Tue, 2005-07-05 at 23:17 -0400, William Waites wrote:
reactor.run(installSignalHandlers=False) may fix that.
Hmmm. I tried that, but then the reactor stalls. No reactor.callLater's actually seem to get called...
That's how it's supposed to work: reactor.run() doesn't exit until you stop the reactor. You need to setup scheduled calls before running the reactor.
I understand that. I had set up a couple of reactor.callLater's before calling reactor.run(). They just don't actually seem to get run. For actual operation, control passes from a thread in the C++ method TelEngine::MessageHandler::received(Message &m) to a python function softswitch.message(m) which then calls into the reactor using callFromThread. Bit I'm not quite there yet, first step was to get the reactor running, which seems to work with threadedselectreactor. Incidentally, I am running the standard reactor like so in the block of code that runs when the Yate module is imported after it initializes the interpreter: from threading import Thread t = Thread(target = reactor.run, kwargs = {'installSignalHandlers' : False}) t.start() This is necessary because the Yate module's initialization method is expected to return. But is this expected to work in your experience? Alternatively the Yate module could run the main interpreter in a separate thread itself that is not expected to return. This is what the (corresponding) asterisk module does. That might be the best way, but then it assumes a main-thread function in python does not return, which is a nuisance if you want to do something in the python module without using twisted. -w
On Wed, Jul 06, 2005 at 11:51:37AM -0400, Itamar Shtull-Trauring wrote:
On Tue, 2005-07-05 at 23:17 -0400, William Waites wrote:
reactor.run(installSignalHandlers=False) may fix that.
Hmmm. I tried that, but then the reactor stalls. No reactor.callLater's actually seem to get called...
That's how it's supposed to work: reactor.run() doesn't exit until you stop the reactor. You need to setup scheduled calls before running
On Wed, 6 Jul 2005 12:11:43 -0400 William Waites <ww@groovy.net> wrote: the
reactor.
I understand that. I had set up a couple of reactor.callLater's before calling reactor.run(). They just don't actually seem to get run.
For actual operation, control passes from a thread in the C++ method TelEngine::MessageHandler::received(Message &m) to a python function softswitch.message(m) which then calls into the reactor using callFromThread. Bit I'm not quite there yet, first step was to get the reactor running, which seems to work with threadedselectreactor.
Please make sure you understand how to use threadedselectreactor. Specifically, you should not call run() on it. You are supposed to call reactor.interleave(func), where func is your waker function. You _must_ do that or threadedselectreactor will not work. See the examples in Twisted/doc/core/examples/threadedselect.
participants (4)
-
Bob Ippolito -
Itamar Shtull-Trauring -
Pavel Pergamenshchik -
William Waites