[Twisted-Python] XML-RPC server
I am quite new to Twisted and I'm having some trouble getting my XML-RPC server to do exactly what I want. We have a relatively simple goal: create an XML-RPC server that will accept a request for a score, call a function that computes that score, and then return the score. The two catches are that 1. the client is actually a .NET web server rather than a Twisted client, and 2. we could have thousands of requests come in relatively simultaneously (though more likely not greater than a few hundred), likely with lots of downtime between these periods of high activity. We are planning to create a look-up table to deal with some of the more common calculations, but I still need to be able to run the other calculations non-sequentially. I know that using a Deferred object is the normal solution, but I'm not entirely sure if it could work here without threads. I read somewhere that when the function call is not from another Twisted application, it's not possible to use the Deferred object. Is this actually the case? Or would it work in my situation since all of the "deferring" would be occurring within the server itself? My current solution involves using threads and a Deferred object. Is this the best way to do this? Can it be done without threads? If so, how and is that a better solution than the one with threads? Here is my sample code, which does a simple calculation rather than the real one I will be using. Both my Twisted testing client and the .NET web server are able to talk to this server and get the correct results back. from twisted.web import resource, server, xmlrpc from twisted.internet import defer, threads from twisted.python import threadable threadable.init() class ScoreServer(xmlrpc.XMLRPC): def xmlrpc_calculateScore(self,params): t = params[0] d = threads.deferToThread(self.calc,t) val = d.addCallback(self.handleSuccess) d.addErrback(self.handleFailure) print t, d return val def calc(self,t): import time time.sleep(t) result = [t,t*10] return result def handleSuccess(self,t): print 'succeeded', t return t def handleFailure(self): print 'failed miserably' return ['Error'] if __name__ == "__main__": from twisted.internet import reactor root = resource.Resource() root.putChild('RPC2', ScoreServer()) reactor.listenTCP(8082, server.Site(root)) reactor.run()
Kelly Vincent wrote:
I am quite new to Twisted and I'm having some trouble getting my XML-RPC server to do exactly what I want.
We have a relatively simple goal: create an XML-RPC server that will accept a request for a score, call a function that computes that score, and then return the score. The two catches are that 1. the client is actually a .NET web server rather than a Twisted client, and 2. we could have thousands of requests come in relatively simultaneously (though more likely not greater than a few hundred), likely with lots of downtime between these periods of high activity. We are planning to create a look-up table to deal with some of the more common calculations, but I still need to be able to run the other calculations non-sequentially. I know that using a Deferred object is the normal solution, but I'm not entirely sure if it could work here without threads. I read somewhere that when the function call is not from another Twisted application, it's not possible to use the Deferred object. Is this actually the case? Or would it work in my situation since all of the "deferring" would be occurring within the server itself? My current solution involves using threads and a Deferred
To my knowledge, a client has nothing to do with twisted internals like deferreds. For example I use a twisted xmlrpc server that get's it's requests from a php script via apache. You also could test your code with a simple python xmlrpc client like: from xmlrpclib import ServerProxy server = ServerProxy("http://yourserver:port") print server try: print server.your.method(attr) except Exception, v: print "ERROR", v - Remi -
Hi Kelly,
On Fri, 26 May 2006 07:36:53 -0500, Kelly Vincent
I am quite new to Twisted and I'm having some trouble getting my XML-RPC server to do exactly what I want.
First off, welcome aboard! Now, on to your problem/question...
We have a relatively simple goal: create an XML-RPC server that will accept a request for a score, call a function that computes that score, and then return the score. The two catches are that 1. the client is actually a .NET web server rather than a Twisted client, and 2. we could have thousands of requests come in relatively simultaneously (though more likely not greater than a few hundred), likely with lots of downtime between these periods of high activity. We are planning to create a look-up table to deal with some of the more common calculations, but I still need to be able to run the other calculations non-sequentially. I know that using a Deferred object is the normal solution, but I'm not entirely sure if it could work here without threads. I read somewhere that when the function call is not from another Twisted application, it's not possible to use the Deferred object. Is this actually the case? Or would it work in my situation since all of the "deferring" would be occurring within the server itself? My current solution involves using threads and a Deferred object. Is this the best way to do this? Can it be done without threads? If so, how and is that a better solution than the one with threads?
This kind of situation is pretty common, and finding the best solution will require you to do some benchmarking. Firstly, benchmark your calculation: How much time does it take to return a result? If the calculation is fairly swift (milliseconds), then don't worry about Deferreds at all; just perform the calculation and return the result. If the calculation takes longer (seconds, minutes, whatever), then you can either use deferToThread, as you are currently, or you break your calculation up into steps, and use reactor.callLater to schedule execution incrementally. That said, run a simple load-testing tool, such as apache's "ab", or "httperf", and see what kind of requests-per-second numbers you get for both approaches. Hope this helps, L. Daniel Burr
participants (3)
-
Kelly Vincent
-
L. Daniel Burr
-
Remi Cool