Newbie to XML-RPC: looking for advice

David Hirschfield davidh at ilm.com
Fri Jan 13 20:50:13 EST 2006


I've written a server-client system using XML-RPC. The server is using 
the twisted.web.xmlrpc.XMLRPC class to handle connections and run 
requests. Clients are just using xmlrpclib.ServerProxy to run remote 
method calls.

I have a few questions about the performance of xmlrpc in general, and 
specifically about increasing the speed with which remote methods return 
their results to the client.

Right now the process of calling a remote method works as follows:

client:
generate some python objects
serialize those objects by cPickling them with cPickle.HIGHEST_PROTOCOL, 
then wrap the pickles with xmlrpclib.Binary() so the data can be sent safely
call the remote method via a ServerProxy object using the Binary object 
as the argument

server:
invoke the method and extract the pickled and Binary()'d arguments back 
into the actual objects
do some work
take result objects and cPickle them and wrap them in a Binary object as 
before
return the result to the client

client:
receive result and unpickle it into real data

All the above works fine...but I'm finding the following: while the 
actual creation and pickling of the objects only takes a millisecond or 
so, the actual time before the client call completes is a third of a 
second or more.

So where's the slowdown? It doesn't appear to be in the 
pickling/unpickling or object creation, so it has to be in xmlrpc 
itself...but what can I do to improve that? It looks like xmlrpclib uses 
xml.parsers.expat if it's available, but are there faster xml libs? 
Looking at the xmlrpclib code itself, it seems to want to find either: 
_xmlrpclib from the code in xmlrpclib.py:

try:
    # optional xmlrpclib accelerator.  for more information on this
    # component, contact info at pythonware.com
    import _xmlrpclib
    FastParser = _xmlrpclib.Parser
    FastUnmarshaller = _xmlrpclib.Unmarshaller
except (AttributeError, ImportError):
    FastParser = FastUnmarshaller = None

or it tries to find sgmlop:

#
# the SGMLOP parser is about 15x faster than Python's builtin
# XML parser.  SGMLOP sources can be downloaded from:
#
#     http://www.pythonware.com/products/xml/sgmlop.htm
#

Does anyone know what the performance gain from using either of those 
above libraries would be?
On the other hand, maybe the slowdown is in twisted.web.xmlrpc? What 
does that module use to do its work? Is it using xmlrpclib underneath?
Other xmlrpc libraries that are significantly faster that I should be 
using instead?

Any help in improving my xmlrpc performance would be greatly appreciated,
-Dave

-- 
Presenting:
mediocre nebula.




More information about the Python-list mailing list