Can XML-RPC performance be improved?

skip at skip at
Tue Mar 21 18:55:44 CET 2006

    Diez> Sion Arrowsmith wrote:
    >> I've got an established client-server application here where there
    >> is now a need to shovel huge amounts of data (structured as lists of
    >> lists) between the two, and the performance bottleneck has become
    >> the amount of time spent parsing XML ...

    Diez> CORBA. Or TCP/IP directly. I know that isn't really the answer
    Diez> you're looking for - but there is an overhead parsing XML, and
    Diez> that can't go away.

Ah, but if both ends of the pipe happen to be running Python, you can
cheat. ;-)  Run your list through marshal.dumps() before passing it to
xmlrpclib, then undo the marshalling on the other end.

    >>> biglist = ["abc"] * 50
    >>> biglist = [biglist] * 50
    >>> import xmlrpclib
    >>> import marshal
    >>> x = xmlrpclib.dumps((biglist,))
    >>> len(x)
    >>> x.count("<")
    >>> y = xmlrpclib.dumps((marshal.dumps(biglist),))
    >>> len(y)
    >>> y.count("<")

If you can swing it, I'd be willing to bet you a beer your XML parsing time
will all but disappear.

If you're not using xmlrpclib with some sort of C helper (like sgmlop), try
that as well.  Big difference parsing XML in C and Python.


More information about the Python-list mailing list