Can XML-RPC performance be improved?
siona at chiark.greenend.org.uk
Thu Mar 23 14:52:03 CET 2006
<skip at pobox.com> wrote:
> Diez> Sion Arrowsmith wrote:
> >> I've got an established client-server application here where there
> >> is now a need to shovel huge amounts of data (structured as lists of
> >> lists) between the two, and the performance bottleneck has become
> >> the amount of time spent parsing XML ...
> Diez> CORBA. Or TCP/IP directly. I know that isn't really the answer
> Diez> you're looking for - but there is an overhead parsing XML, and
> Diez> that can't go away.
>Ah, but if both ends of the pipe happen to be running Python, you can
>cheat. ;-) Run your list through marshal.dumps() before passing it to
>xmlrpclib, then undo the marshalling on the other end.
> [ ... ]
>If you can swing it, I'd be willing to bet you a beer your XML parsing time
>will all but disappear.
I wouldn't take you up on that bet, because I'd already considered
that as a likely solution, and while I was asking here another member
of the team went away and implemented it for me. Except with cPickle
(as suggested elsewhere in the thread) -- we've got DateTime objects
in there, which marshal chokes on. Initial tests show you'd've
comfortably won your beer: 16s XML parsing down to 0.5s parsing plus
>If you're not using xmlrpclib with some sort of C helper (like sgmlop), try
>that as well. Big difference parsing XML in C and Python.
Diez edited out the bit where I said we were using sgmlop and worried
about what it would be like if we weren't 8-)
\S -- siona at chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
More information about the Python-list