[Python-Dev] xmlrpclib speed boost

[Skip Montanaro]

    >> This change may be deemed not to be the correct fix as far backwards
    >> compatibility is concerned (it uses the "from m import x as y"
    >> feature which was new with 2.0 I think).  If someone alters this fix,
    >> please don't put the import back into the functions that call
    >> cgi.escape.

    Thomas> Why not ? Moving the import to the top level just causes the
    Thomas> slowdown to occur at a different moment. If this is really the
    Thomas> problem, the slowdown should occur only the first time you use a
    Thomas> particular function (unless you explicitly un-import cgi somehow
    Thomas> ?)

Trust me, this really *is* the problem.  Instead of getting imported once,
it gets imported once for every string and once for every key in every
dictionary.  I timed it before and after.  Compare Marshaller.dump_string
before

    def dump_string(self, value):
        from cgi import escape
        self.write("<value><string>%s</string></value>\n" % escape(value))

and after

    def dump_string(self, value):
        self.write("<value><string>%s</string></value>\n" % _escape(value))

I don't care how little work it is to import a module a second time, it is
probably going to be on the same order of magnitude as that write call.  In
Marshaller.dump_struct the import was *inside* the for loop.  I originally
pulled it up out of the for loop but left it inside the method.  That got me
about a 25% boost in my simple test.  I was ready to check in that one
change and thought, "aw hell, might as well see what happens if I pull the
import all the way out to the top level".  Dump performance went all the way
back up to where 0.9.8 is.

    Thomas> Importing cgi only in the functions that actually use it, in
    Thomas> order to avoid the slowdown unless it's really necessary, sure
    Thomas> seems like a sensible solution to me :-)

Except dumping structs (dictionaries) and strings happens a lot.  Even if
all we ever did was dump ints, floats and lists, the total cost of my change
would be one import.

Skip