mystifying xmlrpclib performance problem...

[Skip Montanaro]

I'm stumped trying to figure out why Python runs xmlrpclib marshalling and
unmarshalling so much faster on one machine than another, when everything
suggests the results should be similar on the two machines.  Here's a simple
script I've been using to test things:

    import xmlrpclib
    import time

    def test(obj, func, n):
        t = time.clock()
        for i in range(n):
            x = func(obj)
        return time.clock()-t

    def test_dumps(obj, n):
        return test((obj,), xmlrpclib.dumps, n)

    def test_loads(obj, n):
        s = xmlrpclib.dumps((obj,))
        return test(s, xmlrpclib.loads, n)

    obj = ['MusicEntry',
           {'email': 'lalcorn at ultranet', 'time': '7:30pm', 'tickets': '',
            'program': '', 'state': 'MA', 'start': '2002-01-26', 'venueurl': '',
            'country': '', 'performers': ['An Evening with Karen Savoca'],
            'addressid': 7283, 'name': '', 'zip': '', 'city': 'Sudbury',
            'info': 'Reservations required. Please call (978)443-3253 or e-mail Laurie at lalcorn at ultranet.com.',
            'merchandise': [], 'event': '', 'keywords': ['.zyx.41'],
            'submit_time': '2001-08-28', 'key': 325629,
            'active': 1, 'end': '2002-01-26', 'price': '$17', 'address3': '',
            'address1': '', 'venue': 'Fox Run House Concerts',
            'address2': '', 'update_time': '2001-09-22:19:28:44'}]

    n = 1000
    print "%.3g dumps per second" % (n/test_dumps(obj, n))
    n /= 2
    print "%.3g loads per second" % (n/test_loads(obj, n))

Here are the machine parameters and software I am working with:

                Machine 1 (Dell laptop)             Machine 2 (Dell XPS)
    CPU         P-III 450MHz, 256k cache            P-III 550MHz, 512k cache
    Memory      128MB                               256MB
    OS          Mandrake 8.0                        Mandrake 7.2
    Bogomips    897.84                              1094.45
    Python      2.1.1                               2.1.1
    GCC         3.0.1                               3.0.1
    OPT flags   -O6 -mcpu=pentiumpro                -O6 -mcpu=pentiumpro
    pystone.py  5400/5850 (-O)                      5950/6410 (-O)
    xmlrpclib   1.0b3                               1.0b3
    sgmlop      000528                              000528

Unfortunately, when I run the testxmlrpc.py script, Machine 2 runs
substantially slower than Machine 1:

    dumps       407/sec                             222/sec
    loads       104/sec                              89.5/sec

I verified manually that xmlrpclib.getparser() returns an SgmlopParser
instance on both machines.  The test is small enough that it should run
completely in memory.  (Even if it didn't, Machine 1's disk subsystem is
likely to be lower performance than Machine 2's.)

The relative performance of the malloc packages installed on the two
machines should have been exposed by the pystone runs, if in fact there was
a difference.  Just the same, I built a version of GNU malloc (circa 1995) I
had laying around and relinked the Python executables on both machines with
it.  I compiled gmalloc.c with the same optimization flags as I used to
build the Python interpreters:

                Machine 1 (Dell laptop)             Machine 2 (Dell XPS)
    pystone.py  5750/6210 (-O)                      6490/6990 (-O)

Other than demonstrating that the malloc packages were apparently not the
cause of the discrepancy and that you should link Python with a heavily
optimized version of malloc, I saw no relative change on xmlrpclib's
dump/load speed:

    dumps       478/sec                             264/sec
    loads       134/sec                             114/sec

Any ideas what might be causing the performance drop on the supposedly
faster machine?

Thanks,

-- 
Skip Montanaro (skip at pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/