Real-world Python code 700 times slower than C

[Skip Montanaro]

    Brent> I often use a "10x" rule of thumb for comparing Python to C, but
    Brent> I recently hit one real-world case where Python is almost 700
    Brent> times slower than C!  We just rewrote the routine in C and moved
    Brent> on, but this has interesting implications for Python optimization
    Brent> efforts.

    ...

    Brent> What I really want is something spiritually equivalent to a
    Brent> portable inline assembly language with python-ish syntax that
    Brent> generates really fast native code and seamlessly integrates with
    Brent> python.  I can dream can't I?

Brent,

10x is probably a bit optimistic in most cases.  I typically think of Python
as 50-100x slower in most cases.  In this particular case, where you're not
using any of Python's features (high-level data structures and memory
management, good interfaces to other packages that would offset performance
with programmer productivity), Python is going to fare even worse.

I think your "rewrote the routine in C and moved on" is the right thing
here.  Alternatives include weave and PyInline:

    http://www.scipy.org/site_content/weave
    http://pyinline.sourceforge.net/

With a slight modification to your Ramp function, the following code runs
with PyInline and appears to get about a 7.5x speedup and the correct answer
(which is always helpful ;-):

    #!/usr/bin/env python

    import PyInline

    m = PyInline.build(code="""
    void Ramp(PyObject* result, int size, double start, double end)
    {
        double step = (end-start)/(size-1);
        int i;
        for (i = 0; i < size; i++)
            PySequence_SetItem(result, i, PyFloat_FromDouble(start + step*i));
    }
    """, language="C")

    def Ramp(result, size, start, end):
        step = (end-start)/(size-1)
        for i in xrange(size):
            result[i] = start + step*i

    def main(ramp):
        array = [0.0]*10000
        for i in xrange(100):
            ramp(array, 10000, 0.0, 1.0)

    import time
    t = time.time()
    main(m.Ramp)
    print "fast:", time.time()-t

    t = time.time()
    main(Ramp)
    print "slow:", time.time()-t

-- 
Skip Montanaro (skip at pobox.com - http://www.mojam.com/)