Question about scientific calculations in Python

[Fernando Pérez]

Martin Kaufmann wrote:

> 
> How do you glue C and Python together? Do you use any of the wrappers
> (SWIG, weave etc.) or do you do it the hard way? I just didn't
> understand the whole story about these PyObjects.
> 
>>One general comment: The performance killer in many applications like
>>this is doing the innermost loop in the interpreted language (Python,
>>Matlab, etc).  If you can recast your algorithm as operations on
>>arrays rather than individual numbers, and your language does the
>>implicit loops over the elements efficiently, you typically get 10x or
>>so improvements in speed.
> 
> Do you mean the array calculation of NumPy?

Yes. A quick example (trivially easy, but shows the point):
In [21]: x
Out[21]: array([ 0.        ,  0.34906585,  0.6981317 ,  1.04719755,  
1.3962634 ,  1.74532925,
             2.0943951 ,  2.44346095,  2.7925268 ,  3.14159265])

In [22]: sx = zeros(len(x),'f')

# This will be very slow
In [23]: for i in range(len(x)):
   ....:   sx[i] = sin(x[i])
   ....:

# This is very fast: Numeric applies the sin() operation to the whole 
# x array, but the looping is done in C instead of python.
In [24]: sin x
-------> sin (x)
Out[24]: array([  0.00000000e+00,   3.42020143e-01,   6.42787610e-01,   
8.66025404e-01,
              9.84807753e-01,   9.84807753e-01,   8.66025404e-01,   
6.42787610e-01,
              3.42020143e-01,   1.22460635e-16])

# You can see that the result is identical:
In [25]: sx
Out[25]: array([  0.00000000e+00,   3.42020154e-01,   6.42787635e-01,   
8.66025388e-01,
              9.84807730e-01,   9.84807730e-01,   8.66025388e-01,   
6.42787635e-01,
              3.42020154e-01,   1.22460635e-16],'f')


So again, the morale is: at all costs avoid looping in python in 
performance-critical parts of the code, once you've reached the optimization 
stage. In those parts, get creative: weave.blitz, weave.inline, 
weave.ext_tools, SWIG & friends (SILOON, boost, etc) in order of increasing 
complexity.

Cheers,

f