[Numpy-discussion] Showing by examples how Python-Numpy can be efficient even for computationally intensive tasks

[Jerome Kieffer]

On Thu, 26 Nov 2020 22:14:40 +0100 (CET)
PIERRE AUGIER <pierre.augier at univ-grenoble-alpes.fr> wrote:

> I changed the email subject because I'd like to focus less on CO2 (a very interesting subject, but not my focus here) and more on computing...
> 

Hi Pierre,

We may turn the problem in another way around: one should more focus on
the algorithm than on the programming language. 

I would like to share with you one example, where we published how to
speed-up crystallographic computation written in Python. 
https://onlinelibrary.wiley.com/iucr/doi/10.1107/S1600576719008471

One referee asked us to validate vs C and Fortran equivalent code.
C code was as fast as Pythran or Cython and Fortran was still faster
(the std of the Fortran-compiled runtime was much smaller which allows
Fortran to be faster by 3 std !)

But I consider the difference to be marginal at this level !

If one considers the "Moore law", i.e. the time needed for
"performance" to double in different aspects of computing, one gets
18 to 24 month for the number of transistor in a processor, 18 years
for the compilers and 2 years (in average) for the development of new
algorithms. In this sense one should more focus on the algorithm used.

The Table 1 of the article is especially interesting: Pure Python is
10x slower than proper Numpy code, and parallel Pythran is 50x faster
than Numpy (on the given computer) but using the proper algorithm, i.e.
FFT in this case, is 13000x faster !

So I believe that Python, with its expressivity, helps much in
understanding the algorithm and hence to design faster code.

Cheers,

Jerome