[Tutor] improving speed using and recalling C functions

[Gabriele Brambilla]

Hi,

2014-04-10 13:05 GMT-04:00 Martin A. Brown <martin at linux-ip.net>:

>
> Hi there Gabriele,
>
>  : I have a program that is reading near 600000 elements from a
>  : file. For each element it performs 200 times a particular
>  : mathematical operation (a numerical interpolation of a function).
>  : Now these process takes near 8 hours.
>
> Sounds fun!  Here are some thoughts (though I may not have any solid
> answers, I hope these pointers are useful):
>
>   Are you sure (from profiling) that the numerical interpolation
>     is the bottleneck?
>
> I see that the time needed by the code increase when I increase the number
of these operation. It's the only thing I do in the code.
(I'm sorry but I don't know very well what profiling is)


>   What is the numerical operation?
>     1: Is the function implemented in Numpy? [0]
>     2: How about SciPy? [1]
>
> Is interp1d from scipy.interpolate.

I think is the best one


> Is there a domain-specific Python library that already does the
> computation you want to achieve?  If so, then you have "only" the
> format question--how to put your data into the format required by
> the library.  If not, then on to the next question you ask ...
>
> No, interp1d do exactly what I want.



>  : Creating a C function and calling it from the code could improve
>  : the speed? It could be useful that it contains both the
>  : mathematical operation and the for loop or only the mathematical
>  : operation?
>  :
>  : If it is a reasonable solution to try decreasing the
>  : computational time how can I implement this C function?
>
> It is certainly an option!  In the event that you cannot find
> something that is fast enough in Python already and you cannot find
> any C library that has Python bindings, then, whether you have to
> write your own, or you have a third party C library, you have
> several options for writing bindings.


> You can write your Python bindings using:
>
>   * ctypes: https://docs.python.org/2/library/ctypes.html
>   * cffi: http://cffi.readthedocs.org/en/release-0.8/
>   * cython: http://www.cython.org/
>   * roll your own C!
>
> Do you know if does a C library exist with an already implemented function
like interp1d?
I use Anaconda, is there an already implemented mode to do it in Anaconda?
Cython?


> It might be beyond the scope of this list (certainly beyond my
> capabilities) to provide recommendations on which avenue to
> take--but if you are asking, you may already have the tools you need
> to assess for yourself.  There is, however, quite a bit of
> experience loitering around this list, so perhaps somebody else will
> have a suggestion.
>
> -Martin
>
>  [0] http://www.numpy.org/
>  [1] http://www.scipy.org/
>
> --
> Martin A. Brown
> http://linux-ip.net/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20140410/2a5d1aaa/attachment.html>