[Tutor] Cython question

[Stefan Behnel]

Alan Gauld, 02.07.2011 15:28:
> "Albert-Jan Roskam" wrote
>> I used cProfile to find the bottlenecks, the two Python functions
>> getValueChar and getValueNum. These two Python functions simply call two
>> equivalent C functions in a .dll (using ctypes).

The code is currently declared as Windows-only and I don't know any good 
C-level profiling code for that platform. Under Linux, once I'm sure I have 
a CPU bound problem below the Python level, I'd use valgrind and 
KCacheGrind to analyse the performance. That will include all C function 
calls (and even CPU instructions, if you want) in the call trace. Makes it 
a bit less obvious to see what Python is doing, but leads to much more 
detailed results at the C level.

It's also worth keeping in mind that all profiling attempts *always* 
interfere with the normal program execution. The results you get during a 
profiling run may not be what you'd get with profiling disabled. So, 
profiling is nice, but it doesn't replace proper benchmarking.


> In that case cythin will speed up the calling loops but it can't do
> anything to speed up the DLL calls, you have effectively already optimised
> those functions by calling the DLL.
>
>> The problem is that these functions are called as many times as there are
>> VALUES in a file
>
> It might be worth a try if you have very big data sets
> because a C loop is faster than a Python loop. But don't expect order of
> magnitude improvements.

Looking at the code now, it's actually worse than that. The C function call 
does not only go through ctypes, but is additionally wrapped in a method 
call. So the OP is paying the call overhead twice for each field, plus the 
method lookup and some other operations. These things can add up quite easily.

So, iff the conversion code is really a CPU bottleneck, and depending on 
how much work the C functions actually do, the current call overhead, 100 
times per record, may be a substantial part of the game. It's worth seeing 
if it can be dropped at the Python level by removing method lookup and call 
levels (i.e. by inlining the method), but if that's not enough, Cython may 
still be worth it. For one, Cython's call overhead is lower than that of 
ctypes, and if the call is only done once, and the loop is moved into 
Cython (i.e. C) entirely, the overhead will also drop substantially.

It might also be worth running the code in PyPy instead of CPython. PyPy 
will optimise a lot of the overhead away that this code contains.


>> So if I understand you correctly, this is not Cpu bound

I don't have enough information to comment on that.


> It may still be CPU bound in that the CPU is doing all the work, but if
> the CPU time is in the DLL functions rather than in the loop cython
> won't help much.
>
> CPU bound refers to the type of processing - is it lots of logic, math,
> control flows etc? Or is it I/O bound - reading network, disk, or user
> input? Or it might be memory bound - creating lots of in memory objects
> (especially if that results in paging to disk, when it becomes I/O
> bound  too!)
>
> Knowing what is causing the bottleneck will determine how to improve
> things. Use tools like TaskManager in Windows or top in *nix to see
> where the time is going and what resources are being consumed. Fast code
> is not always the answer.

That is very good advice. As a rule of thumb, a process monitor like top 
will tell you how much time is spent in I/O and CPU. If, during a test run 
(with profiling disabled, as that eats time, too!), your CPU usage stays 
close to 100%, your program is CPU bound. If, however, it stays lower, and 
the monitor reports a high I/O waiting time, it's I/O bound. In this case, 
I/O bound is what you want to achieve, because it means that your code is 
running faster than your hard drive can deliver the data.

Stefan