C-extension 2 times slower than exe

[Rolf Wester]

Hello,

thank you all very much for your replies.

I tried to simplify things and make the two versions as comparable as
possible. I put the C++ part of the program into a shared object
libff.so. For the exe the main function is linked against this shared
object. For the python stuff I made an interface consiting of only one
function call_solver with the same code that has the main function used
for the exe. Then I created a wrapper for this interface using swig and
linked interface.o, ff_warp.o and libff.so into _ff.so. The Python code
just imports _ff and calls call_solver wich creates an object of the
class Solver and calls its member solve (the main function of the exe
does the same).

I included some code for timing into the C++-code.

#include <time.h>

//beginning of solve
clock_t t0 = clock();
...
clock_t t1 = clock();
//end of solve
cout << "time used = " << (t1-t0)/CLOCKS_PER_SEC << endl;

I'm using gcc4.5 (latest snapshot) and Python2.6 under Suse 10.3. The
sources are compiled using the flags -fPIC -O3.

Timing:

1) time python ff.py
time used = 3.74
real    0m3.234s
user    0m3.712s
sys     0m0.192s
2) time ff
time used = 2.19
real    0m3.170s
user    0m2.088s
sys     0m0.168s

I tried some other optimizations:

-O0:

1)
time used = 21.91
real    0m21.568s
user    0m22.001s
sys     0m0.160s

2)
time used = 20.87
real    0m22.206s
user    0m20.989s
sys     0m0.148s

-O1
1)
time used = 4.04
real    0m3.660s
user    0m4.000s
sys     0m0.160s

2)
time used = 2.72
real    0m3.454s
user    0m2.648s
sys     0m0.164s

It seems that there is an overhead of about 2 secs within the C++-code.
I'm going to try this out with a problem that takes much more time than
the present one.

Regards

Rolf

Carl Banks wrote:
> On Jun 23, 7:20 am, Rolf Wester <rolf.wes... at ilt.fraunhofer.de> wrote:
>> Philip Semanchuk wrote:
>>
>>> On Jun 23, 2009, at 9:51 AM, Rolf Wester wrote:
>>>> Hi,
>>>> I have a C++ program that I would like to steer using Python. I made the
>>>> wrapper using swig and linked the code (without the main function) into
>>>> a shared object. My Python script loads the extension and calls a
>>>> function of the C-extension, the rest runs entirely within the
>>>> C-extension. For comparison I compiled the code including the main
>>>> function with the same compilation options and linked all into an exe.
>>>> The main function of the exe calls the same function as my Python script
>>>> does. Surprisingly the code in the Python C-extension runs twice as long
>>>> as the same code in the exe. Does anyone know what could be the reason
>>>> for this behaviour?
>>> If the runtime of the C/C++ code is short, the time spent initializing
>>> the Python interpreter might have a big impact on the runtime of the
>>> Python version.
>> The runtime is about 2.5 sec and 5.0 sec respectively. I not only use
>> the time command to measure the time consumed but I also measure the
>> time within the C-code using clock() and get similar result. So the
>> Python startup time is not the reason for the runtime difference I
>> found. Thank you anyway.
> 
> We can't really help you much unless you give us more details about
> what you did (how did you built it, how did you time it, and how are
> you calling the C extension from Python).  All we can do is throw out
> possible ideas, and I will toss out a few, but if that's not it you'll
> have to post details.
> 
> Are you building the extension with the same optimization flags as the
> compiler?
> 
> Are you calling the C code repeatedly from inside a Python loop?
>