[C++-sig] profiling C++ extensions

[greg Landrum]

Thanks for the response Dave.  I've posted to the python list and will see 
if I get anything useful.  Based upon some googling I did this morning, I'm 
not hopeful.

However, I did a bit of screwing around and I have narrowed my depickling 
problems down a bit.  I'm now pretty sure that the inefficiency is taking 
place in the Boost layer.  Here's why I think this.

I've got a C++ library which I've wrapped with both Boost and CXX (I did 
the CXX wrapper long ago and then switched to Boost because I like it a 
*lot* better than CXX).  I'm storing instances of this class (molecules) in 
ZODB, which means I have to be able to pickle it.  That's no problem... 
I've got the pickling stuff working just fine.  However, depickling 
molecules is really slow.  I wanted to do the profiling to track down the 
source of this slowness.

What I did instead is set up some boundary cases.  First I tried depickling 
1e5 2-tuples, to get a feeling for pickle overhead, this takes about 1.4 
seconds on my machine.  Then I tried instantiating 1e5 instances of 
a  python class (just an empty class).  This takes about 0.3 seconds.  Then 
I tried instantiating 1e5 instances of my molecule class using its default 
constructor (which doesn't really do any work).  If I use the CXX version 
of the library, this takes 0.9 seconds.  Doing the same thing with the 
Boost wrapped version of the library takes 6.7 seconds.  This is a killer 
for me.  I need to be able to, on a regular basis, loop through 1e5 or 1e6 
molecules and this object construction overhead is overwhelming the actual 
processing I'm doing.  (the bright side is that I don't need to worry about 
optimizing my own code any further hahahahah <whimper>)

Dave, I'm happy to do what I can to figure out how to solve this problem 
(I'd really rather keep using Boost), but I'd like to get some feeling from 
you as to whether or not it's even solvable.

The timing information here was collected on a Win2K machine using MSVC++6, 
Boost 1.24 and Python 2.1.

Thanks,
-greg