Mailman 3 April 2002 - Cplusplus-sig

Profiling object instantiation... a preliminary report
by Greg Landrum June 26, 2002

June 26, 2002

Re-establishing context: I have an extension class that I've wrapped using both Boost and Cxx. I need to be able to pickle and depickle large numbers (1e5-1e6) of these objects quickly for my target application (storing them in a ZODB and querying the resulting database). I discovered that simple instantiation of 1e5 objects took dramatically longer in Boost than in Cxx (6.7 seconds vs 0.9 seconds). I pulled down a copy of Intel's vtune profiler[1] and profiled my Boost and Cxx wrapped libraries to find the origin of the huge performance difference. After fiddling around for a while figuring out the appropriate magic to make vtune actually work, I found something surprising. I had exposed 4 different constructors for my Mol class. What I failed to realize is that when you overload functions (at least constructors) in BPL, this is handled using exceptions. This was at least partially responsible for the tremendous slow down in the Boost wrapper of my code. When I commented out some constructors, rebuilt, and reprofiled my code I got a run time of 1.9 seconds, only twice the Cxx run time. I'm a lot happier with this, but I'll continue to do a bit of exploration to see if I can bring that down more. I'm rather enjoying vtune now that I've figure out what can be done with it. If anyone wants more detailed information, I'd be happy to provide it. -greg [1] The beta of vtune is available as a free download for linux and windows. These results are under Win2K using MSVC++ v6. ---- greg Landrum (greglandrum(a)earthlink.net) Software Carpenter/Computational Chemist

6 12

Re: operator[]
by David Abrahams May 22, 2002

May 22, 2002

__getitem__ can be used directly, but the signature for __setitem__ is: __setitem__(self, key, value): so you need a wrapper function: template <class Self, class Key, class Value> setitem(Self& self, Key const& key, Value const& value) { self[key] = value; } ... .def("__setitem__", &setitem<Vector2d,int,double>) ----- Original Message ----- From: "chuzo okuda" <okuda1(a)llnl.gov> To: "David Abrahams" <david.abrahams(a)rcn.com> Cc: "Martin Casado" <casado2(a)llnl.gov>; "Susan Hazlett" <shazlett(a)llnl.gov>; "Kathleen McCandless" <mccandless2(a)llnl.gov> Sent: Wednesday, April 24, 2002 7:38 PM Subject: operator[] > Dave, > Now, I have problem with operator[]. > > class Vector2d { > public: > Vector2d():x(0.0),y(0.0) {} > Vector2d(const double xval, const double yval): x(xval),y(yval) {} > ~Vector2d() {} > double getX() {return x;} > double getY() {return y;} > double& operator[](int i) {return i==0 ? x : y;} > > protected: > double x, y; > }; > > int main(int argc, char** argv) { > Vector2d v = Vector2d(1.0,2.0); > cout << v[0] << "; " << v[1] << endl; > v[0] = 5.0; v[1] = 9.0; > cout << v[0] << "; " << v[1] << endl; > } > > And operator[] in C++ support both read and write operation. > It is not clear how to write "__setitem__" and "__getitem__" like: > > .def("__setitem__", &Vector2d::operator[]) > .def("__getitem__", &Vector2d::operator[])

3 6

V2: wrapping int/double bug(?)
by Pearu Peterson May 14, 2002

May 14, 2002

Hi, It seems that V2 does not distinguish between python int and float types. For example, if one defines (note the ordering of int and double methods) m.add( boost::python::class_<GiNaC::ex>("ex") .def_init(boost::python::args<int>()) .def_init(boost::python::args<double const>()) .def("__add__",ex_add_double) .def("__add__",ex_add_int) ); where const GiNaC::ex ex_add_int(const GiNaC::ex& lh,int const rh); const GiNaC::ex ex_add_double(const GiNaC::ex& lh,double const rh); then in python: >>> ex(2) ex(numeric('2')) >>> ex(2.3) ex(numeric('2')) <--- bug: float was converted to int >>> ex(0)+2 ex(numeric('2.0')) <--- bug: int was converted to float >>> ex(0)+2.3 ex(numeric('2.2999999999999998224')) That is, definitions .def_init(boost::python::args<double const>()) and .def("__add__",ex_add_int) are ignored. Any ideas how to fix this? Regards, Pearu

2 6

V2: coerce method howto?
by Pearu Peterson May 1, 2002

May 1, 2002

Hi, Consider the following coerce method template boost::python::tuple ex_coerce(const GiNaC::ex & left, boost::python::ref right) { return boost::python::tuple(left,right); } but when trying to compile it I get boost/boost/python/objects.hpp: In member function `void boost::python::tuple::set_item(unsigned int, const T&) [with T = GiNaC::ex]': boost/boost/python/objects.hpp:74: instantiated from `boost::python::tuple::tuple(const First&, const Second&) [with First = GiNaC::ex, Second = boost::python::ref]' src/_ginac.cpp:29: instantiated from here boost/boost/python/objects.hpp:100: `make_ref' undeclared (first use this function) boost/boost/python/objects.hpp:100: (Each undeclared identifier is reported only once for each function it appears in.) interrupted I guess that it is an issue of not-implemented-feature but I am not sure. So, what is the recommended way to implement coerce method within BPL V2? Pearu

2 5

Re: [C++-sig] crash with gcc 3.0.4
by Robert Andre April 30, 2002

April 30, 2002

>> 1) exceptions sometimes loose their type information so error >> messages are lost (weird). > >It's a long-standing issue with Boost.Python that there is no >generalized mechanism for C++ -> Python exception translation. The only >way to be sure that /some/ information will be preserved is to derive >C++ exceptions from std::exception, and to make sure the what() string >gives you some useful information. I plan to address this problem for >Boost.Python v2 in Q3 of this year. The weird part is that I do derive from std::exception but the information that it is derived from std::exception is somehow lost while unwrapping the call stack. This only happens with cross modules and only for some of the exceptions. >Shot in the dark: if you are having trouble with exceptions thrown >across shared library boundaries, it might help if the shared libs were >explicitly linked to one another. That's an idea though slightly contrary to the purpose of cross modules. >?? What's the difference between a shared and a dynamic library ?? on linux (or elf systems) none as far as I know, on mac osx the shared libraries are different then the dynamically loadable kind which are called bundles -- not sure about other systems. >The Boost.Python v1 cross-module support is somewhat ad-hoc, and >development is proceeding full-speed on Boost.Python v2. Rather than >trying to address this kind of problem in the v1 sources, I plan to >release v2 as a replacement for v1. ok, my guess is that the segfault problem I am having has to do with either how I am building the libraries or more likely a bug in gcc stringstream or the fact that I am compiling with gcc3 on systems setup for gcc2. I've seen it crash in instringstream also so it's not a boost thing. I'm just surprised nobody else has encountered it. We haven't really decided whether to go the boost.python, jython, swig or some other route yet but in case we do use boost it would be helpful if you could keep the strstream option available in v2,v3... so that we can support gcc 3.0 users. thanks dave, rob

2 1

Re: Re: [C++-sig] Boost iterators
by David Abrahams April 26, 2002

April 26, 2002

----- Original Message ----- From: "Martin Casado" <casado2(a)llnl.gov> > I'm not sure how you would expose vector to act like an iterator since > interators need to know the current position, so when you call next() > the position is incremented, bounds are checked, and the correct > exception is thrown if the bounds are overrun. Yeah, good point. > It seems to me that you > would have to write a C++ wrapper around vector<int> and any other type you > would want to expose as an iterator to provide this functionality. Or > add a method, __iter__() which returns a type (like the one I created) > that provides the functionality. > > Wouldn't it be simple to have this done within boost so you don't have > to modify the C++ classes? Sure; I'm just disputing the approach. Rather than adding cruft to each function which returns a vector<int>, What I think you want is a def_iterator() function on class_<>. Then you just write: .add(class_<std::vector<int> >("int_vector") .def_iterator()) or possibly a container_class<...> template derived from class_<...> which does the def_iterator thing for you. If you write this once, you should be able to return and accept vector<int> in many functions. -Dave

2 3

crash with gcc 3.0.4
by Robert Andre April 26, 2002

April 26, 2002

I had previously posted a message which contained some problems I had been experiencing with boost/gcc/linux. The suggestion of using the latest python 2.2 release fixed the pickle test failures - thanks. The other two problems are not boost related but irritating nonetheless: 1) exceptions sometimes loose their type information so error messages are lost (weird). 2) segmentation fault in ostringstream called from boost when using cross module support. I can't seem to reduce the first to a simple test case but the segmentation fault wasn't too hard. It fails on both a linux mandrake kernel 2.2/gcc 3.0.4 and linux redhat 7.1/gcc 3.0.2 (rpm), both dual processor machines. For anybody who is interested in it I posted a small set of files on http://w3.pppl.gov/~randre/bomb.tgz which builds two shared libraries and two dynamic libraries. You have to change a couple of lines in the makefile to reflect the location of your includes and library. One python script runs (ok.py) and the other crashes (bomb.py) depending on the order of loading of the external modules. rob

2 1

V2: Segmentation fault with gcc-2.95.4 but not with gcc-3.0.4
by Pearu Peterson April 23, 2002

April 23, 2002

Hi! I am getting segmentation faults when trying to import gcc-2.95.4 compiled extension module (the source is included at the end of this message) while gcc-3.0.4 compiled module works fine. I am using latest BPL V2 from CVS on Debian Woody, Python 2.2.1c. gdb shows (gdb) run Starting program: /usr/bin/python2.2 (no debugging symbols found)...(no debugging symbols found)... (no debugging symbols found)...[New Thread 1024 (LWP 5716)] (no debugging symbols found)... (no debugging symbols found)...(no debugging symbols found)... Python 2.2.1c1 (#1, Mar 15 2002, 08:13:47) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. (no debugging symbols found)...(no debugging symbols found)... >>> import gs1 (no debugging symbols found)...(no debugging symbols found)... Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1024 (LWP 5716)] 0x400ce64b in PyModule_Type () from /usr/lib/libpython2.2.so.0.0 Compilation flags for gcc: -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -ftemplate-depth-20 Compiling with gcc, linking with g++. Any ideas? Note that I would not mind using gcc-3.x but then for my application I would need to recompile a bunch of libraries (ginac,cln) that otherwise come with the distribution and are compiled with gcc-2.95.4. Therefore, I would first like to try to get BPL V2 working with gcc-2.95.x, if possible. Thanks, Pearu // File gs1.cpp #include <string> #define BOOST_PYTHON_DYNAMIC_LIB #define BOOST_PYTHON_V2 #include <boost/python/module.hpp> namespace { std::string greet() { return "hello, world"; } int square(int number) { return number * number; } } BOOST_PYTHON_MODULE_INIT(gs1) { using namespace boost::python; module m("gs1"); m .def("greet", greet) .def("square", square) ; }

3 6

memory leaks when writing Python extension in MSVC6
by Andranik Araratyan April 19, 2002

April 19, 2002

Hi I am writing an extension module for Python 2.2.1 with MSVC6.0 , WinNT. I get memory leaks after only importing that module in Python. This doesn't depend on what you're doing inside the module. I tried to localize the problem to the following If we take just the following example module (the only source file of that module DLL) =================================== #include "stdafx.h" #include "Python.h" static PyObject* ex_foo(PyObject *self, PyObject *args) { printf("Hello, world\n"); Py_INCREF(Py_None); return Py_None; } static PyMethodDef example_methods[] = { {"foo", ex_foo, 1, "foo() doc string"}, {NULL, NULL} }; extern"C" void initexample(void) { Py_InitModule("example", example_methods); } ========================================== and after just calling the Python's "import example" directive (the loading is successful) and closing it, I got an impressive amount of memory leaks, which disappear if you just exclude #include "stdafx.h" line from above text. Actually, if you even create an empty DLL file from single source file, containing just that single line #include "stdafx.h" and nothing more, you get the same memory leaks (of course, with the difference that Python "import" statement fails in this case). Is there any way to create an extension module with MFC support not to produce those memory leaks?

1 0

V2 enum convertors
by Peter Bienstman April 18, 2002

April 18, 2002

>So, let's discuss the interface. Seems to me something like class_<> >would be good: >enum MyEnum { one = 1, two, three }; >module("m") > .add(enum_<MyEnum>("MyEnum") > .value("one", one) > .value("two", two) > .value("three", three) > ) No objections here. This is probably as concise and as clear as you can get. Peter

4 9