Mailman 3 February 2010 - Cplusplus-sig

boost::python embedding error - runtime - Mac OS X - 1.38
by Igor Karpov Oct. 1, 2014

Oct. 1, 2014

Hi, I am trying to run the following program: #include <iostream> #include <Python.h> #include <boost/python.hpp> using namespace boost::python; using namespace std; int main(int argc, char** argv) { { // Using Python/C Py_Initialize(); PyObject* main_module = PyImport_ImportModule("__main__"); PyObject* globals = PyEval_GetGlobals(); PyObject* locals = PyEval_GetLocals(); PyRun_SimpleString("print 'Hello World, from Python/C!'\n"); Py_Finalize(); } { // Using boost::python Py_Initialize(); object main_module = import("__main__"); object main_namespace = main_module.attr("__dict__"); try { object ignored = exec("print 'Hello World, from boost::python!'\n", main_namespace); Py_Finalize(); } catch (error_already_set const& e) { PyErr_Print(); return 1; } } return 0; } on Linux (both x86 and x86_64), with Ubuntu's default version of Boost (1.34.1) installed, I am able to do so as expected: $ ./embed Hello World, from Python/C! Hello World, from boost::python! on Mac OS X, I get a runtime error: $ ./embed Hello World, from Python/C! Bus Error Running with gdb, this is the stack trace: Hello World, from Python/C! Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x00000000 0x00000000 in ?? () (gdb) where #0 0x00000000 in ?? () #1 0x00bb618f in PyEval_GetGlobals () #2 0x00bce0dd in PyImport_Import () #3 0x00bce2b0 in PyImport_ImportModule () #4 0x008127d4 in boost::python::import (name=@0xbffff4a8) at libs/python/src/import.cpp:20 #5 0x00004446 in main (argc=1, argv=0xbffff510) at embed.cc:22 For this Mac, boost version is latest (1.38) and python version is: Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Is this a bug? I attached the source and the cmake file I am using to build, for convenience. Thanks, --Igor.

4 6

CMake and getting starting with boost.python
by Braddock Gaskill Aug. 5, 2011

Aug. 5, 2011

Yesterday I became a boost.python user, after years of heavy SWIG usage. I would have started using boost.python years ago except for the bjam learning curve. I spent hours TRYING on multiple occasions, but got lost every time trying to get the page one example to build with bjam. Here is how to build the "greet" example with a four line CMake file: cat >>CMakeLists.txt <<EOF LINK_LIBRARIES(boost_python) INCLUDE_DIRECTORIES("/usr/include/python2.5") LINK_DIRECTORIES("/usr/lib/python2.5") # Not needed? ADD_LIBRARY(hello SHARED hello.cpp) EOF You WILL have to `mv libhello.so hello.so` after building. Works for me under Ubuntu Linux 6.10 after apt-get install cmake python2.5-dev boost-python-dev cmake make mv libhello.so hello.so ipython In [1]: import hello In [2]: hello.greet() Out[2]: 'hello, world' Boost.Python has been a joy to learn and use since I got past bjam. I had a moderately large API wrapped in just my first couple hours of learning (less time than I spent messing with bjam). Any additional advice for using CMake to build boost.python? Braddock Gaskill

4 4

How to import a boost::python dll in windows?
by ichijoji Aug. 5, 2011

Aug. 5, 2011

I'm trying to port some research code that uses boost::python to expose a library to python to windows, but I'm having trouble importing the dll into python. I'm compiling using msvc 2005 and using python 2.5, but when I try to import the module I get this: Traceback (most recent call last): File "main.py", line 1, in <module> import object File "c:\code\sge\code\scripts\object.py", line 1, in <module> import sge ImportError: No module named sge My module is named sge and I'm compiling to sge.dll. Is there some obvious reason why python wouldn't be able to find my dll? When I compile on linux into an so this code works fine. -- View this message in context: http://www.nabble.com/How-to-import-a-boost%3A%3Apython-dll-in-windows--tf3… Sent from the Python - c++-sig mailing list archive at Nabble.com.

4 4

profiling python extension
by Alexey Akimov March 1, 2010

March 1, 2010

Dear all, Does anyone know how one may profile a python extention? There is a bunch of literature on profiling of pure binaries (programs and even libraries), pure python code, etc. However the profiling of a python module from python script seems a tricky procedure. What I tried is to pass variant=profile to bjam. Everything works fine and I get .so library corresponding to my module (placed into /profile sub-directory of this particular build). When I execute the python script invoking my module's functions (performance of those functions is most interesting for me) there is no any output generated (something similar to gmon.out). So that is basically my question - how can I retrieve the profiling information about my extension? Of course I can compile a normal executable using the functions in the module and then use a standard approach (and of course gprof), but I hope that there should be more straightforward and easy way to do this (hopefully by passing some arguments to bjam). Thanks in advance. Alexey

4 7

Re: [C++-sig] profiling python extension
by Amos Anderson Feb. 28, 2010

Feb. 28, 2010

> Date: Sat, 27 Feb 2010 21:52:17 -0600 > From: Alexey Akimov <alexey.akimov85(a)gmail.com> > To: "Development of Python/C++ integration" <cplusplus-sig(a)python.org> > Subject: [C++-sig] profiling python extension > Message-ID: > <382d227f1002271952p25858f1eyeb1c495bd282f2b(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Dear all, > > Does anyone know how one may profile a python extention? There is a bunch of > literature on profiling of pure binaries (programs and even libraries), pure > python code, etc. However the profiling of a python module from python > script seems a tricky procedure. > What I tried is to pass variant=profile to bjam. Everything works fine and I > get .so library corresponding to my module (placed into /profile > sub-directory of this particular build). When I execute the python script > invoking my module's functions (performance of those functions is most > interesting for me) there is no any output generated (something similar to > gmon.out). So that is basically my question - how can I retrieve the > profiling information about my extension? Of course I can compile a normal > executable using the functions in the module and then use a standard > approach (and of course gprof), but I hope that there should be more > straightforward and easy way to do this (hopefully by passing some arguments > to bjam). > Thanks in advance. > > Alexey If you're using OSX, then the Shark tool seems to work really well, and is really easy to use. I does not require that you compile in -pg mode. http://developer.apple.com/tools/shark_optimize.html Amos.

3 3

incorrect linking to libpython
by Amos Anderson Feb. 27, 2010

Feb. 27, 2010

Hello -- summary: I'm trying to compile the boost python library using a custom installed python on osx. however, it appears to be linking to the system library, not the installed library. I'm using OSX 10.6.2, which has python 2.6.1 by default. I wanted to try to install 2.6.4 just to see how the process works. I To install python, I did this: ./configure --enable-framework make make install which put the distribution here: /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 I changed a boost config file to read: using python : 2.6 : /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 : /Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 : /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/config ; and compiled the library. However, when I examine the library itself: % otool -L libboost_python.dylib libboost_python.dylib: libboost_python.dylib (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.0) I can see that it linked to the system's version. Why did it choose that one? The problem is that when I run my code, I get the error: Fatal Python error: Interpreter not initialized (version mismatch?) ../../../triad.sh: line 9: 59136 Abort trap python $@ the OS gives me a popup window mentioning all the linked libraries: Binary Images: 0x100000000 - 0x100000ff7 +org.python.python 2.6.4 (2.6.4) <536A2002-9AEC-D7DF-8C2B-6C634FB7F37E> /Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python 0x100003000 - 0x100153fe7 +org.python.python 2.6.4, (c) 2004-2008 Python Software Foundation. (2.6.4) <3B4D8CE1-8D9C-97CB-57BA-A8F4C40CE5F0> /Library/Frameworks/Python.framework/Versions/2.6/Python 0x1002e7000 - 0x1002e8ff7 +time.so ??? (???) <CB23D26C-1417-4D31-9E06-1E33CCDCA83C> /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/time.so 0x100480000 - 0x100493fef +libgcc_s.1.dylib ??? (???) <E51F2952-8BEB-C427-10A9-672FC10E3A54> /usr/local/lib/libgcc_s.1.dylib 0x100753000 - 0x10077bfff +libboost_wserialization.dylib ??? (???) <4FD94C16-75D0-DE5D-A27B-E3AEAEEA8CCC> /Users/amos/triad/trunk/lib/libboost_wserialization.dylib 0x101000000 - 0x1013eeff7 +triad.so ??? (???) <EEF65664-60F0-E721-06A8-82F88CEA1EFC> /Users/amos/triad/trunk/lib/triad.so 0x1020eb000 - 0x102124fef +libboost_serialization.dylib ??? (???) <33DB6DD1-4712-AEE1-0761-18054566A5E4> /Users/amos/triad/trunk/lib/libboost_serialization.dylib 0x1021b1000 - 0x1021f0ff7 +libboost_python.dylib ??? (???) <67258016-C11E-8EDF-BF36-103DBC219C78> /Users/amos/triad/trunk/lib/libboost_python.dylib 0x10223e000 - 0x10234eff7 org.python.python 2.6.1 (2.6.1) <01EBD0D4-F181-045F-C7A8-B7B3C5D4204F> /System/Library/Frameworks/Python.framework/Versions/2.6/Python 0x7fff5fc00000 - 0x7fff5fc3bdef dyld 132.1 (???) <B633F790-4DDB-53CD-7ACF-2A3682BCEA9F> /usr/lib/dyld 0x7fff813dc000 - 0x7fff81459fef libstdc++.6.dylib ??? (???) <35ECA411-2C08-FD7D-11B1-1B7A04921A5C> /usr/lib/libstdc++.6.dylib 0x7fff81f2c000 - 0x7fff81f3dfef libz.1.dylib ??? (???) <3A7A4C48-A4C8-A78A-8B87-C0DDF6601AC8> /usr/lib/libz.1.dylib 0x7fff846e0000 - 0x7fff846e4ff7 libmathCommon.A.dylib ??? (???) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5> /usr/lib/system/libmathCommon.A.dylib 0x7fff888bb000 - 0x7fff88a79ff7 libSystem.B.dylib ??? (???) <526DD3E5-2A8B-4512-ED97-01B832369959> /usr/lib/libSystem.B.dylib 0x7fffffe00000 - 0x7fffffe01fff libSystem.B.dylib ??? (???) <526DD3E5-2A8B-4512-ED97-01B832369959> /usr/lib/libSystem.B.dylib Where i've deleted all the lines with my libraries. Note: I can do this: /Library/Frameworks> sudo mv Python.Framework temp.Python.Framework and then everything works fine. So I assume that the problem is the one python file 2.6.1 which didn't come from the same python path as the other python paths. When I compile, I get this: % bjam -d 2 release ... deleted similar lines .... "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6" -I"boost/boost_1_40_0" -c -o "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/import.o" "boost/boost_1_40_0/libs/python/src/import.cpp" darwin.compile.c++ boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/exec.o "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6" -I"boost/boost_1_40_0" -c -o "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/exec.o" "boost/boost_1_40_0/libs/python/src/exec.cpp" darwin.compile.c++ boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/function_doc_signature.o "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6" -I"boost/boost_1_40_0" -c -o "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/function_doc_signature.o" "boost/boost_1_40_0/libs/python/src/object/function_doc_signature.cpp" darwin.link.dll boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/libboost_python.dylib "g++" -dynamiclib -Wl,-single_module -install_name "libboost_python.dylib" -L"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/config" -o "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/libboost_python.dylib" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/numeric.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/list.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/long.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/dict.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/tuple.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/str.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/slice.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/converter/from_python.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/converter/registry.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/converter/type_id.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/enum.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/class.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/function.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/inheritance.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/life_support.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/pickle_support.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/errors.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/module.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/converter/builtin_converters.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/converter/arg_to_python_base.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/iterator.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/stl_iterator.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object_protocol.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object_operators.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/wrapper.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/import.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/exec.o" "boost/boost_1_40_0/bin.v2/libs/python/build/darwin-4.2.1/release/object/function_doc_signature.o" -lpython2.6 -headerpad_max_install_names -Wl,-dead_strip -no_dead_strip_inits_and_terms So again we can see that it's finding the header files for each object file, but then when they're all being linked, it's seems to be skipping the library that (I think) it should be picking: /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/config> ls -lrt total 208 -rw-r--r-- 1 root admin 3.7K Feb 26 12:25 python.o -rwxr-xr-x@ 1 root admin 7.3K Feb 26 12:25 makesetup* -rwxr-xr-x@ 1 root admin 7.0K Feb 26 12:25 install-sh* -rw-r--r--@ 1 root admin 1.4K Feb 26 12:25 config.c.in -rw-r--r-- 1 root admin 2.0K Feb 26 12:25 config.c -rw-r--r-- 1 root admin 41B Feb 26 12:25 Setup.local -rw-r--r-- 1 root admin 368B Feb 26 12:25 Setup.config -rw-r--r--@ 1 root admin 18K Feb 26 12:25 Setup -rw-r--r-- 1 root admin 42K Feb 26 12:25 Makefile lrwxr-xr-x 1 root admin 15B Feb 26 12:25 libpython2.6.a@ -> ../../../Python I tried changing the name of the library to something else more unique and manually ran the link line (outside of bjam), and I *can* link to it, so that's not the problem. Why isn't it choosing the -L selected libpython2.6.a? Or maybe I'm doing something else wrong. Or maybe it's Apple's fault, or maybe I need to update boost from 1.40. amos.

2 1

Boost python exception translation failure on BlueGene/P
by James Amundson Feb. 25, 2010

Feb. 25, 2010

Boost Python seems to be failing to catch and translate C++ exceptions for me on a BlueGene/P installation. I really don't know what to do next in debugging the problem, so I would appreciate any advice. I have a simple test module, pyexcept ------------------------------------------------------------ #include <boost/python.hpp> #include <iostream> #include <stdexcept> void foo() { std::cout << "in foo, about to raise exception\n"; throw std::runtime_error("runtime_error from foo"); std::cout << "this should never be seen\n"; } using namespace boost::python; BOOST_PYTHON_MODULE(pyexcept) { def("foo",foo); } ------------------------------------------------------------ I test it with the following python script: ------------------------------------------------------------ #!/usr/bin/env python import pyexcept print "about to run pyexcept.foo, catching exception" try: pyexcept.foo() except RuntimeError,e: print "caught RuntimeError,",e ------------------------------------------------------------ On my Linux machine I see: ------------------------------------------------------------ about to run pyexcept.foo, catching exception in foo, about to raise exception caught RuntimeError, runtime_error from foo ------------------------------------------------------------ On the BlueGene/P machine I see: ------------------------------------------------------------ about to run pyexcept.foo, catching exception in foo, about to raise exception terminate called after throwing an instance of 'std::runtime_error' what(): runtime_error from foo ------------------------------------------------------------ I have (cross-)compiled boost myself using the system's installed version of the gnu compilers, 4.1.2: ------------------------------------------------------------ |login2>mpicxx.gnu --version powerpc-bgp-linux-g++ (GCC) 4.1.2 (BGP) ------------------------------------------------------------ (The mpicxx.gnu script is a wrapper around the cross-compiling g++.) Has anyone seen a problem like this? Any ideas as to how to debug it? Thanks for any advice. --Jim Amundson

3 5

Duplicate class and method definition
by André Prins Feb. 22, 2010

Feb. 22, 2010

Hi All, Recently, I had some problems with a duplicate method and class definition begin accepted by boost::python (version 1.41 on MSVC). Consider the following c++-code, which declares a class Foo twice, but with different bindings for the same function. /* C++ code: */ #include <iostream> #include <boost/python.hpp> using namespace boost::python; struct Foo { void f( int m ){ std::cerr << "F set to " << m << std::endl; } void g( int m ){ std::cerr << "G set to " << 10*m << std::endl; } }; BOOST_PYTHON_MODULE( FooPython ) { // First definition binding 'g' class_< Foo, boost::noncopyable >( "Foo" ) .def( "func", &Foo::f ) ; // Second definition binding 'f' class_< Foo, boost::noncopyable >( "Foo" ) .def( "func", &Foo::g ) ; } # Python-code import FooPython f = FooPython.Foo() f.func( 3 ) This outputs the line "G set to 30" and the original function-binding (to f) is gone. When I remove the boost::noncopyable when creating the python-binding Foo, I get an assert upon importing FooPython: file: libs\python\src\converter\registry.cpp @ line 212 expression: slot->m_to_python == 0 It is obvious that this is not the intended use of Boost-python, but I am wondering: why does it assert at runtime without the noncopyable and why does it "more or less work" with the boost::noncopyable. And is there a method to detect such multiple definitions of the same class. Kind Regards, André

2 1

Re: [C++-sig] how do i interrupt a C++ extension?
by Amos Anderson Feb. 19, 2010

Feb. 19, 2010

>>> ----- Original Message ---- >>> From: Amos Anderson <nitroamos(a)gmail.com> >>> To: Development of Python/C++ integration <cplusplus-sig(a)python.org> >>> Sent: Wed, February 3, 2010 12:24:13 PM >>> Subject: [C++-sig] how do i interrupt a C++ extension? >>> >>> Hello -- >>> >>> I've got a python script with C++ extensions. Some of my extensions >>> take a long time to complete, and I don't want to wait for them to >>> finish when I'm debugging stuff. However, when I do Ctrl-C in my >>> terminal, it's completely ignored. So it looks like python is trapping >>> the signal, but apparently can't do anything with it until the >>> extension returns control to the python script. I guess ideally, >>> Ctrl-C would kill the extension and return control to python, >>> generating an exception, but I'd also be ok if Ctrl-C killed the >>> python script too. >>> >>> I've been googling around, but can't figure out how this seemingly >>> simple (and desired) task is accomplished. Anybody know how to do it? >>> Right now, the only solution is Ctrl-Z and kill %1 so I guess that >>> works for now... >>> >>> thanks! >>> >>> Amos. Thanks for all the suggestions, but I just figured out how to do what I want, and I figured I'd share it with everybody. This was my coworker's idea. I just wrap my extension in a separate Process, which can then be killed by the original process as needed. For example: from multiprocessing import Process ... p = Process(target=minimizeStructure, args=(filename,)) p.start() try: p.join() except: p.terminate() print "Error: early termination." and this does exactly what I need. Amos.

1 0

Hi, new around these parts + small link error
by Kevin Daley Feb. 17, 2010

Feb. 17, 2010

Hi, I'm new to these lists, but I'm an experienced developer. Right now, I'm working on a project using a popular game engine. I've got some Boost::Python code that's generating a bit of a link-time error; I was under the impression that constitutes most of the traffic around here. Using MSVC 9 and v.1_36 of the library (not my decision to use old version, unfortunately), I'm getting an error: Spetsnaz_AI.obj : error LNK2001: unresolved external symbol "struct _object * __cdecl boost::python::detail::init_module(char const *,void (__cdecl*)(void))" (?init_module@detail@python@boost@@YAPAU_object@ @PBDP6AXXZ@Z) I've got it down to where this is the only unresolved external. My export code is viewable: http://pastebin.com/m439861b0 . I'm not sure what the error implies. Is the init_module really an external symbol that gets linked? Or does that get defined in my code? I tried examining the definition of the BOOST_PYTHON_MODULE macro and it seemed to indicate the latter. Would appreciate any help or redirection. Many thanks, Kevin

1 0