From blake at hailmail.net  Thu Dec  2 23:23:02 2010
From: blake at hailmail.net (Dane Springmeyer)
Date: Thu, 2 Dec 2010 14:23:02 -0800
Subject: [C++-sig] help finding solution to lack of exception handling with
	boost python on open solaris 64 bit
Message-ID: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>

Hello,

I recently recompiled my C++ app (mapnik.org, which uses boost python for python binding) with -m64 flags for 64 bit operation and this broke the boost python exception handling.

Instead of C++ exceptions being propagated to python exceptions, I get segmentation faults (core dump).

I have a long dependency chain that all needs to be 64 bit, so I'm really stuck unless I can find a way to restore boost python's ability to handle exceptions without crashing.

Exceptions are very common during setup of mapnik, and are needed to report basic configuration issues as a user gets set up.

I'm desperate to try anything that might help - anyone have any suggestions?

What i have tried so far:

1) recompiling all apps with -02 instead of -03
2) making sure all apps are compiled with the exact same version of gcc (at least all the C++ libraries I have compiled from source)
3) ensured that all dynamic libraries link to the same libgcc_s.so.1 and libstdc++.so.6.
4) tried with both boost 1.44 and 1.45

I'm out of ideas - anyone have suggestions to try?

More info below.

Dane

--------------

With 32bit boost, python, and mapnik, exceptions worked fine.

This is on opensolaris:

$ cat /etc/release
                      OpenSolaris Development snv_133 X86
          Copyright 2010 Sun Microsystems, Inc.  All Rights Reserved.
                       Use is subject to license terms.
                          Assembled 15 February 2010

with the sun provided 64 bit python26:

$ /usr/bin/amd64/python --version
Python 2.6.4

Both boost and mapnik are compiled with a *custom* compiled gcc 44:

$ /opt/ts/gcc/4.4/bin/g++ -v       
Using built-in specs.
Target: i386-pc-solaris2.11
Configured with: .././configure --prefix=/opt/ts/gcc/4.4 --bindir=/opt/ts/gcc/4.4/bin --libdir=/opt/ts/gcc/4.4/lib --mandir=/opt/ts/gcc/4.4/share/man --datadir=/opt/ts/gcc/4.4/share --includedir=/opt/ts/gcc/4.4/include --infodir=/opt/ts/gcc/4.4/share/info --libexecdir=/opt/ts/gcc/4.4/lib --sysconfdir=/etc/opt/ts --disable-nls --disable-static --with-gnu-as --with-as=/usr/gnu/bin/as --without-gnu-ld --with-ld=/usr/ccs/bin/ld --enable-threads=posix --enable-shared --enable-multilib --enable-nls --without-x --with-system-zlib --enable-languages=c,c++,f95,objc --with-mpfr=/opt/ts --with-gmp=/opt/ts
Thread model: posix
gcc version 4.4.4 (GCC) 

( I was unable to get boost compiled with the gcc 3.4.3 or gcc 4.3.3 provided by sun packages, for more info see: http://trac.mapnik.org/wiki/OpenSolarisInstallation/TroubleShooting)


So, instead of throwing an understandable exception, boost python crashes and this is what a normal backtrace looks like in gbd:

(gdb) bt
#0  0x000000000005f666 in ?? ()
#1  0xfffffd7fff2ec5d1 in _Unwind_RaiseException_Body () from /usr/lib/amd64/libc.so.1
#2  0xfffffd7fff2ec855 in _Unwind_RaiseException () from /usr/lib/amd64/libc.so.1
#3  0xfffffd7ffa98bb39 in __cxa_throw (obj=<value optimized out>, tinfo=0x1, dest=0x474e5543432b2b00)
   at ../../../../.././libstdc++-v3/libsupc++/eh_throw.cc:78
#4  0xfffffd7ff3aa2de2 in boost::python::throw_error_already_set () from /usr/local/lib/libboost_python.so.1.44.0
#5  0xfffffd7ff3a9ae31 in boost::python::objects::function::argument_error () from /usr/local/lib/libboost_python.so.1.44.0
#6  0xfffffd7ff3a9b4fb in boost::python::objects::function::call () from /usr/local/lib/libboost_python.so.1.44.0
#7  0xfffffd7ff3a9b730 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke () from /usr/local/lib/libboost_python.so.1.44.0
#8  0xfffffd7ff3aa3053 in boost::python::detail::exception_handler::operator() () from /usr/local/lib/libboost_python.so.1.44.0
#9  0xfffffd7ff28fb1b2 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<std::exception, void (*)(std::exception const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(std::exception const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke ()
  from /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
#10 0xfffffd7ff3aa2e25 in boost::python::handle_exception_impl () from /usr/local/lib/libboost_python.so.1.44.0
#11 0xfffffd7ff3a97d50 in function_call () from /usr/local/lib/libboost_python.so.1.44.0
#12 0xfffffd7ff6ff9aad in PyObject_Call () from /usr/lib/amd64/libpython2.6.so.1.0
#13 0xfffffd7ff700af29 in instancemethod_call () from /usr/lib/amd64/libpython2.6.so.1.0
#14 0xfffffd7ff6ff9aad in PyObject_Call () from /usr/lib/amd64/libpython2.6.so.1.0
#15 0xfffffd7ff705d8a1 in slot_tp_init () from /usr/lib/amd64/libpython2.6.so.1.0
#16 0xfffffd7ff7051c06 in type_call () from /usr/lib/amd64/libpython2.6.so.1.0
#17 0xfffffd7ff6ff9aad in PyObject_Call () from /usr/lib/amd64/libpython2.6.so.1.0
#18 0xfffffd7ff709c1c0 in do_call () from /usr/lib/amd64/libpython2.6.so.1.0
#19 0xfffffd7ff709b672 in call_function () from /usr/lib/amd64/libpython2.6.so.1.0
#20 0xfffffd7ff709810d in PyEval_EvalFrameExReal () from /usr/lib/amd64/libpython2.6.so.1.0
#21 0xfffffd7ff7094d9d in PyEval_EvalFrameEx () from /usr/lib/amd64/libpython2.6.so.1.0
#22 0xfffffd7ff709a27d in PyEval_EvalCodeEx () from /usr/lib/amd64/libpython2.6.so.1.0
#23 0xfffffd7ff7094caa in PyEval_EvalCode () from /usr/lib/amd64/libpython2.6.so.1.0
#24 0xfffffd7ff70bd385 in run_mod () from /usr/lib/amd64/libpython2.6.so.1.0
#25 0xfffffd7ff70bd27c in PyRun_StringFlags () from /usr/lib/amd64/libpython2.6.so.1.0
#26 0xfffffd7ff70bc53d in PyRun_SimpleStringFlags () from /usr/lib/amd64/libpython2.6.so.1.0
#27 0xfffffd7ff70ca5f0 in Py_Main () from /usr/lib/amd64/libpython2.6.so.1.0
#28 0x0000000000400b3c in _start ()


Here is the output of ldd for both the main mapnik library and the mapnik python bindings:

$ ldd /usr/local/lib/libmapnik.so
       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
       libz.so.1 =>     /lib/64/libz.so.1
       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
       libproj.so.0 =>  /usr/local/lib/libproj.so.0
       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
       libboost_filesystem.so.1.44.0 =>         /usr/local/lib/libboost_filesystem.so.1.44.0
       libboost_regex.so.1.44.0 =>      /usr/local/lib/libboost_regex.so.1.44.0
       libxml2.so.2 =>  /lib/64/libxml2.so.2
       libboost_thread.so.1.44.0 =>     /usr/local/lib/libboost_thread.so.1.44.0
       libboost_system.so.1.44.0 =>     /usr/local/lib/libboost_system.so.1.44.0
       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
       libm.so.2 =>     /lib/64/libm.so.2
       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
       libc.so.1 =>     /lib/64/libc.so.1
       libdl.so.1 =>    /lib/64/libdl.so.1
       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
       libpthread.so.1 =>       /lib/64/libpthread.so.1
       librt.so.1 =>    /lib/64/librt.so.1
       libsocket.so.1 =>        /lib/64/libsocket.so.1
       libnsl.so.1 =>   /lib/64/libnsl.so.1
       libmp.so.2 =>    /lib/64/libmp.so.2
       libmd.so.1 =>    /lib/64/libmd.so.1
       libscf.so.1 =>   /lib/64/libscf.so.1
       libuutil.so.1 =>         /lib/64/libuutil.so.1
       libgen.so.1 =>   /lib/64/libgen.so.1
       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1

$ ldd /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so 
       libmapnik.so =>  /usr/local/lib/libmapnik.so
       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
       libboost_python.so.1.44.0 =>     /usr/local/lib/libboost_python.so.1.44.0
       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
       libm.so.2 =>     /lib/64/libm.so.2
       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
       libz.so.1 =>     /lib/64/libz.so.1
       libproj.so.0 =>  /usr/local/lib/libproj.so.0
       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
       libboost_filesystem.so.1.44.0 =>         /usr/local/lib/libboost_filesystem.so.1.44.0
       libboost_regex.so.1.44.0 =>      /usr/local/lib/libboost_regex.so.1.44.0
       libxml2.so.2 =>  /lib/64/libxml2.so.2
       libboost_thread.so.1.44.0 =>     /usr/local/lib/libboost_thread.so.1.44.0
       libboost_system.so.1.44.0 =>     /usr/local/lib/libboost_system.so.1.44.0
       libc.so.1 =>     /usr/lib/amd64/libc.so.1
       librt.so.1 =>    /lib/64/librt.so.1
       libdl.so.1 =>    /lib/64/libdl.so.1
       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
       libpthread.so.1 =>       /lib/64/libpthread.so.1
       libsocket.so.1 =>        /lib/64/libsocket.so.1
       libnsl.so.1 =>   /lib/64/libnsl.so.1
       libmp.so.2 =>    /lib/64/libmp.so.2
       libmd.so.1 =>    /lib/64/libmd.so.1
       libscf.so.1 =>   /lib/64/libscf.so.1
       libuutil.so.1 =>         /lib/64/libuutil.so.1
       libgen.so.1 =>   /lib/64/libgen.so.1
       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1

Here is a simple way to prompt a segfault:

$ /usr/bin/amd64/python
Python 2.6.4 (r264:75706, Feb 14 2010, 14:06:28) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import mapnik
>>> m = mapnik.Map(256,256)
>>> m
<mapnik._mapnik.Map object at 0x55a1b0>
# then, prompt a boost python argument error
>>> m2 = mapnik.Map('foo')
Segmentation Fault




From charlessolar at gmail.com  Fri Dec  3 01:05:11 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Thu, 2 Dec 2010 18:05:11 -0600
Subject: [C++-sig] help finding solution to lack of exception handling
 with boost python on open solaris 64 bit
In-Reply-To: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
References: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
Message-ID: <AANLkTikUVGvgVGYANc+V1i+-s1gjOYjQvkukrRF2RLw3@mail.gmail.com>

This happens in my application as well, dont know if its a known issue for
boost python or not..
I too would like to see a solution; I myself have not had anytime to look
into it since exceptions are a rare thing in my app.
I can confirm however that a similar problem occurs on Solaris Sparcv9
machines, so I am guessing it has something to do with how solaris handles
64 bit exception handling.  Something boost is not setup to handle I would
suppose.

Charles

On Thu, Dec 2, 2010 at 4:23 PM, Dane Springmeyer <blake at hailmail.net> wrote:

> Hello,
>
> I recently recompiled my C++ app (mapnik.org, which uses boost python for
> python binding) with -m64 flags for 64 bit operation and this broke the
> boost python exception handling.
>
> Instead of C++ exceptions being propagated to python exceptions, I get
> segmentation faults (core dump).
>
> I have a long dependency chain that all needs to be 64 bit, so I'm really
> stuck unless I can find a way to restore boost python's ability to handle
> exceptions without crashing.
>
> Exceptions are very common during setup of mapnik, and are needed to report
> basic configuration issues as a user gets set up.
>
> I'm desperate to try anything that might help - anyone have any
> suggestions?
>
> What i have tried so far:
>
> 1) recompiling all apps with -02 instead of -03
> 2) making sure all apps are compiled with the exact same version of gcc (at
> least all the C++ libraries I have compiled from source)
> 3) ensured that all dynamic libraries link to the same libgcc_s.so.1 and
> libstdc++.so.6.
> 4) tried with both boost 1.44 and 1.45
>
> I'm out of ideas - anyone have suggestions to try?
>
> More info below.
>
> Dane
>
> --------------
>
> With 32bit boost, python, and mapnik, exceptions worked fine.
>
> This is on opensolaris:
>
> $ cat /etc/release
>                      OpenSolaris Development snv_133 X86
>          Copyright 2010 Sun Microsystems, Inc.  All Rights Reserved.
>                       Use is subject to license terms.
>                          Assembled 15 February 2010
>
> with the sun provided 64 bit python26:
>
> $ /usr/bin/amd64/python --version
> Python 2.6.4
>
> Both boost and mapnik are compiled with a *custom* compiled gcc 44:
>
> $ /opt/ts/gcc/4.4/bin/g++ -v
> Using built-in specs.
> Target: i386-pc-solaris2.11
> Configured with: .././configure --prefix=/opt/ts/gcc/4.4
> --bindir=/opt/ts/gcc/4.4/bin --libdir=/opt/ts/gcc/4.4/lib
> --mandir=/opt/ts/gcc/4.4/share/man --datadir=/opt/ts/gcc/4.4/share
> --includedir=/opt/ts/gcc/4.4/include --infodir=/opt/ts/gcc/4.4/share/info
> --libexecdir=/opt/ts/gcc/4.4/lib --sysconfdir=/etc/opt/ts --disable-nls
> --disable-static --with-gnu-as --with-as=/usr/gnu/bin/as --without-gnu-ld
> --with-ld=/usr/ccs/bin/ld --enable-threads=posix --enable-shared
> --enable-multilib --enable-nls --without-x --with-system-zlib
> --enable-languages=c,c++,f95,objc --with-mpfr=/opt/ts --with-gmp=/opt/ts
> Thread model: posix
> gcc version 4.4.4 (GCC)
>
> ( I was unable to get boost compiled with the gcc 3.4.3 or gcc 4.3.3
> provided by sun packages, for more info see:
> http://trac.mapnik.org/wiki/OpenSolarisInstallation/TroubleShooting)
>
>
> So, instead of throwing an understandable exception, boost python crashes
> and this is what a normal backtrace looks like in gbd:
>
> (gdb) bt
> #0  0x000000000005f666 in ?? ()
> #1  0xfffffd7fff2ec5d1 in _Unwind_RaiseException_Body () from
> /usr/lib/amd64/libc.so.1
> #2  0xfffffd7fff2ec855 in _Unwind_RaiseException () from
> /usr/lib/amd64/libc.so.1
> #3  0xfffffd7ffa98bb39 in __cxa_throw (obj=<value optimized out>,
> tinfo=0x1, dest=0x474e5543432b2b00)
>   at ../../../../.././libstdc++-v3/libsupc++/eh_throw.cc:78
> #4  0xfffffd7ff3aa2de2 in boost::python::throw_error_already_set () from
> /usr/local/lib/libboost_python.so.1.44.0
> #5  0xfffffd7ff3a9ae31 in boost::python::objects::function::argument_error
> () from /usr/local/lib/libboost_python.so.1.44.0
> #6  0xfffffd7ff3a9b4fb in boost::python::objects::function::call () from
> /usr/local/lib/libboost_python.so.1.44.0
> #7  0xfffffd7ff3a9b730 in
> boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous
> namespace)::bind_return, void>::invoke () from
> /usr/local/lib/libboost_python.so.1.44.0
> #8  0xfffffd7ff3aa3053 in
> boost::python::detail::exception_handler::operator() () from
> /usr/local/lib/libboost_python.so.1.44.0
> #9  0xfffffd7ff28fb1b2 in
> boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
> boost::python::detail::translate_exception<std::exception, void
> (*)(std::exception const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>,
> boost::_bi::value<void (*)(std::exception const&)> > >, bool,
> boost::python::detail::exception_handler const&, boost::function0<void>
> const&>::invoke ()
>  from /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
> #10 0xfffffd7ff3aa2e25 in boost::python::handle_exception_impl () from
> /usr/local/lib/libboost_python.so.1.44.0
> #11 0xfffffd7ff3a97d50 in function_call () from
> /usr/local/lib/libboost_python.so.1.44.0
> #12 0xfffffd7ff6ff9aad in PyObject_Call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #13 0xfffffd7ff700af29 in instancemethod_call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #14 0xfffffd7ff6ff9aad in PyObject_Call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #15 0xfffffd7ff705d8a1 in slot_tp_init () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #16 0xfffffd7ff7051c06 in type_call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #17 0xfffffd7ff6ff9aad in PyObject_Call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #18 0xfffffd7ff709c1c0 in do_call () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #19 0xfffffd7ff709b672 in call_function () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #20 0xfffffd7ff709810d in PyEval_EvalFrameExReal () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #21 0xfffffd7ff7094d9d in PyEval_EvalFrameEx () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #22 0xfffffd7ff709a27d in PyEval_EvalCodeEx () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #23 0xfffffd7ff7094caa in PyEval_EvalCode () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #24 0xfffffd7ff70bd385 in run_mod () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #25 0xfffffd7ff70bd27c in PyRun_StringFlags () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #26 0xfffffd7ff70bc53d in PyRun_SimpleStringFlags () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #27 0xfffffd7ff70ca5f0 in Py_Main () from
> /usr/lib/amd64/libpython2.6.so.1.0
> #28 0x0000000000400b3c in _start ()
>
>
> Here is the output of ldd for both the main mapnik library and the mapnik
> python bindings:
>
> $ ldd /usr/local/lib/libmapnik.so
>       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
>       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
>       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
>       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
>       libz.so.1 =>     /lib/64/libz.so.1
>       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
>       libproj.so.0 =>  /usr/local/lib/libproj.so.0
>       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
>       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
>       libboost_filesystem.so.1.44.0 =>
> /usr/local/lib/libboost_filesystem.so.1.44.0
>       libboost_regex.so.1.44.0 =>
>  /usr/local/lib/libboost_regex.so.1.44.0
>       libxml2.so.2 =>  /lib/64/libxml2.so.2
>       libboost_thread.so.1.44.0 =>
> /usr/local/lib/libboost_thread.so.1.44.0
>       libboost_system.so.1.44.0 =>
> /usr/local/lib/libboost_system.so.1.44.0
>       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>       libm.so.2 =>     /lib/64/libm.so.2
>       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>       libc.so.1 =>     /lib/64/libc.so.1
>       libdl.so.1 =>    /lib/64/libdl.so.1
>       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
>       libpthread.so.1 =>       /lib/64/libpthread.so.1
>       librt.so.1 =>    /lib/64/librt.so.1
>       libsocket.so.1 =>        /lib/64/libsocket.so.1
>       libnsl.so.1 =>   /lib/64/libnsl.so.1
>       libmp.so.2 =>    /lib/64/libmp.so.2
>       libmd.so.1 =>    /lib/64/libmd.so.1
>       libscf.so.1 =>   /lib/64/libscf.so.1
>       libuutil.so.1 =>         /lib/64/libuutil.so.1
>       libgen.so.1 =>   /lib/64/libgen.so.1
>       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
>
> $ ldd /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
>       libmapnik.so =>  /usr/local/lib/libmapnik.so
>       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
>       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
>       libboost_python.so.1.44.0 =>
> /usr/local/lib/libboost_python.so.1.44.0
>       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>       libm.so.2 =>     /lib/64/libm.so.2
>       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
>       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
>       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
>       libz.so.1 =>     /lib/64/libz.so.1
>       libproj.so.0 =>  /usr/local/lib/libproj.so.0
>       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
>       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
>       libboost_filesystem.so.1.44.0 =>
> /usr/local/lib/libboost_filesystem.so.1.44.0
>       libboost_regex.so.1.44.0 =>
>  /usr/local/lib/libboost_regex.so.1.44.0
>       libxml2.so.2 =>  /lib/64/libxml2.so.2
>       libboost_thread.so.1.44.0 =>
> /usr/local/lib/libboost_thread.so.1.44.0
>       libboost_system.so.1.44.0 =>
> /usr/local/lib/libboost_system.so.1.44.0
>       libc.so.1 =>     /usr/lib/amd64/libc.so.1
>       librt.so.1 =>    /lib/64/librt.so.1
>       libdl.so.1 =>    /lib/64/libdl.so.1
>       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
>       libpthread.so.1 =>       /lib/64/libpthread.so.1
>       libsocket.so.1 =>        /lib/64/libsocket.so.1
>       libnsl.so.1 =>   /lib/64/libnsl.so.1
>       libmp.so.2 =>    /lib/64/libmp.so.2
>       libmd.so.1 =>    /lib/64/libmd.so.1
>       libscf.so.1 =>   /lib/64/libscf.so.1
>       libuutil.so.1 =>         /lib/64/libuutil.so.1
>       libgen.so.1 =>   /lib/64/libgen.so.1
>       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
>
> Here is a simple way to prompt a segfault:
>
> $ /usr/bin/amd64/python
> Python 2.6.4 (r264:75706, Feb 14 2010, 14:06:28) [C] on sunos5
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import mapnik
> >>> m = mapnik.Map(256,256)
> >>> m
> <mapnik._mapnik.Map object at 0x55a1b0>
> # then, prompt a boost python argument error
> >>> m2 = mapnik.Map('foo')
> Segmentation Fault
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101202/6df7ef75/attachment-0001.html>

From talljimbo at gmail.com  Fri Dec  3 02:13:10 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 02 Dec 2010 17:13:10 -0800
Subject: [C++-sig] help finding solution to lack of exception handling
 with boost python on open solaris 64 bit
In-Reply-To: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
References: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
Message-ID: <4CF84426.7080804@gmail.com>

On 12/02/2010 02:23 PM, Dane Springmeyer wrote:
> Hello,
>
> I recently recompiled my C++ app (mapnik.org, which uses boost python for python binding) with -m64 flags for 64 bit operation and this broke the boost python exception handling.
>
> Instead of C++ exceptions being propagated to python exceptions, I get segmentation faults (core dump).
>
> I have a long dependency chain that all needs to be 64 bit, so I'm really stuck unless I can find a way to restore boost python's ability to handle exceptions without crashing.
>
> Exceptions are very common during setup of mapnik, and are needed to report basic configuration issues as a user gets set up.
>
> I'm desperate to try anything that might help - anyone have any suggestions?
>
> What i have tried so far:
>
> 1) recompiling all apps with -02 instead of -03
> 2) making sure all apps are compiled with the exact same version of gcc (at least all the C++ libraries I have compiled from source)
> 3) ensured that all dynamic libraries link to the same libgcc_s.so.1 and libstdc++.so.6.
> 4) tried with both boost 1.44 and 1.45
>
> I'm out of ideas - anyone have suggestions to try?
>

I'm not running Solaris, so I can't help too much specifically, but it 
sounds like you might learn a lot from trying exceptions in a non-Boost 
Python module compiled in C++.  Perhaps just take the first example here:

http://docs.python.org/extending/

and add a function that throws and catches an exception.

If that succeeds, try throwing and catching in a shared library that's 
linked to your Python module, and then try throwing and catching across 
the library boundary.

You might also want to check the compile flags for Python itself, just 
in case there's something obviously different there between 64-bit and 
32-bit on Solaris.


Good luck!

Jim Bosch



From rwgk at yahoo.com  Fri Dec  3 06:36:37 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 2 Dec 2010 21:36:37 -0800 (PST)
Subject: [C++-sig] help finding solution to lack of exception handling
	with boost python on open solaris 64 bit
In-Reply-To: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
References: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>
Message-ID: <25283.509.qm@web111415.mail.gq1.yahoo.com>

In the early days (~2002) we had trouble with EH on some Linux platforms, too. 
One workaround that helped for some things was to catch exceptions in the same 
.so where they were raised (e.g. boost::python::throw_error_already_set() 
ensures the exception is thrown in the libboost_python.so). Similar to Jim's 
suggestion, I'd try to find out if this is also your problem. In the worst case 
I'd combine all extensions into one (assuming you have multiple).
Another option is to try different dlopen flags, e.g.
sys.setdlopenflags(dl.RTLD_NOW|dl.RTLD_GLOBAL)

Ralf




----- Original Message ----
> From: Dane Springmeyer <blake at hailmail.net>
> To: Development of Python/C++ integration <cplusplus-sig at python.org>
> Cc: mapnik-devel <mapnik-devel at lists.berlios.de>
> Sent: Thu, December 2, 2010 2:23:02 PM
> Subject: [C++-sig] help finding solution to lack of exception handling with 
>boost python on open solaris 64 bit
> 
> Hello,
> 
> I recently recompiled my C++ app (mapnik.org, which uses boost  python for 
>python binding) with -m64 flags for 64 bit operation and this broke  the boost 
>python exception handling.
> 
> Instead of C++ exceptions being  propagated to python exceptions, I get 
>segmentation faults (core dump).
> 
> I  have a long dependency chain that all needs to be 64 bit, so I'm really 
>stuck  unless I can find a way to restore boost python's ability to handle 
>exceptions  without crashing.
> 
> Exceptions are very common during setup of mapnik, and  are needed to report 
>basic configuration issues as a user gets set  up.
> 
> I'm desperate to try anything that might help - anyone have any  suggestions?
> 
> What i have tried so far:
> 
> 1) recompiling all apps  with -02 instead of -03
> 2) making sure all apps are compiled with the exact  same version of gcc (at 
>least all the C++ libraries I have compiled from  source)
> 3) ensured that all dynamic libraries link to the same libgcc_s.so.1  and 
>libstdc++.so.6.
> 4) tried with both boost 1.44 and 1.45
> 
> I'm out of  ideas - anyone have suggestions to try?
> 
> More info  below.
> 
> Dane
> 
> --------------
> 
> With 32bit boost, python, and  mapnik, exceptions worked fine.
> 
> This is on opensolaris:
> 
> $ cat  /etc/release
>                        OpenSolaris Development snv_133 X86
>            Copyright 2010 Sun Microsystems, Inc.  All Rights  Reserved.
>                         Use is subject to license terms.
>                            Assembled 15  February 2010
> 
> with the sun provided 64 bit python26:
> 
> $  /usr/bin/amd64/python --version
> Python 2.6.4
> 
> Both boost and mapnik are  compiled with a *custom* compiled gcc 44:
> 
> $ /opt/ts/gcc/4.4/bin/g++  -v      
> Using built-in specs.
> Target:  i386-pc-solaris2.11
> Configured with: .././configure --prefix=/opt/ts/gcc/4.4  
>--bindir=/opt/ts/gcc/4.4/bin --libdir=/opt/ts/gcc/4.4/lib  
>--mandir=/opt/ts/gcc/4.4/share/man --datadir=/opt/ts/gcc/4.4/share  
>--includedir=/opt/ts/gcc/4.4/include --infodir=/opt/ts/gcc/4.4/share/info  
>--libexecdir=/opt/ts/gcc/4.4/lib --sysconfdir=/etc/opt/ts --disable-nls  
>--disable-static --with-gnu-as --with-as=/usr/gnu/bin/as --without-gnu-ld  
>--with-ld=/usr/ccs/bin/ld --enable-threads=posix --enable-shared  
>--enable-multilib --enable-nls --without-x --with-system-zlib  
>--enable-languages=c,c++,f95,objc --with-mpfr=/opt/ts  --with-gmp=/opt/ts
> Thread model: posix
> gcc version 4.4.4 (GCC) 
> 
> ( I  was unable to get boost compiled with the gcc 3.4.3 or gcc 4.3.3 provided 
>by sun  packages, for more info see: 
>http://trac.mapnik.org/wiki/OpenSolarisInstallation/TroubleShooting)
> 
> 
> So,  instead of throwing an understandable exception, boost python crashes and 
>this  is what a normal backtrace looks like in gbd:
> 
> (gdb) bt
> #0   0x000000000005f666 in ?? ()
> #1  0xfffffd7fff2ec5d1 in  _Unwind_RaiseException_Body () from 
>/usr/lib/amd64/libc.so.1
> #2   0xfffffd7fff2ec855 in _Unwind_RaiseException () from  
>/usr/lib/amd64/libc.so.1
> #3  0xfffffd7ffa98bb39 in __cxa_throw  (obj=<value optimized out>, tinfo=0x1, 
>dest=0x474e5543432b2b00)
>     at ../../../../.././libstdc++-v3/libsupc++/eh_throw.cc:78
> #4   0xfffffd7ff3aa2de2 in boost::python::throw_error_already_set () from  
>/usr/local/lib/libboost_python.so.1.44.0
> #5  0xfffffd7ff3a9ae31 in  boost::python::objects::function::argument_error () 
>from  /usr/local/lib/libboost_python.so.1.44.0
> #6  0xfffffd7ff3a9b4fb in  boost::python::objects::function::call () from  
>/usr/local/lib/libboost_python.so.1.44.0
> #7  0xfffffd7ff3a9b730 in  
>boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous
>  namespace)::bind_return, void>::invoke () from  
>/usr/local/lib/libboost_python.so.1.44.0
> #8  0xfffffd7ff3aa3053 in  boost::python::detail::exception_handler::operator() 
>() from  /usr/local/lib/libboost_python.so.1.44.0
> #9  0xfffffd7ff28fb1b2 in  
>boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,  
>boost::python::detail::translate_exception<std::exception, void  
>(*)(std::exception const&)>, boost::_bi::list3<boost::arg<1>,  boost::arg<2>, 
>boost::_bi::value<void (*)(std::exception  const&)> > >, bool, 
>boost::python::detail::exception_handler  const&, boost::function0<void> 
>const&>::invoke ()
>    from /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
> #10  0xfffffd7ff3aa2e25 in boost::python::handle_exception_impl () from  
>/usr/local/lib/libboost_python.so.1.44.0
> #11 0xfffffd7ff3a97d50 in  function_call () from 
>/usr/local/lib/libboost_python.so.1.44.0
> #12  0xfffffd7ff6ff9aad in PyObject_Call () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #13 0xfffffd7ff700af29 in  instancemethod_call () from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #14  0xfffffd7ff6ff9aad in PyObject_Call () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #15 0xfffffd7ff705d8a1 in slot_tp_init ()  from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #16 0xfffffd7ff7051c06 in type_call  () from 
/usr/lib/amd64/libpython2.6.so.1.0
> #17 0xfffffd7ff6ff9aad in  PyObject_Call () from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #18  0xfffffd7ff709c1c0 in do_call () from /usr/lib/amd64/libpython2.6.so.1.0
> #19  0xfffffd7ff709b672 in call_function () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #20 0xfffffd7ff709810d in  PyEval_EvalFrameExReal () from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #21  0xfffffd7ff7094d9d in PyEval_EvalFrameEx () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #22 0xfffffd7ff709a27d in  PyEval_EvalCodeEx () from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #23  0xfffffd7ff7094caa in PyEval_EvalCode () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #24 0xfffffd7ff70bd385 in run_mod () from  /usr/lib/amd64/libpython2.6.so.1.0
> #25 0xfffffd7ff70bd27c in  PyRun_StringFlags () from 
>/usr/lib/amd64/libpython2.6.so.1.0
> #26  0xfffffd7ff70bc53d in PyRun_SimpleStringFlags () from  
>/usr/lib/amd64/libpython2.6.so.1.0
> #27 0xfffffd7ff70ca5f0 in Py_Main () from  /usr/lib/amd64/libpython2.6.so.1.0
> #28 0x0000000000400b3c in _start  ()
> 
> 
> Here is the output of ldd for both the main mapnik library and the  mapnik 
>python bindings:
> 
> $ ldd /usr/local/lib/libmapnik.so
>         libfreetype.so.6 =>       /usr/lib/64/libfreetype.so.6
>        libltdl.so.3 =>   /usr/lib/64/libltdl.so.3
>        libpng12.so.0 =>          /usr/lib/64/libpng12.so.0
>         libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
>         libz.so.1 =>     /lib/64/libz.so.1
>         libjpeg.so.62 =>          /usr/lib/64/libjpeg.so.62
>        libproj.so.0 =>   /usr/local/lib/libproj.so.0
>        libicuuc.so.44 =>         /usr/local/lib/libicuuc.so.44
>         libicui18n.so.44 =>       /usr/local/lib/libicui18n.so.44
>         libboost_filesystem.so.1.44.0 =>          
>/usr/local/lib/libboost_filesystem.so.1.44.0
>         libboost_regex.so.1.44.0 =>       
>/usr/local/lib/libboost_regex.so.1.44.0
>        libxml2.so.2  =>  /lib/64/libxml2.so.2
>         libboost_thread.so.1.44.0 =>      
>/usr/local/lib/libboost_thread.so.1.44.0
>         libboost_system.so.1.44.0 =>      
>/usr/local/lib/libboost_system.so.1.44.0
>        libstdc++.so.6  =>         /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>        libm.so.2  =>     /lib/64/libm.so.2
>        libgcc_s.so.1  =>          /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>        libc.so.1  =>     /lib/64/libc.so.1
>        libdl.so.1  =>    /lib/64/libdl.so.1
>        libicudata.so.44  =>      /usr/local/lib/libicudata.so.44
>         libpthread.so.1 =>        /lib/64/libpthread.so.1
>        librt.so.1 =>     /lib/64/librt.so.1
>        libsocket.so.1 =>         /lib/64/libsocket.so.1
>        libnsl.so.1  =>   /lib/64/libnsl.so.1
>        libmp.so.2 =>     /lib/64/libmp.so.2
>        libmd.so.1 =>     /lib/64/libmd.so.1
>        libscf.so.1 =>    /lib/64/libscf.so.1
>        libuutil.so.1 =>          /lib/64/libuutil.so.1
>        libgen.so.1  =>   /lib/64/libgen.so.1
>        libsmbios.so.1  =>        /usr/lib/64/libsmbios.so.1
> 
> $ ldd  /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so 
>         libmapnik.so =>  /usr/local/lib/libmapnik.so
>         libpng12.so.0 =>          /usr/lib/64/libpng12.so.0
>        libjpeg.so.62 =>          /usr/lib/64/libjpeg.so.62
>         libboost_python.so.1.44.0 =>      
>/usr/local/lib/libboost_python.so.1.44.0
>        libstdc++.so.6  =>         /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>        libm.so.2  =>     /lib/64/libm.so.2
>        libgcc_s.so.1  =>          /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>         libfreetype.so.6 =>       /usr/lib/64/libfreetype.so.6
>        libltdl.so.3 =>   /usr/lib/64/libltdl.so.3
>        libtiff.so.3 =>   /usr/lib/64/libtiff.so.3
>        libz.so.1 =>      /lib/64/libz.so.1
>        libproj.so.0 =>   /usr/local/lib/libproj.so.0
>        libicuuc.so.44 =>         /usr/local/lib/libicuuc.so.44
>         libicui18n.so.44 =>       /usr/local/lib/libicui18n.so.44
>         libboost_filesystem.so.1.44.0 =>          
>/usr/local/lib/libboost_filesystem.so.1.44.0
>         libboost_regex.so.1.44.0 =>       
>/usr/local/lib/libboost_regex.so.1.44.0
>        libxml2.so.2  =>  /lib/64/libxml2.so.2
>         libboost_thread.so.1.44.0 =>      
>/usr/local/lib/libboost_thread.so.1.44.0
>         libboost_system.so.1.44.0 =>      
>/usr/local/lib/libboost_system.so.1.44.0
>        libc.so.1  =>     /usr/lib/amd64/libc.so.1
>        librt.so.1  =>    /lib/64/librt.so.1
>        libdl.so.1  =>    /lib/64/libdl.so.1
>        libicudata.so.44  =>      /usr/local/lib/libicudata.so.44
>         libpthread.so.1 =>        /lib/64/libpthread.so.1
>        libsocket.so.1 =>         /lib/64/libsocket.so.1
>        libnsl.so.1  =>   /lib/64/libnsl.so.1
>        libmp.so.2 =>     /lib/64/libmp.so.2
>        libmd.so.1 =>     /lib/64/libmd.so.1
>        libscf.so.1 =>    /lib/64/libscf.so.1
>        libuutil.so.1 =>          /lib/64/libuutil.so.1
>        libgen.so.1  =>   /lib/64/libgen.so.1
>        libsmbios.so.1  =>        /usr/lib/64/libsmbios.so.1
> 
> Here is a  simple way to prompt a segfault:
> 
> $ /usr/bin/amd64/python
> Python 2.6.4  (r264:75706, Feb 14 2010, 14:06:28) [C] on sunos5
> Type "help", "copyright",  "credits" or "license" for more information.
> >>> import  mapnik
> >>> m = mapnik.Map(256,256)
> >>>  m
> <mapnik._mapnik.Map object at 0x55a1b0>
> # then, prompt a boost  python argument error
> >>> m2 = mapnik.Map('foo')
> Segmentation  Fault
> 
> 
> 
> _______________________________________________
> Cplusplus-sig  mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> 

From s_sourceforge at nedprod.com  Fri Dec  3 14:51:01 2010
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Fri, 03 Dec 2010 13:51:01 -0000
Subject: [C++-sig] help finding solution to lack of exception handling
	with boost python on open solaris 64 bit
In-Reply-To: <AANLkTikUVGvgVGYANc+V1i+-s1gjOYjQvkukrRF2RLw3@mail.gmail.com>
References: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>,
	<AANLkTikUVGvgVGYANc+V1i+-s1gjOYjQvkukrRF2RLw3@mail.gmail.com>
Message-ID: <4CF8F5C5.6749.954DEEF1@s_sourceforge.nedprod.com>

Have you checked to ensure your applications meet the criteria 
specified in http://www.nedprod.com/programs/gccvisibility.html?

Usually this problem is due to a failure to fully implement this. Or 
it's a bug in GCC :)

Niall

On 2 Dec 2010 at 18:05, Charles Solar wrote:

> This happens in my application as well, dont know if its a known issue for
> boost python or not..
> I too would like to see a solution; I myself have not had anytime to look
> into it since exceptions are a rare thing in my app.
> I can confirm however that a similar problem occurs on Solaris Sparcv9
> machines, so I am guessing it has something to do with how solaris handles
> 64 bit exception handling.  Something boost is not setup to handle I would
> suppose.
> 
> Charles
> 
> On Thu, Dec 2, 2010 at 4:23 PM, Dane Springmeyer <blake at hailmail.net> wrote:
> 
> > Hello,
> >
> > I recently recompiled my C++ app (mapnik.org, which uses boost python for
> > python binding) with -m64 flags for 64 bit operation and this broke the
> > boost python exception handling.
> >
> > Instead of C++ exceptions being propagated to python exceptions, I get
> > segmentation faults (core dump).
> >
> > I have a long dependency chain that all needs to be 64 bit, so I'm really
> > stuck unless I can find a way to restore boost python's ability to handle
> > exceptions without crashing.
> >
> > Exceptions are very common during setup of mapnik, and are needed to report
> > basic configuration issues as a user gets set up.
> >
> > I'm desperate to try anything that might help - anyone have any
> > suggestions?
> >
> > What i have tried so far:
> >
> > 1) recompiling all apps with -02 instead of -03
> > 2) making sure all apps are compiled with the exact same version of gcc (at
> > least all the C++ libraries I have compiled from source)
> > 3) ensured that all dynamic libraries link to the same libgcc_s.so.1 and
> > libstdc++.so.6.
> > 4) tried with both boost 1.44 and 1.45
> >
> > I'm out of ideas - anyone have suggestions to try?
> >
> > More info below.
> >
> > Dane
> >
> > --------------
> >
> > With 32bit boost, python, and mapnik, exceptions worked fine.
> >
> > This is on opensolaris:
> >
> > $ cat /etc/release
> >                      OpenSolaris Development snv_133 X86
> >          Copyright 2010 Sun Microsystems, Inc.  All Rights Reserved.
> >                       Use is subject to license terms.
> >                          Assembled 15 February 2010
> >
> > with the sun provided 64 bit python26:
> >
> > $ /usr/bin/amd64/python --version
> > Python 2.6.4
> >
> > Both boost and mapnik are compiled with a *custom* compiled gcc 44:
> >
> > $ /opt/ts/gcc/4.4/bin/g++ -v
> > Using built-in specs.
> > Target: i386-pc-solaris2.11
> > Configured with: .././configure --prefix=/opt/ts/gcc/4.4
> > --bindir=/opt/ts/gcc/4.4/bin --libdir=/opt/ts/gcc/4.4/lib
> > --mandir=/opt/ts/gcc/4.4/share/man --datadir=/opt/ts/gcc/4.4/share
> > --includedir=/opt/ts/gcc/4.4/include --infodir=/opt/ts/gcc/4.4/share/info
> > --libexecdir=/opt/ts/gcc/4.4/lib --sysconfdir=/etc/opt/ts --disable-nls
> > --disable-static --with-gnu-as --with-as=/usr/gnu/bin/as --without-gnu-ld
> > --with-ld=/usr/ccs/bin/ld --enable-threads=posix --enable-shared
> > --enable-multilib --enable-nls --without-x --with-system-zlib
> > --enable-languages=c,c++,f95,objc --with-mpfr=/opt/ts --with-gmp=/opt/ts
> > Thread model: posix
> > gcc version 4.4.4 (GCC)
> >
> > ( I was unable to get boost compiled with the gcc 3.4.3 or gcc 4.3.3
> > provided by sun packages, for more info see:
> > http://trac.mapnik.org/wiki/OpenSolarisInstallation/TroubleShooting)
> >
> >
> > So, instead of throwing an understandable exception, boost python crashes
> > and this is what a normal backtrace looks like in gbd:
> >
> > (gdb) bt
> > #0  0x000000000005f666 in ?? ()
> > #1  0xfffffd7fff2ec5d1 in _Unwind_RaiseException_Body () from
> > /usr/lib/amd64/libc.so.1
> > #2  0xfffffd7fff2ec855 in _Unwind_RaiseException () from
> > /usr/lib/amd64/libc.so.1
> > #3  0xfffffd7ffa98bb39 in __cxa_throw (obj=<value optimized out>,
> > tinfo=0x1, dest=0x474e5543432b2b00)
> >   at ../../../../.././libstdc++-v3/libsupc++/eh_throw.cc:78
> > #4  0xfffffd7ff3aa2de2 in boost::python::throw_error_already_set () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #5  0xfffffd7ff3a9ae31 in boost::python::objects::function::argument_error
> > () from /usr/local/lib/libboost_python.so.1.44.0
> > #6  0xfffffd7ff3a9b4fb in boost::python::objects::function::call () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #7  0xfffffd7ff3a9b730 in
> > boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous
> > namespace)::bind_return, void>::invoke () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #8  0xfffffd7ff3aa3053 in
> > boost::python::detail::exception_handler::operator() () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #9  0xfffffd7ff28fb1b2 in
> > boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
> > boost::python::detail::translate_exception<std::exception, void
> > (*)(std::exception const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>,
> > boost::_bi::value<void (*)(std::exception const&)> > >, bool,
> > boost::python::detail::exception_handler const&, boost::function0<void>
> > const&>::invoke ()
> >  from /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
> > #10 0xfffffd7ff3aa2e25 in boost::python::handle_exception_impl () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #11 0xfffffd7ff3a97d50 in function_call () from
> > /usr/local/lib/libboost_python.so.1.44.0
> > #12 0xfffffd7ff6ff9aad in PyObject_Call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #13 0xfffffd7ff700af29 in instancemethod_call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #14 0xfffffd7ff6ff9aad in PyObject_Call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #15 0xfffffd7ff705d8a1 in slot_tp_init () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #16 0xfffffd7ff7051c06 in type_call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #17 0xfffffd7ff6ff9aad in PyObject_Call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #18 0xfffffd7ff709c1c0 in do_call () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #19 0xfffffd7ff709b672 in call_function () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #20 0xfffffd7ff709810d in PyEval_EvalFrameExReal () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #21 0xfffffd7ff7094d9d in PyEval_EvalFrameEx () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #22 0xfffffd7ff709a27d in PyEval_EvalCodeEx () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #23 0xfffffd7ff7094caa in PyEval_EvalCode () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #24 0xfffffd7ff70bd385 in run_mod () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #25 0xfffffd7ff70bd27c in PyRun_StringFlags () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #26 0xfffffd7ff70bc53d in PyRun_SimpleStringFlags () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #27 0xfffffd7ff70ca5f0 in Py_Main () from
> > /usr/lib/amd64/libpython2.6.so.1.0
> > #28 0x0000000000400b3c in _start ()
> >
> >
> > Here is the output of ldd for both the main mapnik library and the mapnik
> > python bindings:
> >
> > $ ldd /usr/local/lib/libmapnik.so
> >       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
> >       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
> >       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
> >       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
> >       libz.so.1 =>     /lib/64/libz.so.1
> >       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
> >       libproj.so.0 =>  /usr/local/lib/libproj.so.0
> >       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
> >       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
> >       libboost_filesystem.so.1.44.0 =>
> > /usr/local/lib/libboost_filesystem.so.1.44.0
> >       libboost_regex.so.1.44.0 =>
> >  /usr/local/lib/libboost_regex.so.1.44.0
> >       libxml2.so.2 =>  /lib/64/libxml2.so.2
> >       libboost_thread.so.1.44.0 =>
> > /usr/local/lib/libboost_thread.so.1.44.0
> >       libboost_system.so.1.44.0 =>
> > /usr/local/lib/libboost_system.so.1.44.0
> >       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
> >       libm.so.2 =>     /lib/64/libm.so.2
> >       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
> >       libc.so.1 =>     /lib/64/libc.so.1
> >       libdl.so.1 =>    /lib/64/libdl.so.1
> >       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
> >       libpthread.so.1 =>       /lib/64/libpthread.so.1
> >       librt.so.1 =>    /lib/64/librt.so.1
> >       libsocket.so.1 =>        /lib/64/libsocket.so.1
> >       libnsl.so.1 =>   /lib/64/libnsl.so.1
> >       libmp.so.2 =>    /lib/64/libmp.so.2
> >       libmd.so.1 =>    /lib/64/libmd.so.1
> >       libscf.so.1 =>   /lib/64/libscf.so.1
> >       libuutil.so.1 =>         /lib/64/libuutil.so.1
> >       libgen.so.1 =>   /lib/64/libgen.so.1
> >       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
> >
> > $ ldd /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
> >       libmapnik.so =>  /usr/local/lib/libmapnik.so
> >       libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
> >       libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
> >       libboost_python.so.1.44.0 =>
> > /usr/local/lib/libboost_python.so.1.44.0
> >       libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
> >       libm.so.2 =>     /lib/64/libm.so.2
> >       libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
> >       libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
> >       libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
> >       libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
> >       libz.so.1 =>     /lib/64/libz.so.1
> >       libproj.so.0 =>  /usr/local/lib/libproj.so.0
> >       libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
> >       libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
> >       libboost_filesystem.so.1.44.0 =>
> > /usr/local/lib/libboost_filesystem.so.1.44.0
> >       libboost_regex.so.1.44.0 =>
> >  /usr/local/lib/libboost_regex.so.1.44.0
> >       libxml2.so.2 =>  /lib/64/libxml2.so.2
> >       libboost_thread.so.1.44.0 =>
> > /usr/local/lib/libboost_thread.so.1.44.0
> >       libboost_system.so.1.44.0 =>
> > /usr/local/lib/libboost_system.so.1.44.0
> >       libc.so.1 =>     /usr/lib/amd64/libc.so.1
> >       librt.so.1 =>    /lib/64/librt.so.1
> >       libdl.so.1 =>    /lib/64/libdl.so.1
> >       libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
> >       libpthread.so.1 =>       /lib/64/libpthread.so.1
> >       libsocket.so.1 =>        /lib/64/libsocket.so.1
> >       libnsl.so.1 =>   /lib/64/libnsl.so.1
> >       libmp.so.2 =>    /lib/64/libmp.so.2
> >       libmd.so.1 =>    /lib/64/libmd.so.1
> >       libscf.so.1 =>   /lib/64/libscf.so.1
> >       libuutil.so.1 =>         /lib/64/libuutil.so.1
> >       libgen.so.1 =>   /lib/64/libgen.so.1
> >       libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
> >
> > Here is a simple way to prompt a segfault:
> >
> > $ /usr/bin/amd64/python
> > Python 2.6.4 (r264:75706, Feb 14 2010, 14:06:28) [C] on sunos5
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import mapnik
> > >>> m = mapnik.Map(256,256)
> > >>> m
> > <mapnik._mapnik.Map object at 0x55a1b0>
> > # then, prompt a boost python argument error
> > >>> m2 = mapnik.Map('foo')
> > Segmentation Fault
> >
> >
> >
> > _______________________________________________
> > Cplusplus-sig mailing list
> > Cplusplus-sig at python.org
> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> >
> 


-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From blake at hailmail.net  Sat Dec  4 00:47:31 2010
From: blake at hailmail.net (Dane Springmeyer)
Date: Fri, 3 Dec 2010 15:47:31 -0800
Subject: [C++-sig] help finding solution to lack of exception handling
	with boost python on open solaris 64 bit
In-Reply-To: <4CF8F5C5.6749.954DEEF1@s_sourceforge.nedprod.com>
References: <827B3364-5590-4394-AB72-BE04437BFFF7@hailmail.net>,
	<AANLkTikUVGvgVGYANc+V1i+-s1gjOYjQvkukrRF2RLw3@mail.gmail.com>
	<4CF8F5C5.6749.954DEEF1@s_sourceforge.nedprod.com>
Message-ID: <788FA1AF-8DAD-42C0-99ED-ED25AE18BE51@hailmail.net>

Thanks everyone for your ideas! Exactly what I was looking for.

With your help I I've narrowed it down to duplicate libgcc_s.so versions being imported, as this solves it:

LD_PRELOAD=/usr/sfw/lib/amd64/libgcc_s.so

So I can now do:

LD_PRELOAD=/usr/sfw/lib/amd64/libgcc_s.so /usr/bin/amd64/python -c "import mapnik;mapnik.Map('this should throw exception');"

...which now throws a proper argument error exception.

So, my error is now clearer: I've been using sun provided 64 bit python compiled with a different gcc version than I built both boost and mapnik with. Why I could get away with this in 32 bit mode is beyond me.

My understanding is that by using LD_PRELOAD I am able to force the libgcc_s.so version that python itself is linked to - to be loaded first (rather than the libgcc_s.so that boost_python and _mapnik.so are linked to). This allows both the exception handling to work again and the program (surprisingly) to run properly (tests pass at least).

I figure, since I was unable to compile boost_python (or boost proper) with the gcc version from sun (and that the sun python was compiled with), my only proper route from here is to compile python myself using my custom version of gcc.

Dane


>> 
>> On Thu, Dec 2, 2010 at 4:23 PM, Dane Springmeyer <blake at hailmail.net> wrote:
>> 
>>> Hello,
>>> 
>>> I recently recompiled my C++ app (mapnik.org, which uses boost python for
>>> python binding) with -m64 flags for 64 bit operation and this broke the
>>> boost python exception handling.
>>> 
>>> Instead of C++ exceptions being propagated to python exceptions, I get
>>> segmentation faults (core dump).
>>> 
>>> I have a long dependency chain that all needs to be 64 bit, so I'm really
>>> stuck unless I can find a way to restore boost python's ability to handle
>>> exceptions without crashing.
>>> 
>>> Exceptions are very common during setup of mapnik, and are needed to report
>>> basic configuration issues as a user gets set up.
>>> 
>>> I'm desperate to try anything that might help - anyone have any
>>> suggestions?
>>> 
>>> What i have tried so far:
>>> 
>>> 1) recompiling all apps with -02 instead of -03
>>> 2) making sure all apps are compiled with the exact same version of gcc (at
>>> least all the C++ libraries I have compiled from source)
>>> 3) ensured that all dynamic libraries link to the same libgcc_s.so.1 and
>>> libstdc++.so.6.
>>> 4) tried with both boost 1.44 and 1.45
>>> 
>>> I'm out of ideas - anyone have suggestions to try?
>>> 
>>> More info below.
>>> 
>>> Dane
>>> 
>>> --------------
>>> 
>>> With 32bit boost, python, and mapnik, exceptions worked fine.
>>> 
>>> This is on opensolaris:
>>> 
>>> $ cat /etc/release
>>>                     OpenSolaris Development snv_133 X86
>>>         Copyright 2010 Sun Microsystems, Inc.  All Rights Reserved.
>>>                      Use is subject to license terms.
>>>                         Assembled 15 February 2010
>>> 
>>> with the sun provided 64 bit python26:
>>> 
>>> $ /usr/bin/amd64/python --version
>>> Python 2.6.4
>>> 
>>> Both boost and mapnik are compiled with a *custom* compiled gcc 44:
>>> 
>>> $ /opt/ts/gcc/4.4/bin/g++ -v
>>> Using built-in specs.
>>> Target: i386-pc-solaris2.11
>>> Configured with: .././configure --prefix=/opt/ts/gcc/4.4
>>> --bindir=/opt/ts/gcc/4.4/bin --libdir=/opt/ts/gcc/4.4/lib
>>> --mandir=/opt/ts/gcc/4.4/share/man --datadir=/opt/ts/gcc/4.4/share
>>> --includedir=/opt/ts/gcc/4.4/include --infodir=/opt/ts/gcc/4.4/share/info
>>> --libexecdir=/opt/ts/gcc/4.4/lib --sysconfdir=/etc/opt/ts --disable-nls
>>> --disable-static --with-gnu-as --with-as=/usr/gnu/bin/as --without-gnu-ld
>>> --with-ld=/usr/ccs/bin/ld --enable-threads=posix --enable-shared
>>> --enable-multilib --enable-nls --without-x --with-system-zlib
>>> --enable-languages=c,c++,f95,objc --with-mpfr=/opt/ts --with-gmp=/opt/ts
>>> Thread model: posix
>>> gcc version 4.4.4 (GCC)
>>> 
>>> ( I was unable to get boost compiled with the gcc 3.4.3 or gcc 4.3.3
>>> provided by sun packages, for more info see:
>>> http://trac.mapnik.org/wiki/OpenSolarisInstallation/TroubleShooting)
>>> 
>>> 
>>> So, instead of throwing an understandable exception, boost python crashes
>>> and this is what a normal backtrace looks like in gbd:
>>> 
>>> (gdb) bt
>>> #0  0x000000000005f666 in ?? ()
>>> #1  0xfffffd7fff2ec5d1 in _Unwind_RaiseException_Body () from
>>> /usr/lib/amd64/libc.so.1
>>> #2  0xfffffd7fff2ec855 in _Unwind_RaiseException () from
>>> /usr/lib/amd64/libc.so.1
>>> #3  0xfffffd7ffa98bb39 in __cxa_throw (obj=<value optimized out>,
>>> tinfo=0x1, dest=0x474e5543432b2b00)
>>>  at ../../../../.././libstdc++-v3/libsupc++/eh_throw.cc:78
>>> #4  0xfffffd7ff3aa2de2 in boost::python::throw_error_already_set () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #5  0xfffffd7ff3a9ae31 in boost::python::objects::function::argument_error
>>> () from /usr/local/lib/libboost_python.so.1.44.0
>>> #6  0xfffffd7ff3a9b4fb in boost::python::objects::function::call () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #7  0xfffffd7ff3a9b730 in
>>> boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous
>>> namespace)::bind_return, void>::invoke () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #8  0xfffffd7ff3aa3053 in
>>> boost::python::detail::exception_handler::operator() () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #9  0xfffffd7ff28fb1b2 in
>>> boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
>>> boost::python::detail::translate_exception<std::exception, void
>>> (*)(std::exception const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>,
>>> boost::_bi::value<void (*)(std::exception const&)> > >, bool,
>>> boost::python::detail::exception_handler const&, boost::function0<void>
>>> const&>::invoke ()
>>> from /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
>>> #10 0xfffffd7ff3aa2e25 in boost::python::handle_exception_impl () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #11 0xfffffd7ff3a97d50 in function_call () from
>>> /usr/local/lib/libboost_python.so.1.44.0
>>> #12 0xfffffd7ff6ff9aad in PyObject_Call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #13 0xfffffd7ff700af29 in instancemethod_call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #14 0xfffffd7ff6ff9aad in PyObject_Call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #15 0xfffffd7ff705d8a1 in slot_tp_init () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #16 0xfffffd7ff7051c06 in type_call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #17 0xfffffd7ff6ff9aad in PyObject_Call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #18 0xfffffd7ff709c1c0 in do_call () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #19 0xfffffd7ff709b672 in call_function () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #20 0xfffffd7ff709810d in PyEval_EvalFrameExReal () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #21 0xfffffd7ff7094d9d in PyEval_EvalFrameEx () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #22 0xfffffd7ff709a27d in PyEval_EvalCodeEx () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #23 0xfffffd7ff7094caa in PyEval_EvalCode () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #24 0xfffffd7ff70bd385 in run_mod () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #25 0xfffffd7ff70bd27c in PyRun_StringFlags () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #26 0xfffffd7ff70bc53d in PyRun_SimpleStringFlags () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #27 0xfffffd7ff70ca5f0 in Py_Main () from
>>> /usr/lib/amd64/libpython2.6.so.1.0
>>> #28 0x0000000000400b3c in _start ()
>>> 
>>> 
>>> Here is the output of ldd for both the main mapnik library and the mapnik
>>> python bindings:
>>> 
>>> $ ldd /usr/local/lib/libmapnik.so
>>>      libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
>>>      libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
>>>      libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
>>>      libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
>>>      libz.so.1 =>     /lib/64/libz.so.1
>>>      libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
>>>      libproj.so.0 =>  /usr/local/lib/libproj.so.0
>>>      libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
>>>      libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
>>>      libboost_filesystem.so.1.44.0 =>
>>> /usr/local/lib/libboost_filesystem.so.1.44.0
>>>      libboost_regex.so.1.44.0 =>
>>> /usr/local/lib/libboost_regex.so.1.44.0
>>>      libxml2.so.2 =>  /lib/64/libxml2.so.2
>>>      libboost_thread.so.1.44.0 =>
>>> /usr/local/lib/libboost_thread.so.1.44.0
>>>      libboost_system.so.1.44.0 =>
>>> /usr/local/lib/libboost_system.so.1.44.0
>>>      libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>>>      libm.so.2 =>     /lib/64/libm.so.2
>>>      libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>>>      libc.so.1 =>     /lib/64/libc.so.1
>>>      libdl.so.1 =>    /lib/64/libdl.so.1
>>>      libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
>>>      libpthread.so.1 =>       /lib/64/libpthread.so.1
>>>      librt.so.1 =>    /lib/64/librt.so.1
>>>      libsocket.so.1 =>        /lib/64/libsocket.so.1
>>>      libnsl.so.1 =>   /lib/64/libnsl.so.1
>>>      libmp.so.2 =>    /lib/64/libmp.so.2
>>>      libmd.so.1 =>    /lib/64/libmd.so.1
>>>      libscf.so.1 =>   /lib/64/libscf.so.1
>>>      libuutil.so.1 =>         /lib/64/libuutil.so.1
>>>      libgen.so.1 =>   /lib/64/libgen.so.1
>>>      libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
>>> 
>>> $ ldd /usr/local/lib/python2.6/site-packages/mapnik/64/_mapnik.so
>>>      libmapnik.so =>  /usr/local/lib/libmapnik.so
>>>      libpng12.so.0 =>         /usr/lib/64/libpng12.so.0
>>>      libjpeg.so.62 =>         /usr/lib/64/libjpeg.so.62
>>>      libboost_python.so.1.44.0 =>
>>> /usr/local/lib/libboost_python.so.1.44.0
>>>      libstdc++.so.6 =>        /opt/ts/gcc/4.4/lib/amd64/libstdc++.so.6
>>>      libm.so.2 =>     /lib/64/libm.so.2
>>>      libgcc_s.so.1 =>         /opt/ts/gcc/4.4/lib/amd64/libgcc_s.so.1
>>>      libfreetype.so.6 =>      /usr/lib/64/libfreetype.so.6
>>>      libltdl.so.3 =>  /usr/lib/64/libltdl.so.3
>>>      libtiff.so.3 =>  /usr/lib/64/libtiff.so.3
>>>      libz.so.1 =>     /lib/64/libz.so.1
>>>      libproj.so.0 =>  /usr/local/lib/libproj.so.0
>>>      libicuuc.so.44 =>        /usr/local/lib/libicuuc.so.44
>>>      libicui18n.so.44 =>      /usr/local/lib/libicui18n.so.44
>>>      libboost_filesystem.so.1.44.0 =>
>>> /usr/local/lib/libboost_filesystem.so.1.44.0
>>>      libboost_regex.so.1.44.0 =>
>>> /usr/local/lib/libboost_regex.so.1.44.0
>>>      libxml2.so.2 =>  /lib/64/libxml2.so.2
>>>      libboost_thread.so.1.44.0 =>
>>> /usr/local/lib/libboost_thread.so.1.44.0
>>>      libboost_system.so.1.44.0 =>
>>> /usr/local/lib/libboost_system.so.1.44.0
>>>      libc.so.1 =>     /usr/lib/amd64/libc.so.1
>>>      librt.so.1 =>    /lib/64/librt.so.1
>>>      libdl.so.1 =>    /lib/64/libdl.so.1
>>>      libicudata.so.44 =>      /usr/local/lib/libicudata.so.44
>>>      libpthread.so.1 =>       /lib/64/libpthread.so.1
>>>      libsocket.so.1 =>        /lib/64/libsocket.so.1
>>>      libnsl.so.1 =>   /lib/64/libnsl.so.1
>>>      libmp.so.2 =>    /lib/64/libmp.so.2
>>>      libmd.so.1 =>    /lib/64/libmd.so.1
>>>      libscf.so.1 =>   /lib/64/libscf.so.1
>>>      libuutil.so.1 =>         /lib/64/libuutil.so.1
>>>      libgen.so.1 =>   /lib/64/libgen.so.1
>>>      libsmbios.so.1 =>        /usr/lib/64/libsmbios.so.1
>>> 
>>> Here is a simple way to prompt a segfault:
>>> 
>>> $ /usr/bin/amd64/python
>>> Python 2.6.4 (r264:75706, Feb 14 2010, 14:06:28) [C] on sunos5
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import mapnik
>>>>>> m = mapnik.Map(256,256)
>>>>>> m
>>> <mapnik._mapnik.Map object at 0x55a1b0>
>>> # then, prompt a boost python argument error
>>>>>> m2 = mapnik.Map('foo')
>>> Segmentation Fault
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> 
>> 
> 
> 
> -- 
> Technology & Consulting Services - ned Productions Limited.
> http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
> 472909.
> 
> 
> 
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101203/d302789b/attachment-0001.html>

From mistersheik at gmail.com  Mon Dec  6 09:24:01 2010
From: mistersheik at gmail.com (Neil Girdhar)
Date: Mon, 6 Dec 2010 03:24:01 -0500
Subject: [C++-sig] using ostream crashes boost.python
Message-ID: <AANLkTi=wQ4sJP5bVf3O7krm9RpiOQr9Xrvs93mBOMxCB@mail.gmail.com>

Hi all,

First of all, many thanks for those that are developing boost.python.
I discovered it last week, and I'm very excited about it. :)

I'm having a problem with C++ streams.  For some reason, serializing
numbers crashes boost.python.  I have a minimal example here:

http://stackoverflow.com/questions/4363967/using-ostream-crashes-boost-python

Any help is greatly appreciated.

Neil

From josh.davidson at lmco.com  Wed Dec  8 08:40:15 2010
From: josh.davidson at lmco.com (Davidson, Josh)
Date: Wed, 08 Dec 2010 00:40:15 -0700
Subject: [C++-sig] pygccxml and template classes
Message-ID: <3602620DAE03994D80A0E6387C1A3B83BBD83A53@HDXMSP8.us.lmco.com>

I'm having some trouble getting pyccxml to parse specialized classes.  Consider the following simple header file:


#ifndef TEMPLATECLASS_HPP_
#define TEMPLATECLASS_HPP_

template <typename Type1, typename Type2>
class TemplateClass {
public:
    TemplateClass(){}

protected:
    int x;
    Type1 y;
    Type2 z;


};

#endif


Now, if I parse it with the following statements:
loggers.gccxml.setLevel(logging.WARNING)
config = parser.config_t(gccxml_path="gccxml", include_paths=includes, define_symbols=defines)
#file is above header file
decls = parser.parse([file], config)
xml = declarations.get_global_namespace(decls)

It parses OK, but xml.classes() returns nothing.  If I remove the template types, it works fine.  What am I doing wrong?

Thanks,
Josh

From roman.yakovenko at gmail.com  Wed Dec  8 13:03:30 2010
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Wed, 8 Dec 2010 14:03:30 +0200
Subject: [C++-sig] pygccxml and template classes
In-Reply-To: <3602620DAE03994D80A0E6387C1A3B83BBD83A53@HDXMSP8.us.lmco.com>
References: <3602620DAE03994D80A0E6387C1A3B83BBD83A53@HDXMSP8.us.lmco.com>
Message-ID: <AANLkTimxd29bsF8gK8hzOqD6X+RZVavOYxfuta7x7jF=@mail.gmail.com>

On Wed, Dec 8, 2010 at 9:40 AM, Davidson, Josh <josh.davidson at lmco.com>wrote:

> I'm having some trouble getting pyccxml to parse specialized classes.
>  Consider the following simple header file:
>
>
> #ifndef TEMPLATECLASS_HPP_
> #define TEMPLATECLASS_HPP_
>
> template <typename Type1, typename Type2>
> class TemplateClass {
>
   ...

> };
>
> #endif
>
>
> Now, if I parse it with the following statements:
> loggers.gccxml.setLevel(logging.WARNING)
> config = parser.config_t(gccxml_path="gccxml", include_paths=includes,
> define_symbols=defines)
> #file is above header file
> decls = parser.parse([file], config)
> xml = declarations.get_global_namespace(decls)
>
> It parses OK, but xml.classes() returns nothing.  If I remove the template
> types, it works fine.  What am I doing wrong?
>
>
Nothing.

GCC-XML is not able to dump template classes, but only template
instantiations.


HTH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101208/4a186e6f/attachment.html>

From bradbell at seanet.com  Wed Dec  8 16:22:42 2010
From: bradbell at seanet.com (Brad Bell)
Date: Wed, 08 Dec 2010 07:22:42 -0800
Subject: [C++-sig] Cygwin and boost::python exceptions
Message-ID: <4CFFA2C2.6070906@seanet.com>

There seems to be a problem with the example on
     
http://www.boost.org/doc/libs/1_45_0/libs/python/doc/v2/exception_translator.html
when I compile it I get the following message:
--------------------------------------------------------------------------
bradbell at apl-9232eef971d ~/pycppad/boost_py
$ g++  -I/usr/include/python2.6 -g -c exception.cpp
exception.cpp: In function `void translate(const my_exception&)':
exception.cpp:14: error: passing `const my_exception' as `this' argument 
of `con
st char* my_exception::what()' discards qualifiers
exception.cpp: In function `void something_which_throws()':
exception.cpp:19: error: expected primary-expression before `...' token
exception.cpp:19: error: expected `;' before `...' token
exception.cpp:21: error: expected primary-expression before `...' token
exception.cpp:21: error: expected `;' before `...' token

bradbell at apl-9232eef971d ~/pycppad/boost_py
$ g++ --version
g++ (GCC) 4.3.4 20090804 (release) 1
--------------------------------------------------------------------------

I have figured out a way around this problem, see the attached file 
exception.sh. This works fine under unix, but under cygwin, the 
exception does not seem to be caught by the python 'except' section of 
the 'try' block.

--------------------------------------------------------------------------
Here is the result of running exception.sh under unix:

[bradbell at localhost trash]$ ./exception.sh
cat << EOF > exception.cpp
g++  -I/usr/include/python2.7 -g -c exception.cpp
g++ -shared  \
     -g \
     exception.o \
     -L/usr/lib -L/usr/lib/python2.7/config  \
     -lboost_python-mt -lpython2.7 \
     -o exception.so
cat << EOF > exception.py
exception.sh: OK
...
--------------------------------------------------------------------------
Here is the result of running under cygwin:

$ ./exception.sh
cat << EOF > exception.cpp
g++  -I/usr/include/python2.6 -g -c exception.cpp
g++ -shared -Wl,--enable-auto-image-base \
         -g \
         exception.o \
         -L/usr/lib -L/usr/lib/python2.6/config  \
         -lboost_python-mt -lpython2.6 \
         -o exception.dll
cat << EOF > exception.py
terminate called after throwing an instance of 'my_exception'
   what():  my error message
./exception.sh: line 97:  2108 Aborted                 (core dumped) 
python exception.py
exception.sh: Error

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: exception.sh
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101208/3f10e235/attachment.ksh>

From max.stricker at student.tugraz.at  Sat Dec 11 13:29:05 2010
From: max.stricker at student.tugraz.at (Max Stricker)
Date: Sat, 11 Dec 2010 13:29:05 +0100
Subject: [C++-sig] boost.python errors on Linux
Message-ID: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>

Hi,

i need to expose some C++ methods to python and came accross boost.python.
I installed libboost-python1.42-dev on an Ubuntu machine and tried to work with a
simple program found in the docs:

#include <iostream>

class Foo {
	public:
		void bar() {
			std::cout << "Hello" << std::endl;
		}
};

#include <boost/python.hpp>
#include <Python.h>
using namespace boost::python;

BOOST_PYTHON_MODULE(libboopyclass) {
	boost::python::class_<Foo>("Foo")
		.def("bar", &Foo::bar);
}


I build it using: gcc foo.cpp  -I /usr/include/python2.6 -l python2.6 -l boost_python

I get a huge error list indicating that objects like PyObject or PyTypeObject could not be found,
the detailed output is available here: http://pastebin.com/TyN9dZ09

Has anyone an idea whats going on here? How can I get this minimal example to work?


Regards,
Max


From seefeld at sympatico.ca  Sat Dec 11 15:03:17 2010
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Sat, 11 Dec 2010 09:03:17 -0500
Subject: [C++-sig] boost.python errors on Linux
In-Reply-To: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>
References: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>
Message-ID: <4D0384A5.9020704@sympatico.ca>

On 12/11/2010 07:29 AM, Max Stricker wrote:
> Hi,
>
> i need to expose some C++ methods to python and came accross boost.python.
> I installed libboost-python1.42-dev on an Ubuntu machine and tried to work with a
> simple program found in the docs:
>
> #include<iostream>
>
> class Foo {
> 	public:
> 		void bar() {
> 			std::cout<<  "Hello"<<  std::endl;
> 		}
> };
>
> #include<boost/python.hpp>
> #include<Python.h>
> using namespace boost::python;
>
> BOOST_PYTHON_MODULE(libboopyclass) {
> 	boost::python::class_<Foo>("Foo")
> 		.def("bar",&Foo::bar);
> }
>
>
> I build it using: gcc foo.cpp  -I /usr/include/python2.6 -l python2.6 -l boost_python
>
> I get a huge error list indicating that objects like PyObject or PyTypeObject could not be found,
> the detailed output is available here: http://pastebin.com/TyN9dZ09

Add the '-shared' option (to indicate that you want to build a shared 
library), remove the space in '-l boost_python', and specify the output 
filename (such as '-o foo.so'). Finally, you should compile with g++, 
not gcc.
For example:

g++ -shared -o foo.so foo.cpp -I/usr/include/python2.6 -lpython2.6 
-lboost_python

     Stefan


-- 

       ...ich hab' noch einen Koffer in Berlin...


From max.stricker at student.tugraz.at  Sat Dec 11 17:10:47 2010
From: max.stricker at student.tugraz.at (Max Stricker)
Date: Sat, 11 Dec 2010 17:10:47 +0100
Subject: [C++-sig] boost.python errors on Linux
In-Reply-To: <4D0384A5.9020704@sympatico.ca>
References: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>
	<4D0384A5.9020704@sympatico.ca>
Message-ID: <6676C683-C10C-480E-971F-58D49D839496@student.tugraz.at>

Am 11.12.2010 um 15:03 schrieb Stefan Seefeld:

> On 12/11/2010 07:29 AM, Max Stricker wrote:
>> Hi,
>> 
>> i need to expose some C++ methods to python and came accross boost.python.
>> I installed libboost-python1.42-dev on an Ubuntu machine and tried to work with a
>> simple program found in the docs:
>> 
>> #include<iostream>
>> 
>> class Foo {
>> 	public:
>> 		void bar() {
>> 			std::cout<<  "Hello"<<  std::endl;
>> 		}
>> };
>> 
>> #include<boost/python.hpp>
>> #include<Python.h>
>> using namespace boost::python;
>> 
>> BOOST_PYTHON_MODULE(libboopyclass) {
>> 	boost::python::class_<Foo>("Foo")
>> 		.def("bar",&Foo::bar);
>> }
>> 
>> 
>> I build it using: gcc foo.cpp  -I /usr/include/python2.6 -l python2.6 -l boost_python
>> 
>> I get a huge error list indicating that objects like PyObject or PyTypeObject could not be found,
>> the detailed output is available here: http://pastebin.com/TyN9dZ09
> 
> Add the '-shared' option (to indicate that you want to build a shared library), remove the space in '-l boost_python', and specify the output filename (such as '-o foo.so'). Finally, you should compile with g++, not gcc.
> For example:
> 
> g++ -shared -o foo.so foo.cpp -I/usr/include/python2.6 -lpython2.6 -lboost_python
> 
>    Stefan

I tried your suggestions but it leads to the same error message (using g++ -shared foo.cpp -o foo.so  -I /usr/include/python2.6 -lpython2.6 -lboost_python).
Could there be something wrong with the installation?
In the meantime I tried it on an OSX machine, installed boost 1.45 by hand (using bjam --with-python). The error is exactly the same.
I suppose the code should be fine as its from the docs. Am I missing an important step?

Max


From seefeld at sympatico.ca  Sat Dec 11 17:17:06 2010
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Sat, 11 Dec 2010 11:17:06 -0500
Subject: [C++-sig] boost.python errors on Linux
In-Reply-To: <6676C683-C10C-480E-971F-58D49D839496@student.tugraz.at>
References: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>	<4D0384A5.9020704@sympatico.ca>
	<6676C683-C10C-480E-971F-58D49D839496@student.tugraz.at>
Message-ID: <4D03A402.5040602@sympatico.ca>

On 12/11/2010 11:10 AM, Max Stricker wrote:
>
> I tried your suggestions but it leads to the same error message (using g++ -shared foo.cpp -o foo.so  -I /usr/include/python2.6 -lpython2.6 -lboost_python).
> Could there be something wrong with the installation?

Your error message suggests the compiler couldn't find the Python 
headers. Do you have Python 2.6 installed ? Are the headers in 
/usr/include/python2.6 ? If not, you need to adjust your command line to 
whatever Python installation you have. (If you use system python 
packages, make sure to also install the "python-dev" package, which 
provides the headers as well as linkable DSOs.)

HTH,
         Stefan

-- 

       ...ich hab' noch einen Koffer in Berlin...


From scom at onet.eu  Sat Dec 11 18:02:20 2010
From: scom at onet.eu (scom)
Date: Sat, 11 Dec 2010 18:02:20 +0100
Subject: [C++-sig] boost.python errors on Linux
In-Reply-To: <4D03A402.5040602@sympatico.ca>
References: <8C7C9151-3AF4-4490-8609-D9A8E897787A@student.tugraz.at>	<4D0384A5.9020704@sympatico.ca>	<6676C683-C10C-480E-971F-58D49D839496@student.tugraz.at>
	<4D03A402.5040602@sympatico.ca>
Message-ID: <4D03AE9C.7030908@onet.eu>

W dniu 2010-12-11 17:17, Stefan Seefeld pisze:
> On 12/11/2010 11:10 AM, Max Stricker wrote:
>>
>> I tried your suggestions but it leads to the same error message 
>> (using g++ -shared foo.cpp -o foo.so  -I /usr/include/python2.6 
>> -lpython2.6 -lboost_python).
>> Could there be something wrong with the installation?
>
> Your error message suggests the compiler couldn't find the Python 
> headers. Do you have Python 2.6 installed ? Are the headers in 
> /usr/include/python2.6 ? If not, you need to adjust your command line 
> to whatever Python installation you have. (If you use system python 
> packages, make sure to also install the "python-dev" package, which 
> provides the headers as well as linkable DSOs.)
>
> HTH,
>         Stefan
>

Hi,

I think there is still unnecessary space here: -I /usr/include/python2.6
Should be just -I/usr/include/python2.6

Cheers,
Sebastian




From jacob at davisphoto.net  Sun Dec 12 22:02:24 2010
From: jacob at davisphoto.net (Jacob Davis)
Date: Sun, 12 Dec 2010 14:02:24 -0700
Subject: [C++-sig] problems with typedef void *
Message-ID: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>

Howdy!

I'm wrapping a C libary with Boost Python and having a really difficult time
with void * typedefs. For example, here's a typedef and the function header
that uses it:

typedef void * EmoEngineEventHandle;
EmoEngineEventHandle EE_EmoEngineEventCreate();

And my undoubtedly overly simplistic wrapper for the function:

def("EE_EmoEngineEventCreate", EE_EmoEngineEventCreate);

The compiler complains with lots of similar messages:

1>C:\boost_1_45_0\boost/python/detail/caller.hpp(223) : error C2027: use of
undefined type
'boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<T>'
1>        with
1>        [
1>            T=result_t
1>        ]
1>        C:\boost_1_45_0\boost/python/detail/caller.hpp(200) : while
compiling class template member function 'PyObject
*boost::python::detail::caller_arity<0>::impl<F,Policies,Sig>::operator
()(PyObject *,PyObject *)'
1>        with
1>        [
1>            F=void (__cdecl *)(void),
1>            Policies=boost::python::default_call_policies,
1>            Sig=boost::mpl::vector1<EmoEngineEventHandle>
1>        ]

There are also functions that pass these typedefs as args, and I expect I'll
have problems with these, too. i.e.:

void EE_EmoEngineEventFree(EmoEngineEventHandle hEvent);

There are a *lot* of these functions in the library I'm wrapping that use
and return void * typedefs. Is there a way to translate the typedefs to get
all of my function wrappers to pass them properly? Failing that, how do I
modify my wrappers to support the typedefs?

Much obliged,
Jacob Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101212/53f4c844/attachment.html>

From talljimbo at gmail.com  Sun Dec 12 23:04:31 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Sun, 12 Dec 2010 14:04:31 -0800
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
Message-ID: <4D0546EF.8040905@gmail.com>

On 12/12/2010 01:02 PM, Jacob Davis wrote:
> Howdy!
>
> I'm wrapping a C libary with Boost Python and having a really difficult
> time with void * typedefs. For example, here's a typedef and the
> function header that uses it:
>
> typedef void * EmoEngineEventHandle;
> EmoEngineEventHandle EE_EmoEngineEventCreate();
>
<snip>
> There are also functions that pass these typedefs as args, and I expect
> I'll have problems with these, too. i.e.:
>
> void EE_EmoEngineEventFree(EmoEngineEventHandle hEvent);
>
> There are a *lot* of these functions in the library I'm wrapping that
> use and return void * typedefs. Is there a way to translate the typedefs
> to get all of my function wrappers to pass them properly? Failing that,
> how do I modify my wrappers to support the typedefs?
>

Hopefully someone else will have a more elegant solution, because 
everything I can think of is pretty ugly.  Boost.Python is totally based 
on templates, and the complete lack of truly distinct types is a big 
problem, because it has no way to know what Python types to map to when 
all the C++ types are indistinct.

I think you'll want to define a custom wrapper class for each of these 
typedefs with a distinct type.  From your example, it looks like many of 
these might be similar enough that you could use a template:

----------------------------------------------------

enum VoidTag { EmoEngineEventHandleTag, ... };

template <VoidTag tag, void * (*ctor)(), void (*dtor)(void * )>
struct VoidWrapper : private boost::noncopyable {

     VoidWrapper() : p(ctor()) {}

     ~VoidWrapper() { dtor(p); }

     void * p;

};

typedef VoidWrapper<
     EmoEngineEventHandleTag,
     &EE_EmoEngineEventCreate,
     &EE_EmoEngineEventFree
 > EmoEngineEventHandleWrapper;

-----------------------------------------------------

Now, for all the other (non-constructor/destructor) functions that take 
these, you'll need to write a function wrapper that takes arguments by 
the wrapper classes, calls the C function, and returns using the wrapper 
classes (you'll have to think about ownership semantics in that case, of 
course).  Then you can use Boost.Python to wrap the wrapper functions. 
You might be able to use templates for the function wrappers too, if 
certain signatures are common enough.

You could also approach this by writing custom Boost.Python call 
policies, I think, but that would require a lot of advanced Boost.Python-fu.

Finally, I wouldn't normally recommend SWIG, because I think 
Boost.Python is generally much better, especially for wrapping C++ code. 
  But in this case, since you're wrapping pure C using a lot of opaque 
pointers, SWIG's approach might be a better fit, since its code 
generation naturally considers everything an opaque pointer.  If you 
don't have to use Boost.Python for other reasons, it might be worth a 
look, at least.

Good luck!

Jim Bosch

From seefeld at sympatico.ca  Mon Dec 13 00:23:47 2010
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Sun, 12 Dec 2010 18:23:47 -0500
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
Message-ID: <4D055983.8070601@sympatico.ca>

On 12/12/2010 04:02 PM, Jacob Davis wrote:
> Howdy!
>
> I'm wrapping a C libary with Boost Python and having a really 
> difficult time with void * typedefs. For example, here's a typedef and 
> the function header that uses it:
>
> typedef void * EmoEngineEventHandle;
> EmoEngineEventHandle EE_EmoEngineEventCreate();
>
> And my undoubtedly overly simplistic wrapper for the function:
>
> def("EE_EmoEngineEventCreate", EE_EmoEngineEventCreate);
>
> The compiler complains with lots of similar messages:
>
> 1>C:\boost_1_45_0\boost/python/detail/caller.hpp(223) : error C2027: 
> use of undefined type 
> 'boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<T>'

There are a couple of issues with your approach. Note that the line 
above is a trick to pass a message through a (very noisy) compiler 
error. The "specify_a_return_value_policy..." piece above is telling you 
that you can't wrap a function returning a void pointer without 
explicitly telling boost.python what to do with the pointer. Normally it 
would attempt to pass by-value, but that obviously doesn't work (you 
can't dereference a void pointer !).
But, if you want to pass by-reference, how should the pointer be treated 
? Who owns the data ?

More importantly, you don't use a strong type system in your API, since 
"types" are aliasing void pointers. Thus, the compiler can't help you 
distinguish e.g. function overloads, and there is no way to 
auto-generate a type registry.

You may wrap your API with a C++ wrapper, though I honestly doubt this 
is practical. In fact, I'm in doubt that boost.python is the right 
choice for your Python wrapping effort. Have you considered alternatives ?

     Stefan

-- 

       ...ich hab' noch einen Koffer in Berlin...


From s_sourceforge at nedprod.com  Mon Dec 13 12:03:53 2010
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Mon, 13 Dec 2010 11:03:53 -0000
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
Message-ID: <4D05FD99.3039.7130A86@s_sourceforge.nedprod.com>

On 12 Dec 2010 at 14:02, Jacob Davis wrote:

> I'm wrapping a C libary with Boost Python and having a really difficult time
> with void * typedefs. For example, here's a typedef and the function header
> that uses it:
> 
> typedef void * EmoEngineEventHandle;
> EmoEngineEventHandle EE_EmoEngineEventCreate();
> [snip]
> There are a *lot* of these functions in the library I'm wrapping that use
> and return void * typedefs. Is there a way to translate the typedefs to get
> all of my function wrappers to pass them properly? Failing that, how do I
> modify my wrappers to support the typedefs?

It's actually easier than you might think.

Firstly I would suggest you look at the thread regarding the patch I 
submitted in 2006 adding void * support to BPL which starts at 
http://mail.python.org/pipermail/cplusplus-sig/2006-
February/009836.html. What my original patch did was to declare a 
trampoline type struct void_ {}; and then cast all void * to void_ *. 
This gave the BPL runtime type registry enough information to work 
with.

(I might add that as this thread showed, Dave scrubbed my trampoline 
type implementation for something better. I know not what, but it may 
be worth your while finding out)

Secondly last time I looked GCCXML nowadays dumps out typedefs, so if 
you feed your C headers to GCCXML you can produce a list of all 
possible typedef names. It's now trivial to modify a Py++ or whatever 
bindings generator to spew out a cast to for example a struct 
EmoEngineEventHandle_ * whenever it sees a EmoEngineEventHandle.

And that ought to do it. Shouldn't take you more than thirty or forty 
hours. And if you integrate all these steps into a build process it 
can all be automated once you've completed it. Also, Roman may or may 
not be interested in adding this feature to Py++.

Hope this helped,
Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From jacob at davisphoto.net  Tue Dec 14 22:03:04 2010
From: jacob at davisphoto.net (Jacob Davis)
Date: Tue, 14 Dec 2010 14:03:04 -0700
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <4D0546EF.8040905@gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
	<4D0546EF.8040905@gmail.com>
Message-ID: <AANLkTi=e+jAxFtvyd1TT6+2otg4ApNF66NSQWj56xwPx@mail.gmail.com>

On Sun, Dec 12, 2010 at 3:04 PM, Jim Bosch <talljimbo at gmail.com> wrote:

> I think you'll want to define a custom wrapper class for each of these
> typedefs with a distinct type.  From your example, it looks like many of
> these might be similar enough that you could use a template:
>
> [snip]

>
> Now, for all the other (non-constructor/destructor) functions that take
> these, you'll need to write a function wrapper that takes arguments by the
> wrapper classes, calls the C function, and returns using the wrapper classes
> (you'll have to think about ownership semantics in that case, of course).
>  Then you can use Boost.Python to wrap the wrapper functions.
>
 [snip]

>
> Finally, I wouldn't normally recommend SWIG, because I think Boost.Python
> is generally much better, especially for wrapping C++ code.  But in this
> case, since you're wrapping pure C using a lot of opaque pointers, SWIG's
> approach might be a better fit, since its code generation naturally
> considers everything an opaque pointer.  If you don't have to use
> Boost.Python for other reasons, it might be worth a look, at least.
>
>
Thanks Jim!  I kind of expected that I would need to do wrapper classes and
functions, but was hoping it wouldn't quite come to that. I Niall's patch
doesn't work out well, I might just do this. As for SWIG, I considered it,
but chose Boost for the sake of learning a bit more about it. Maybe it would
be wise to just scrap it and go with SWIG though.

- Jacob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101214/31f01934/attachment.html>

From roman.yakovenko at gmail.com  Tue Dec 14 22:20:25 2010
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 14 Dec 2010 23:20:25 +0200
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <AANLkTi=e+jAxFtvyd1TT6+2otg4ApNF66NSQWj56xwPx@mail.gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>
	<4D0546EF.8040905@gmail.com>
	<AANLkTi=e+jAxFtvyd1TT6+2otg4ApNF66NSQWj56xwPx@mail.gmail.com>
Message-ID: <AANLkTi=uGCL6crqYLhGpSHC6BTrchmXx1g2R2kQprSPh@mail.gmail.com>

On Tue, Dec 14, 2010 at 11:03 PM, Jacob Davis <jacob at davisphoto.net> wrote:

>
>
> On Sun, Dec 12, 2010 at 3:04 PM, Jim Bosch <talljimbo at gmail.com> wrote:
>
>> I think you'll want to define a custom wrapper class for each of these
>> typedefs with a distinct type.  From your example, it looks like many of
>> these might be similar enough that you could use a template:
>>
>> [snip]
>
>>
>> Now, for all the other (non-constructor/destructor) functions that take
>> these, you'll need to write a function wrapper that takes arguments by the
>> wrapper classes, calls the C function, and returns using the wrapper classes
>> (you'll have to think about ownership semantics in that case, of course).
>>  Then you can use Boost.Python to wrap the wrapper functions.
>>
>  [snip]
>
>>
>> Finally, I wouldn't normally recommend SWIG, because I think Boost.Python
>> is generally much better, especially for wrapping C++ code.  But in this
>> case, since you're wrapping pure C using a lot of opaque pointers, SWIG's
>> approach might be a better fit, since its code generation naturally
>> considers everything an opaque pointer.  If you don't have to use
>> Boost.Python for other reasons, it might be worth a look, at least.
>>
>>
> Thanks Jim!  I kind of expected that I would need to do wrapper classes and
> functions, but was hoping it wouldn't quite come to that. I Niall's patch
> doesn't work out well, I might just do this. As for SWIG, I considered it,
> but chose Boost for the sake of learning a bit more about it. Maybe it would
> be wise to just scrap it and go with SWIG though.
>
>
>From my experience, if you have to expose a C API and you have a dynamic
library, the best way to access it is to use ctypes (which comes with
Python). If the API is pretty complex, then you can create a facade to it,
but already in Python. ctypes module removes a huge amount of headaches.

HTH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101214/c84e35f7/attachment.html>

From s_sourceforge at nedprod.com  Tue Dec 14 23:28:28 2010
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Tue, 14 Dec 2010 22:28:28 -0000
Subject: [C++-sig] problems with typedef void *
In-Reply-To: <AANLkTinH+z6Jd+-PjR+7oa7Mbx3kGXwNzR8EPv7CooxE@mail.gmail.com>
References: <AANLkTi=f3tmASsZKshY8Pd4oQEf1q85rskTSjUtOX0NF@mail.gmail.com>,
	<4D05FD99.3039.7130A86@s_sourceforge.nedprod.com>,
	<AANLkTinH+z6Jd+-PjR+7oa7Mbx3kGXwNzR8EPv7CooxE@mail.gmail.com>
Message-ID: <4D07EF8C.20787.EAC2676@s_sourceforge.nedprod.com>

On 14 Dec 2010 at 14:07, Jacob Davis wrote:

> I read through the thread back when I started work on this problem, but
> figured if it had been scrubbed out it might not be wise to use it. I'll go
> back and check it out soon, though.

Trampoline types are a very commonly used idiom in C++ 
metaprogramming. They let you work around pre-C++0x limitations.

> The last I saw on the subject, void
> pointers were supposed to "just work", but I wonder if they "just work"
> outside the context of a typedef obfuscation. If this doesn't work out, I'll
> probably just do an extra layer of wrapping as Jim suggested.

There is absolutely no reason why you can't declare all your 
typedefed void * as opaque pointers and get to work. The only problem 
will be that BPL won't know what to do with them (as they all look 
like void * and therefore are indistinguishable) and therefore can't 
touch them in any way except to pass them around unmodified. This 
probably won't be much use to you.

I might add that another - and better solution - is to modify the C 
library to use proper opaque structures like you're supposed to in C 
rather than void * for everything. I would assume that below the 
header API its source is casting them to internal structures anyway, 
so you should be able to modify its header to use declared but not 
defined structs with the names that it's typedefing them to e.g.

typedef void *EmoEngineEventHandle;

... this goes to:

struct EmoEngineEventHandle_t;
typedef struct EmoEngineEventHandle_t *EmoEngineEventHandle;

Theoretically this should let you compile your C library just fine if 
it is indeed casting to internal structs. Even if it doesn't, it 
would be very easy to fix.

It isn't 100% legal C or C++ though as it's type punning. However I 
can't think of a compiler which won't allow it.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From ndbecker2 at gmail.com  Wed Dec 15 01:54:40 2010
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 14 Dec 2010 19:54:40 -0500
Subject: [C++-sig] expose functor?
Message-ID: <ie93kh$dju$1@dough.gmane.org>

Suppose I have a functor, e.g.:

template<typename F>
struct mag_sqr1 {
  F operator() (F z) {
    return (z * z);
  }
};

Any way to expose this a a python function?

def ("mag_sqr", ???);


From austin.bingham at gmail.com  Mon Dec 20 16:04:06 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Mon, 20 Dec 2010 16:04:06 +0100
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
Message-ID: <AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>

I'm in the process of upgrading my project to boost-1.45.0 and
python3, and I started seeing strange problems with some of my
boost.python-based extension modules. Essentially, some of my modules
are reporting the wrong module name, i.e. if I
"print(module_object.__name__)", I get the wrong value. I threw this
into gdb, and it looks like the problem might have to do with the
implementation of bp::detail::init_module().

I ran python under gdb, but a breakpoint on init_module(), and
imported the modules with which I'm seeing problems. The first call to
init_module() initializes the static PyModuleDef struct "moduledef"
properly. However, the second call shows that the "m_name" field it
left as-is from the first call (since the object is static); I would
have expected that it needs to be updated with the new module name.
This stale (I think) module def is then used for the rest of the
initialization, and I'm guessing that this is the cause of the
problems I'm seeing. Certainly, it seems that the name of the first
module imported seems to be getting used as the name of the second
module.

I only noticed this problem when I tried to pickle some of the types
exposed by my module. The pickle module in python3 complained that it
didn't know anything about the broken modules I was getting.

Is this a bug in boost.python? I plan to write a small patch tomorrow
to set the m_name field in init_module(), so maybe this will fix the
problem.

If this isn't a boost.python bug, does anyone have any idea what might
be going wrong? And how is it correct that the static moduledef is
reused without updating its name?

Thanks,
Austin

From rwgk at yahoo.com  Tue Dec 21 00:48:35 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Mon, 20 Dec 2010 15:48:35 -0800 (PST)
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
Message-ID: <368567.86690.qm@web111407.mail.gq1.yahoo.com>

Hi Austin,
Just a remark: the python3 support isn't routinely exercised.
Overlooked incompatibilities are possible.
I only have time to maintain the python2 support. We need a
volunteer to port the unit tests to python3.
Ralf



----- Original Message ----
> From: Austin Bingham <austin.bingham at gmail.com>
> To: Development of Python/C++ integration <cplusplus-sig at python.org>
> Sent: Mon, December 20, 2010 7:04:06 AM
> Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing 
>conflicts?
> 
> I'm in the process of upgrading my project to boost-1.45.0 and
> python3, and I  started seeing strange problems with some of my
> boost.python-based extension  modules. Essentially, some of my modules
> are reporting the wrong module name,  i.e. if I
> "print(module_object.__name__)", I get the wrong value. I threw  this
> into gdb, and it looks like the problem might have to do with  the
> implementation of bp::detail::init_module().
> 
> I ran python under  gdb, but a breakpoint on init_module(), and
> imported the modules with which  I'm seeing problems. The first call to
> init_module() initializes the static  PyModuleDef struct "moduledef"
> properly. However, the second call shows that  the "m_name" field it
> left as-is from the first call (since the object is  static); I would
> have expected that it needs to be updated with the new  module name.
> This stale (I think) module def is then used for the rest of  the
> initialization, and I'm guessing that this is the cause of  the
> problems I'm seeing. Certainly, it seems that the name of the  first
> module imported seems to be getting used as the name of the  second
> module.
> 
> I only noticed this problem when I tried to pickle some  of the types
> exposed by my module. The pickle module in python3 complained  that it
> didn't know anything about the broken modules I was  getting.
> 
> Is this a bug in boost.python? I plan to write a small patch  tomorrow
> to set the m_name field in init_module(), so maybe this will fix  the
> problem.
> 
> If this isn't a boost.python bug, does anyone have any  idea what might
> be going wrong? And how is it correct that the static  moduledef is
> reused without updating its  name?
> 
> Thanks,
> Austin
> _______________________________________________
> Cplusplus-sig  mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
> 

From austin.bingham at gmail.com  Tue Dec 21 08:29:04 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Tue, 21 Dec 2010 08:29:04 +0100
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <368567.86690.qm@web111407.mail.gq1.yahoo.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
	<368567.86690.qm@web111407.mail.gq1.yahoo.com>
Message-ID: <AANLkTinR1=5c6Pah8tcGQur9CrJGjQXvknAo7nBudWzi@mail.gmail.com>

I'm happy to help with this, at least as my time permits. I've looked
at the docs regarding the regression tests, and everything seems
fairly straightforward (though I've never really wrapped my head
around jam!). Is it just a question of building boost against python3,
running the tests, and fixing whatever breaks?

Austin

On Tue, Dec 21, 2010 at 12:48 AM, Ralf W. Grosse-Kunstleve
<rwgk at yahoo.com> wrote:
> Hi Austin,
> Just a remark: the python3 support isn't routinely exercised.
> Overlooked incompatibilities are possible.
> I only have time to maintain the python2 support. We need a
> volunteer to port the unit tests to python3.

From trigves at yahoo.com  Tue Dec 21 08:43:25 2010
From: trigves at yahoo.com (Trigve Siver)
Date: Mon, 20 Dec 2010 23:43:25 -0800 (PST)
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <368567.86690.qm@web111407.mail.gq1.yahoo.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
	<368567.86690.qm@web111407.mail.gq1.yahoo.com>
Message-ID: <160896.92539.qm@web110407.mail.gq1.yahoo.com>

> From: Ralf W. Grosse-Kunstleve <rwgk at yahoo.com>

> To: Development of Python/C++ integration <cplusplus-sig at python.org>
> Sent: Tue, December 21, 2010 12:48:35 AM
> Subject: Re: [C++-sig] Multiple calls to bp::detail::init_module() causing 
>conflicts?
> 
> Hi Austin,
> Just a remark: the python3 support isn't routinely  exercised.
> Overlooked incompatibilities are possible.
> I only have time to  maintain the python2 support. We need a
> volunteer to port the unit tests to  python3.
> Ralf
> 

I'm testing release branch against python 3 (see Trigve in 
http://www.boost.org/development/tests/release/developer/summary.html). Adding 
trunk testing against python 3 would be also welcomed.

Trigve



      

From rwgk at yahoo.com  Tue Dec 21 09:21:42 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 21 Dec 2010 00:21:42 -0800 (PST)
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <160896.92539.qm@web110407.mail.gq1.yahoo.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
	<368567.86690.qm@web111407.mail.gq1.yahoo.com>
	<160896.92539.qm@web110407.mail.gq1.yahoo.com>
Message-ID: <499663.70683.qm@web111409.mail.gq1.yahoo.com>

> I'm testing  release branch against python 3 (see Trigve in 

> http://www.boost.org/development/tests/release/developer/summary.html).  Adding 
>
> trunk testing against python 3 would be also  welcomed.

Oh, sorry, I must be unaware of something major...
boost/libs/python/test/*.py are written in Python2 syntax.
How are these tests run with Python3? 2to3 behind the scenes?

Ralf

From austin.bingham at gmail.com  Tue Dec 21 09:23:52 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Tue, 21 Dec 2010 09:23:52 +0100
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <499663.70683.qm@web111409.mail.gq1.yahoo.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
	<368567.86690.qm@web111407.mail.gq1.yahoo.com>
	<160896.92539.qm@web110407.mail.gq1.yahoo.com>
	<499663.70683.qm@web111409.mail.gq1.yahoo.com>
Message-ID: <AANLkTik8wSf0zATG0u3t_Dt=fLWT-s=4bFLurPdcJ_FO@mail.gmail.com>

I think so...I saw a 2to3 being executed when I tried running the
regression tests, at least.

Austin

On Tue, Dec 21, 2010 at 9:21 AM, Ralf W. Grosse-Kunstleve
<rwgk at yahoo.com> wrote:
>> I'm testing ?release branch against python 3 (see Trigve in
>
>> http://www.boost.org/development/tests/release/developer/summary.html). ?Adding
>>
>> trunk testing against python 3 would be also ?welcomed.
>
> Oh, sorry, I must be unaware of something major...
> boost/libs/python/test/*.py are written in Python2 syntax.
> How are these tests run with Python3? 2to3 behind the scenes?
>
> Ralf
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From austin.bingham at gmail.com  Thu Dec 23 11:01:53 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Thu, 23 Dec 2010 11:01:53 +0100
Subject: [C++-sig] builtin_converters.hpp using outdated API on win64 builds
Message-ID: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>

I'm currently trying to build boost-1.45 against python-3.1.3 on win64
(visual studio 2009 sp1). This build is failing because
boost/python/converter/builtin_converters.hpp is using APIs that have
been removed from python3 without checking the python version; at
lines 138 and 144 PyInt_FromLong is being used.

If I understand what's going in that code, the fix is to update line
125 to check the python version. I think we can just add:

  && PY_VERSION_HEX < 0x03000000

to the preprocessor conditional.

I've made this change locally, and it seems to fix things. Does this
seem like the correct fix? If so, how do I get a patch to the right
people (or at least ask them to make this change)?

Austin

From rwgk at yahoo.com  Thu Dec 23 12:49:49 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 23 Dec 2010 03:49:49 -0800 (PST)
Subject: [C++-sig] builtin_converters.hpp using outdated API on win64
	builds
In-Reply-To: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
References: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
Message-ID: <487504.66064.qm@web111412.mail.gq1.yahoo.com>

----- Original Message ----

> I'm currently trying to build boost-1.45 against python-3.1.3 on  win64
> (visual studio 2009 sp1). This build is failing  because
> boost/python/converter/builtin_converters.hpp is using APIs that  have
> been removed from python3 without checking the python version;  at
> lines 138 and 144 PyInt_FromLong is being used.
> 
> If I understand  what's going in that code, the fix is to update line
> 125 to check the python  version. I think we can just add:
> 
>   && PY_VERSION_HEX <  0x03000000
> 
> to the preprocessor conditional.
> 
> I've made this change  locally, and it seems to fix things. Does this
> seem like the correct fix?

Did you test what happens when you have a function returning a size_t?
E.g.

std::size_t dummy() { return 123; }

def("dummy", dummy);

In Python3, what is the Python type you get?
In Python2 it is 123L without the special code for WIN64.

Ralf

From austin.bingham at gmail.com  Thu Dec 23 13:22:47 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Thu, 23 Dec 2010 13:22:47 +0100
Subject: [C++-sig] builtin_converters.hpp using outdated API on win64
	builds
In-Reply-To: <487504.66064.qm@web111412.mail.gq1.yahoo.com>
References: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
	<487504.66064.qm@web111412.mail.gq1.yahoo.com>
Message-ID: <AANLkTi=-AyKY4fGN4tz46HHto1ZFU=uDF-eD-AsEMAjR@mail.gmail.com>

On Thu, Dec 23, 2010 at 12:49 PM, Ralf W. Grosse-Kunstleve
<rwgk at yahoo.com> wrote:
> ----- Original Message ----
>
>> I'm currently trying to build boost-1.45 against python-3.1.3 on ?win64
>> (visual studio 2009 sp1). This build is failing ?because
>> boost/python/converter/builtin_converters.hpp is using APIs that ?have
>> been removed from python3 without checking the python version; ?at
>> lines 138 and 144 PyInt_FromLong is being used.
>>
>> If I understand ?what's going in that code, the fix is to update line
>> 125 to check the python ?version. I think we can just add:
>>
>> ? && PY_VERSION_HEX < ?0x03000000
>>
>> to the preprocessor conditional.
>>
>> I've made this change ?locally, and it seems to fix things. Does this
>> seem like the correct fix?
>
> Did you test what happens when you have a function returning a size_t?
> E.g.
>
> std::size_t dummy() { return 123; }
>
> def("dummy", dummy);
>
> In Python3, what is the Python type you get?
> In Python2 it is 123L without the special code for WIN64.

I haven't directly tested it, but as I understand things there is only
one built-in integral type (int) in python3:

   http://docs.python.org/release/3.0.1/whatsnew/3.0.html#integers

If/when I get a chance, I'll try to answer this question a bit more
authoritatively. However, it seems pretty clear to me that size_t must
get translated to and int in python3.

Austin

From matt-bradbury at live.co.uk  Thu Dec 23 14:12:40 2010
From: matt-bradbury at live.co.uk (Matthew Bradbury)
Date: Thu, 23 Dec 2010 13:12:40 +0000 (UTC)
Subject: [C++-sig]
	=?utf-8?q?builtin=5Fconverters=2Ehpp_using_outdated_API?=
	=?utf-8?q?_on_win64_builds?=
References: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
Message-ID: <loom.20101223T140801-361@post.gmane.org>

> I've made this change locally, and it seems to fix things. Does this
> seem like the correct fix? If so, how do I get a patch to the right
> people (or at least ask them to make this change)?
> 
> Austin
> 


There is a ticket in boost for this
(https://svn.boost.org/trac/boost/ticket/4627) where you can submit patches.


From rwgk at yahoo.com  Thu Dec 23 17:34:10 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 23 Dec 2010 08:34:10 -0800 (PST)
Subject: [C++-sig] builtin_converters.hpp using outdated API on win64
	builds
In-Reply-To: <AANLkTi=-AyKY4fGN4tz46HHto1ZFU=uDF-eD-AsEMAjR@mail.gmail.com>
References: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
	<487504.66064.qm@web111412.mail.gq1.yahoo.com>
	<AANLkTi=-AyKY4fGN4tz46HHto1ZFU=uDF-eD-AsEMAjR@mail.gmail.com>
Message-ID: <508597.8890.qm@web111411.mail.gq1.yahoo.com>

Hi Austin,

> I haven't directly tested it, but as I understand things there  is only

> one built-in integral type (int) in python3:
> 
>     http://docs.python.org/release/3.0.1/whatsnew/3.0.html#integers
> 
> If/when I  get a chance, I'll try to answer this question a bit more
> authoritatively.  However, it seems pretty clear to me that size_t must
> get translated to and  int in  python3.

OK, I'll check in your suggested change.

Ralf

From austin.bingham at gmail.com  Fri Dec 24 07:32:37 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Fri, 24 Dec 2010 07:32:37 +0100
Subject: [C++-sig] builtin_converters.hpp using outdated API on win64
	builds
In-Reply-To: <508597.8890.qm@web111411.mail.gq1.yahoo.com>
References: <AANLkTinYBuuXGpEn6zCnn82jCVe=kZEPZgnzTka+6rbH@mail.gmail.com>
	<487504.66064.qm@web111412.mail.gq1.yahoo.com>
	<AANLkTi=-AyKY4fGN4tz46HHto1ZFU=uDF-eD-AsEMAjR@mail.gmail.com>
	<508597.8890.qm@web111411.mail.gq1.yahoo.com>
Message-ID: <AANLkTinACJRVzjhjQMyrV8DJnODXM9H08G_yZytzBXqw@mail.gmail.com>

> OK, I'll check in your suggested change.

Great, I see it in the trunk now. Thanks!

Austin

From austin.bingham at gmail.com  Sat Dec 25 10:18:37 2010
From: austin.bingham at gmail.com (Austin Bingham)
Date: Sat, 25 Dec 2010 10:18:37 +0100
Subject: [C++-sig] Multiple calls to bp::detail::init_module() causing
	conflicts?
In-Reply-To: <AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
References: <AANLkTikwr2rX6-AU4pqBotAcOhYrk-WGj7r_CaB0WYUE@mail.gmail.com>
	<AANLkTinLTXeYQbnGoCMg+a6Ykrw6xLpPOUpR-Q_4DZrV@mail.gmail.com>
Message-ID: <AANLkTik_1neJf70FsSi0eULvpUwPBDhnCn7S4+s4UA__@mail.gmail.com>

I've attached a patch that implements what I think is a reasonable fix
for the problem I found. Essentially, it implements
BOOST_PYTHON_MODULE under python3 such that each invocation of the
macro creates a new static PyModuleDef. I also reimplemented
bp::detail::init_module for the python3 case to take a PyModuleDef
parameter rather than a name.

As far as I can tell, these changes work well. I ran bjam in
"libs/python/test" and didn't see any failures...I saw a lot of lines
saying "**passed**", but none saying "**failed**", so I assume things
are ok. If there are more tests that need to be run, please let me
know, and tell me how to run them.

I also tested the changes against a project which was manifesting the
original problem I described, and the problems went away. That is,
with the changes in this patch, my extension modules all have the
correct "__name__" now.

Austin

On Mon, Dec 20, 2010 at 4:04 PM, Austin Bingham
<austin.bingham at gmail.com> wrote:
> I'm in the process of upgrading my project to boost-1.45.0 and
> python3, and I started seeing strange problems with some of my
> boost.python-based extension modules. Essentially, some of my modules
> are reporting the wrong module name, i.e. if I
> "print(module_object.__name__)", I get the wrong value. I threw this
> into gdb, and it looks like the problem might have to do with the
> implementation of bp::detail::init_module().
>
> I ran python under gdb, but a breakpoint on init_module(), and
> imported the modules with which I'm seeing problems. The first call to
> init_module() initializes the static PyModuleDef struct "moduledef"
> properly. However, the second call shows that the "m_name" field it
> left as-is from the first call (since the object is static); I would
> have expected that it needs to be updated with the new module name.
> This stale (I think) module def is then used for the rest of the
> initialization, and I'm guessing that this is the cause of the
> problems I'm seeing. Certainly, it seems that the name of the first
> module imported seems to be getting used as the name of the second
> module.
>
> I only noticed this problem when I tried to pickle some of the types
> exposed by my module. The pickle module in python3 complained that it
> didn't know anything about the broken modules I was getting.
>
> Is this a bug in boost.python? I plan to write a small patch tomorrow
> to set the m_name field in init_module(), so maybe this will fix the
> problem.
>
> If this isn't a boost.python bug, does anyone have any idea what might
> be going wrong? And how is it correct that the static moduledef is
> reused without updating its name?
>
> Thanks,
> Austin
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: python3_bp_module.diff
Type: text/x-diff
Size: 5334 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101225/9bab7240/attachment.diff>

From andrew.ho at utah.edu  Thu Dec 30 09:04:12 2010
From: andrew.ho at utah.edu (Andrew Ho)
Date: Thu, 30 Dec 2010 01:04:12 -0700
Subject: [C++-sig] Boost.Python and python 3.x?
Message-ID: <AANLkTikGwdzmQ9-5muWSMs8GnAgm5CtKjYQ7r9TmKRZf@mail.gmail.com>

Hi, I'm trying to build Boost.Python to use Python 3.1, however I'm getting
a bunch of errors (missing functions/members, etc.).
I was able to get Boost.Python to compile correctly when using Python 2.6 so
I don't think the problem is in my setup.
I did a search and as far as I can tell there's no support for python 3.x
being built at Boost 1.41, however I haven't found anything for Boost 1.45.
There were references to a GSoC project to fix this, did this ever get
finished? If so, is there something different between the Python 3.x and
Python 2.x jam files that would cause this (other than referencing the
correct includes & libs)?

Here's my jam file for python 3.1 (C:\Python31 contains the source download
of Python 3.1.2, built for Windows x64 release platforms):

using msvc ;
import os ;
using python : 3.1
   : # sources
        C:\\Python31
    : # requirements
        <include>C:\\Python31\\Include
        <include>C:\\Python31\\PC
        <library-path>C:\\Python31\\PCBuild\\amd64
    ;

And I'm compile with this command:
bjam link=static threading=multi address-model=64 variant=release
toolset=msvc-9.0 -a --with-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101230/768130fb/attachment.html>

From helloworld922 at gmail.com  Thu Dec 30 09:10:26 2010
From: helloworld922 at gmail.com (Andrew Ho)
Date: Thu, 30 Dec 2010 01:10:26 -0700
Subject: [C++-sig] Boost.Python and python 3.x?
Message-ID: <AANLkTin0J2F_boEExkLhh8TzaWN1+-Tg-oLW3z2ivUn0@mail.gmail.com>

Hi, I'm trying to build Boost.Python to use Python 3.1, however I'm getting
a bunch of errors (missing functions/members, etc.).
I was able to get Boost.Python to compile correctly when using Python 2.6 so
I don't think the problem is in my setup.
I did a search and as far as I can tell there's no support for python 3.x
being built at Boost 1.41, however I haven't found anything for Boost 1.45.
There were references to a GSoC project to fix this, did this ever get
finished? If so, is there something different between the Python 3.x and
Python 2.x jam files that would cause this (other than referencing the
correct includes & libs)?

Here's my jam file for python 3.1 (C:\Python31 contains the source download
of Python 3.1.2, built for Windows x64 release platforms):

using msvc ;
import os ;
using python : 3.1
   : # sources
        C:\\Python31
    : # requirements
        <include>C:\\Python31\\Include
        <include>C:\\Python31\\PC
        <library-path>C:\\Python31\\
PCBuild\\amd64
    ;

And I'm compile with this command:
bjam link=static threading=multi address-model=64 variant=release
toolset=msvc-9.0 -a --with-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101230/cb384489/attachment.html>

From rwgk at yahoo.com  Thu Dec 30 18:33:27 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 30 Dec 2010 09:33:27 -0800 (PST)
Subject: [C++-sig] Boost.Python and python 3.x?
In-Reply-To: <AANLkTikGwdzmQ9-5muWSMs8GnAgm5CtKjYQ7r9TmKRZf@mail.gmail.com>
References: <AANLkTikGwdzmQ9-5muWSMs8GnAgm5CtKjYQ7r9TmKRZf@mail.gmail.com>
Message-ID: <452571.41742.qm@web111411.mail.gq1.yahoo.com>

Hi Andrew,

Yes, the GSoC project was finished.
I checked in a few Python3 related fixes after the
1.45 release that may be important for you, in
particular the one below, which you could easily apply
manually to your installation.

Ralf

Index: boost/python/converter/builtin_converters.hpp
===================================================================
--- boost/python/converter/builtin_converters.hpp       (revision 67431)
+++ boost/python/converter/builtin_converters.hpp       (revision 67432)
@@ -122,7 +122,7 @@
 BOOST_PYTHON_TO_INT(int)
 BOOST_PYTHON_TO_INT(long)

-# if defined(_MSC_VER) && defined(_WIN64)
+# if defined(_MSC_VER) && defined(_WIN64) && PY_VERSION_HEX < 0x03000000
 /* Under 64-bit Windows std::size_t is "unsigned long long". To avoid
    getting a Python long for each std::size_t the value is checked before
    the conversion. A std::size_t is converted to a simple Python int




>
>From: Andrew Ho <andrew.ho at utah.edu>
>To: boost.python mailing list <cplusplus-sig at python.org>
>Sent: Thu, December 30, 2010 12:04:12 AM
>Subject: [C++-sig] Boost.Python and python 3.x?
>
>Hi, I'm trying to build Boost.Python to use Python 3.1, however I'm getting a 
>bunch of errors (missing functions/members, etc.).
>I was able to get Boost.Python to compile correctly when using Python 2.6 so I 
>don't think the problem is in my setup.
>I did a search and as far as I can tell there's no support for python 3.x being 
>built at Boost 1.41, however I haven't found anything for Boost 1.45. There were 
>references to a GSoC project to fix this, did this ever get finished? If so, is 
>there something different between the Python 3.x and Python 2.x jam files that 
>would cause this (other than referencing the correct includes & libs)?
>
>Here's my jam file for python 3.1 (C:\Python31 contains the source download of 
>Python 3.1.2, built for Windows x64 release platforms):
>
>using msvc ;
>import os ;
>using python : 3.1
>   : # sources
>        C:\\Python31
>    : # requirements
>        <include>C:\\Python31\\Include
>        <include>C:\\Python31\\PC
>        <library-path>C:\\Python31\\PCBuild\\amd64
>    ;
>
>And I'm compile with this command:
>bjam link=static threading=multi address-model=64 variant=release 
>toolset=msvc-9.0 -a --with-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20101230/753cd0ee/attachment.html>