[C++-sig] "version mismatch?" because of libpython link problem?

[Amos Anderson]

Summary: I have a setup which works on Linux, but not OSX 10.6.3. I
want my build system to build its own python, build boost linking to
that python, build my extensions linking to boost, and then run my
python application with my c++ extensions. I'm getting the "version
mismatch" error, and I can't figure out why.

Any ideas on things I could try to get it to work?


I want to bundle python with my source code because I need to control
the version. I'm compiling my c++ extensions with boost. But there's
something wrong because at the end, I'm rewarded with:
Fatal Python error: Interpreter not initialized (version mismatch?)
../triad.sh: line 19: 83478 Abort trap
$trunk/src/python/bin/python $@

This process does work on Linux, so there must be something peculiar
about doing this on OSX 10.6.3.

1) I build python:
./configure --prefix=/Users/amos/test_fresh_compile/trunk/src/python

I can see that libpython2.6.a was compiled like this:

ar cr libpython2.6.a Modules/getbuildinfo.o^M
ar cr libpython2.6.a Parser/acceler.o Parser/grammar1.o
Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o
Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o
Parser/grammar.o Parser/pgen.o Parser/myreadline.o
Parser/tokenizer.o^M
ar cr libpython2.6.a Objects/abstract.o Objects/boolobject.o
Objects/bufferobject.o Objects/bytes_methods.o
Objects/bytearrayobject.o Objects/cellobject.o Objects/classobject.o
Objects/cobject.o Objects/codeobject.o Objects/complexobject.o
Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o
Objects/gen\
object.o Objects/fileobject.o Objects/floatobject.o
Objects/frameobject.o Objects/funcobject.o Objects/intobject.o
Objects/iterobject.o Objects/listobject.o Objects/longobject.o
Objects/dictobject.o Objects/methodobject.o Objects/moduleobject.o
Objects/object.o Objects/obmalloc.o Objects/rangeobject.o
Objects/seto\
bject.o Objects/sliceobject.o Objects/stringobject.o
Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o
Objects/weakrefobject.o Objects/unicodeobject.o
Objects/unicodectype.o^M
ar cr libpython2.6.a Python/_warnings.o Python/Python-ast.o
Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o
Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o
Python/frozenmain.o Python/future.o Python/getargs.o
Python/getcompiler.o Python/getcopyright.o Python/getmtime.o
Python/getpla\
tform.o Python/getversion.o Python/graminit.o Python/import.o
Python/importdl.o Python/marshal.o Python/modsupport.o
Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o
Python/pyarena.o Python/pyfpe.o Python/pymath.o Python/pystate.o
Python/pythonrun.o Python/structmember.o Python/symtable.o
Python/sysmodule.\
o Python/traceback.o Python/getopt.o Python/pystrcmp.o
Python/pystrtod.o Python/formatter_unicode.o Python/formatter_string.o
Python/dynload_shlib.o  Python/mactoolboxglue.o Python/thread.o^M
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols^M
ar cr libpython2.6.a Modules/config.o Modules/getpath.o Modules/main.o
Modules/gcmodule.o ^M
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols^M
ar cr libpython2.6.a Modules/threadmodule.o  Modules/signalmodule.o
Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o
Modules/_sre.o  Modules/_codecsmodule.o  Modules/zipimport.o
Modules/symtablemodule.o  Modules/xxsubtype.o^M
/usr/bin/ranlib: file: libpython2.6.a(pymath.o) has no symbols^M
ranlib libpython2.6.a^M
ranlib: file: libpython2.6.a(pymath.o) has no symbols^M


2) Build boost. In bjam I have:

using python
      : 2.6
      : python/bin/python2.6
      : python/include/python2.6
      : python/lib/python2.6/config
      :
      ;

which produces lines like:
darwin.compile.c++
boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/list.o

    "g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline
-Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC
-DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE  -I"boost/boost_1_42_0"
-I"python/include/python2.6" -c -o
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/list.o"
"boost/boost_1_42_0/libs/python/src/list.cpp"


so it appears to be linking to the version of python I just built.

3) Build libboost_python:

darwin.link.dll
boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/libboost_python.dylib

    "g++" -dynamiclib -Wl,-single_module -install_name
"libboost_python.dylib" -L"python/lib/python2.6/config" -o
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/libboost_python.dylib"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/numeric.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/list.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/long.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/dict.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/tuple.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/str.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/slice.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/converter/from_python.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/converter/registry.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/converter/type_id.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/enum.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/class.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/function.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/inheritance.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/life_support.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/pickle_support.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/errors.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/module.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/converter/builtin_converters.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/converter/arg_to_python_base.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/iterator.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/stl_iterator.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object_protocol.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object_operators.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/wrapper.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/import.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/exec.o"
"boost/boost_1_42_0/bin.v2/libs/python/build/darwin-4.2.1/release-triad/object/function_doc_signature.o"
 -lpython2.6    -headerpad_max_install_names -g -Wl,-dead_strip
-no_dead_strip_inits_and_terms

4) Investigate libraries:
> otool -L libboost_python.dylib
libboost_python.dylib:
	libboost_python.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 125.0.1)


now this is peculiar, because if I build with my /Library installed
framework version of python instead, I get:

> otool -L libboost_python.dylib
libboost_python.dylib:
	libboost_python.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Library/Frameworks/Python.framework/Versions/2.6/Python
(compatibility version 2.6.0, current version 2.6.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 125.0.1)

so this is my first clue that something went wrong... however, I can't
figure out any way to get the link stage for libboost_python.dylib to
include the python library I want it to. I can even change the name
-lpython2.6 to -lpythonamos, and it gives an error (so I know it's
looking where I think it is), and then I copy libpython2.6.dylib to
libpythonamos.dylib and i don't get an error. This suggests it's
finding the library but it doesn't like it.

the problem isn't 32 vs 64 bits:

> lipo -info libboost_python.dylib
Non-fat file: libboost_python.dylib is architecture: x86_64

> lipo -info libpython2.6.dylib
input file libpython2.6.dylib is not a fat file
Non-fat file: libpython2.6.dylib is architecture: x86_64



It's like the library requires some special OSX blessing that's
normally only given when a framework is built.