SWIG runtime libs/Python distutils combo on Mac
This problem concerns using distutils to compile a set of python modules that are generated by SWIG, so I am cross-posting to both SWIG and distutils mailing lists. I use SWIG to wrap several packages that are intended to be both independent and interoperable. So I am creating separate python extensions that link to the swig runtime libraries. Previously, I linked to installed system runtime libraries, but this is not the preferred approach, and indeed, swig 1.3.22 does not support it. Instead, I am supposed to generate my own local runtime library, and link to it. Fine; this would seem to be more portable. The SWIG documentation says to do the following (for Linux), with the caveat that compiling dynamic libraries is different on every system: ------------------ % swig -runtime -python swigrun.i % gcc -fpic swigrun_wrap.c -o swigrun_wrap.o % gcc -shared swigrun_wrap.o -o libswigrunpy.so Now, you compile all of the normal SWIG modules using the -noruntime option: % swig -noruntime -c++ -python A.i % swig -noruntime -c++ -python B.i Finally, when linking the dynamically loadable modules, you need to link again the special runtime library above. For example (Linux) : % g++ -shared A_wrap.o -L. -lswigrunpy -o _A.so % g++ -shared B_wrap.o -L. -lswigrunpy -o _B.so ------------------ To get around the portability problem, I have decided to use python's distutils module to handle the compilation issues. In my setup.py script, I define SwigRun = Extension("PyTrilinos.libswigrun", ["src/swigrun_wrap.c"]) and this works, as build/lib.blahblahblah/PyTrilinos/libswigrun.so gets created properly. When I define real extensions in setup.py, they are created with keyword arguments library_dirs = [..., "build/lib.blahblahblah/PyTrilinos", ...], libraries = [..., "swigrun", ...], which gives me the link command arguments I want: -Lbuild/lib.blahblahblah/PyTrilinons -lswigrun. However, on my Mac, when the link command is invoked, I get ld: can't locate file for: -lswigrun I strongly suspect that the Mac's ld command does not search for libswigrun.so when -lswigrun is given as a command line argument (the old system libraries have .la and .dylib extensions). But I cannot figure out how to get setup.py to compile to a different suffix. Any suggestions would be appreciated. ** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-5451 ** ** Albuquerque, NM 87185-0316 Email: wfspotz@sandia.gov **
On Sep 27, 2004, at 12:03 PM, Bill Spotz wrote:
To get around the portability problem, I have decided to use python's distutils module to handle the compilation issues. In my setup.py script, I define
SwigRun = Extension("PyTrilinos.libswigrun", ["src/swigrun_wrap.c"])
and this works, as build/lib.blahblahblah/PyTrilinos/libswigrun.so gets created properly. When I define real extensions in setup.py, they are created with keyword arguments
library_dirs = [..., "build/lib.blahblahblah/PyTrilinos", ...], libraries = [..., "swigrun", ...],
which gives me the link command arguments I want: -Lbuild/lib.blahblahblah/PyTrilinons -lswigrun. However, on my Mac, when the link command is invoked, I get
ld: can't locate file for: -lswigrun
I strongly suspect that the Mac's ld command does not search for libswigrun.so when -lswigrun is given as a command line argument (the old system libraries have .la and .dylib extensions). But I cannot figure out how to get setup.py to compile to a different suffix.
You can't do this on OS X. You can link to two types of files in OS X, static libraries and dylibs. Python extension modules are bundles (runtime loadable code) and can not be linked to at compile time. You should either compile swigrun_wrap into every extension, or make it a static library (which is more or less the same thing). -bob
I'm pretty sure I cannot compile swigrun_wrap into every extension or make it a static library, since each extension will get its own copy and that will defeat the purpose of the runtime libraries. Sounds like I need to create a dylib, that python distutils can't help me, and that I'll have to learn libtool if I want to make it portable. Does that sound about right? On Sep 27, 2004, at 10:19 AM, Bob Ippolito wrote:
On Sep 27, 2004, at 12:03 PM, Bill Spotz wrote:
To get around the portability problem, I have decided to use python's distutils module to handle the compilation issues. In my setup.py script, I define
SwigRun = Extension("PyTrilinos.libswigrun", ["src/swigrun_wrap.c"])
and this works, as build/lib.blahblahblah/PyTrilinos/libswigrun.so gets created properly. When I define real extensions in setup.py, they are created with keyword arguments
library_dirs = [..., "build/lib.blahblahblah/PyTrilinos", ...], libraries = [..., "swigrun", ...],
which gives me the link command arguments I want: -Lbuild/lib.blahblahblah/PyTrilinons -lswigrun. However, on my Mac, when the link command is invoked, I get
ld: can't locate file for: -lswigrun
I strongly suspect that the Mac's ld command does not search for libswigrun.so when -lswigrun is given as a command line argument (the old system libraries have .la and .dylib extensions). But I cannot figure out how to get setup.py to compile to a different suffix.
You can't do this on OS X. You can link to two types of files in OS X, static libraries and dylibs. Python extension modules are bundles (runtime loadable code) and can not be linked to at compile time. You should either compile swigrun_wrap into every extension, or make it a static library (which is more or less the same thing).
-bob
** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-5451 ** ** Albuquerque, NM 87185-0316 Email: wfspotz@sandia.gov **
On Sep 29, 2004, at 4:43 PM, Bill Spotz wrote:
On Sep 27, 2004, at 10:19 AM, Bob Ippolito wrote:
On Sep 27, 2004, at 12:03 PM, Bill Spotz wrote:
To get around the portability problem, I have decided to use python's distutils module to handle the compilation issues. In my setup.py script, I define
SwigRun = Extension("PyTrilinos.libswigrun", ["src/swigrun_wrap.c"])
and this works, as build/lib.blahblahblah/PyTrilinos/libswigrun.so gets created properly. When I define real extensions in setup.py, they are created with keyword arguments
library_dirs = [..., "build/lib.blahblahblah/PyTrilinos", ...], libraries = [..., "swigrun", ...],
which gives me the link command arguments I want: -Lbuild/lib.blahblahblah/PyTrilinons -lswigrun. However, on my Mac, when the link command is invoked, I get
ld: can't locate file for: -lswigrun
I strongly suspect that the Mac's ld command does not search for libswigrun.so when -lswigrun is given as a command line argument (the old system libraries have .la and .dylib extensions). But I cannot figure out how to get setup.py to compile to a different suffix.
You can't do this on OS X. You can link to two types of files in OS X, static libraries and dylibs. Python extension modules are bundles (runtime loadable code) and can not be linked to at compile time. You should either compile swigrun_wrap into every extension, or make it a static library (which is more or less the same thing). I'm pretty sure I cannot compile swigrun_wrap into every extension or make it a static library, since each extension will get its own copy and that will defeat the purpose of the runtime libraries. Sounds like I need to create a dylib, that python distutils can't help me, and that I'll have to learn libtool if I want to make it portable. Does that sound about right?
I don't think your assessment of swigrun_wrap is correct. -bob
participants (2)
-
Bill Spotz -
Bob Ippolito