[Distutils] 2 platforms down, only Mac to go...

Phillip J. Eby pje at telecommunity.com
Sat Jan 14 02:37:08 CET 2006

I've got shared libraries working on Windows and Linux -- without any 
LD_LIBRARY_PATH nonsense, or any .so file rewriting.  I have not had a 
chance to try this on OS X, however.  If there's anybody out there who has 
experience wrestling with the Mac OS linker (hi, Bob! <wink>) has a chance 
to try it out and maybe tweak it a little bit, I'd appreciate it.

The code is in the subversion trunk now, and the source includes a 
tests/shlib_test subdirectory that you can change to and run "setup.py 
test" to build/link/test, e.g.:

     cd tests/shlib_test
     PYTHONPATH=../.. python setup.py test

This just verifies that the basic linking works, i.e., whether my guesses 
for the linker options in build_ext.py were correct.  The second test is to 
verify that it works from a different directory:

     cd ..
     PYTHONPATH=..:shlib_test python -c "import hello"

If this works without crashing, then the dynamic linking works even from a 
different directory.

Based on my limited guesswork, the final test is the one most likely to 
fail on OS X (assuming the link step succeeded, and I'm not sure it 
will!).  Rename the test directory temporarily:

     mv shlib_test something_else
     PYTHONPATH=..:something_else python -c "import hello"
     mv something_else shlib_test

If this works without a bit of hacking, I'll be rather surprised, but 
*very* pleased.  If all three tests work without any changes to the source, 
it's probably a bug.  ;)  (Specifically, it probably means setuptools fell 
back to static linking, and thus isn't having any dynamic link problems.)

The basic idea of what I'm doing here is that I try to set the runtime 
library search path of each extension (that depends on a shared library in 
the project), to include the current directory *at runtime*, then make a 
Python wrapper for the extension that temporarily changes the current 
directory while doing the import.

The tricky bit -- if I understand correctly -- is that OS X has no concept 
of a runtime library search path, so what this probably *should* be doing 
is specifying something like "-dylib_file 
libhellolib.dylib:./libhellolib.dylib" in the link options for the 'hello' 
extension.  What I'm not clear on is whether this is actually allowed by 
the linker, or if it even does what I think it does in the first 
place.  Even if it's allowed and does the right thing, I have a sneaky 
suspicion that perhaps the path will be converted to an absolute path at 
link time, rather than leaving the './' in place, given the general 
reputation for deviousness possessed by the Mac OS linker.  :)

On the other hand, perhaps it's possible to fudge the paths in the dylib 
once it's built, such that the './' is in the file?  Any thoughts or 
suggestions would be welcome.

In a couple of weeks I'll have a chance to play around with this on a Mac 
myself, but since I'm starting from scratch (zero Mac experience 
whatsoever), I'm hoping maybe someone more experienced could provide some 
degree of guidance by then.  Thanks.

More information about the Distutils-SIG mailing list