[Pythonmac-SIG] building extensions that link against dynamic libraries - Fixed!

[Bob Tolbert]

Thanks for the quick response. All your ideas gave me enough energy to  
re-double my efforts to find and fix this problem. I did not know about  
all the DYLD_* environment variables and I recommend anyone else  
working on extensions/dylibs on OS X become familiar with these as well  
as otool.

After turning on DYLD_PRINT_LIBRARIES, I could see that all the correct  
libraries were in fact loading yet the import would still hang. I  
finally decided to go look for another project wraps C++ into a Python  
extension.  Of course, wxPython is just such a beast so I went digging  
through the source to find appropriate switches to compiling and  
linking my original dylibs, as well as building the extension.  
Addtionally, I re-read the discussion of about a month ago giving  
multiple choices for building extensions in general.

My final changes that get everything to work:

Changes while build base C++ libraries.

1. Added -dynamic to CXXFLAGS. I already had -fPIC, but -dynamic seems  
to be used as well. (don't know if this is absolutely required)

2. Linked all dylibs with "-dynamiclib -single_module -undefined  
dynamic_lookup" as recommended in the archives, plus the -single_module  
switch came from the wxMAC makefiles.

Then to build the extension, I used:

"-bundle -undefined dynamic_lookup"

and made sure not to use "-framework Python"

 From this I get a bundle that can successfully be imported into the  
system Framework Python as well as Fink's python. This gives maximum  
flexibility to my customers to use either one which was definitely a  
goal. Clearly, I didn't use distutils since it would use "-framework  
Python" by default.

Thanks to Jack for spurring me on with these other ideas and to Robin  
Dunn et. al. for going down this path before me.

Bob

Robert W. Tolbert  Ph.D.			 p. (505) 473-7385  Ext. 83
OpenEye Scientific Software	         f. (505) 473-0833
3600 Cerrillos Road, Suite 1107
Santa Fe, NM 87507

On Apr 26, 2004, at 4:58 AM, Jack Jansen wrote:

>
> On 26-apr-04, at 6:47, Robert Tolbert wrote:
>
>> I hate to dredge up an old topic, but I've spent more time chasing  
>> this weird problem and am still befuddled as to why I can't get this  
>> to work.
>>
>> The original question was posted here:
>>
>> http://mail.python.org/pipermail/pythonmac-sig/2004-January/ 
>> 009911.html
>>
>> where I asked some questions about importing an extension that is  
>> linked against a shared dynamic library that I've also built. In a  
>> nutshell, I am wrapping a large C++ API in Python using SWIG. I can  
>> built the base C++ libraries as either static libs or dynamic libs. I  
>> can built the extension linked against either of these. And here is  
>> where the difference lies.
>>
>> If I link against the static libraries, my extension loads and works  
>> just fine.
>>
>> If I link against the dynamic libraries, Python hangs trying to  
>> import the extension. Ctrl-C won't interupt it, I have to Ctrl-Z and  
>> kill -9 to get it to stop.
>
> First I want to reiterate the last suggestion I made back in January:  
> use some of the DYLD debugging environment variables to see if you can  
> figure out what is going on. Especially now that you have it working  
> with fink-python that could turn out to be very helpful (see where  
> fink-python mac apple-python differ). DYLD_PRINT_LIBRARIES comes to  
> mind, and you may also want to play with DYLD_BIND_AT_LAUNCH and  
> DYLD_ABORT_MULTIPLE_INITS
>
> Another suggestion is to (temporarily) make sure you don't need  
> DYLD_LIBRARY_PATH by putting your shared libraries in /usr/local/lib.  
> This'll let you work out whether the problem is somehow related to how  
> you set DYLD_LIBRARY_PATH or how it is picked up or something.
> --
> Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
> If I can't dance I don't want to be part of your revolution -- Emma  
> Goldman