[Patches] [ python-Patches-1454844 ] Use dlopen() to load extensions on Darwin, where possible

[SourceForge.net]

Patches item #1454844, was opened at 2006-03-20 11:41
Message generated for change (Comment added) made by barnert
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1454844&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Zach Pincus (zpincus)
Assigned to: Nobody/Anonymous (nobody)
Summary: Use dlopen() to load extensions on Darwin, where possible

Initial Comment:
Here is a patch to make Python on OS X 10.3 and above use dlopen() 
(via dynload_shlib.c) to load extension modules, and to make the dl 
module avaliable. 

The patch was generated against the SVN head as of yesterday, but can 
be applied (and I have done so with success) to Python 2.4.2.

RATIONALE
---------
On most unix-like operating systems, Python uses the dlopen() call to 
load extension modules. In addition, the way that these modules are 
opened can be modified via 'sys.setdlopenflags(...)'.

Modifications of how extensions are loaded are useful for several 
reasons (enough so that the standard Python docs (e.g. http://
docs.python.org/lib/module-sys.html ). In particular, if multiple 
modules need to share symbols, a call to sys.setdlopenflags is 
necessary.

Symbol sharing is especially important for interoperability of modules 
which wrap C++ classes, because GCC creates classes that resolve 
their run-time type identification by symbol identity. Thus, symbols 
must be shared globally for many C++ features to work properly for 
objects passed between modules.

On OS X / Darwin, Python uses some NeXT-derived APIs to load 
modules. Though these APIs provide analogues to the dlopenflags 
used to control how dlopen() loads modules, this interface is *not* 
exposed to the Python interpreter. Worse, sys.setdlopenflags remains 
available on Darwin, though calls to it are never heeded.

Fortunately, on OS X 10.3 and above, Apple has included dlopen as a 
standard function. In 10.3, this call is provided by a compatibility API; 
in 10.4, the dlopen() call is written to interface directly with the library 
loading mechanism and is now the recommended method for libraries 
to be opened for non Carbon/Cocoa tools.

IMPLEMENTATION
--------------
This (trivial) patch instructs the Python build process to use 
dynload_shlib.c (which uses dlopen) instead of dynload_next.c (which 
uses the NeXT-derived APIs). It also allows for the dl module to be built 
in order to provide access to the proper values for the various dlopen 
flags.

TESTING
-------
This patch can be configured and built into executables that build and 
test correctly on 10.3 and 10.4.

Because Python 2.5 and 2.4 do not currently compile properly on OS X 
10.2, I have not built or tested this patch on that OS version. However, 
the configure and compile process does select the appropriate 
dynload_next.c file to use, and compiles that correctly before breaking 
elsewhere. Thus, if the other errors are fixed for 10.2, these patches 
will work fine. (This is because they only change Python's behavior for 
10.3 and up.)

PATCHES
-------
There are three main components to the attached patch.

The first is a patch the 'configure.in' file to use dynload_shlib.c when it 
can, and a patch to the 'configure' file to sync it up with 'configure.in'. 

The second is a minor change to 'setup.py' and the dl module test to 
allow the dl module to be built and tested on OS X systems where dlfcn 
is available. (10.3 and above.)

The last part of the patchfile should be considered optional. This patch 
applies to 'Lib/test/regrtest.py', and it tells the testing suite that the dl 
test is not expected to be skipped anymore.

This is optional because if Python is ever built on 10.2, the test script 
will expect dl to work, when it only works on 10.3 and above. However, 
if Python on 10.2 is officially not supported, then this change should 
be made to properly test the dl functionality on all supported OS X 
platforms.



----------------------------------------------------------------------

Comment By: Andrew Barnert (barnert)
Date: 2006-03-21 18:31

Message:
Logged In: YES 
user_id=1473180

One minor issue: Your pattern will match Darwin 10.0, which will come out in 
a few years as part of OS X 10.7, by which time nobody will remember why 
this script is written this way. Since it's easy to handle now, we might as well 
do so. On a side note, there are no 2.x-4.x versions (1.5 was renamed 5.1), 
so anyone who matches Darwin/2.* will be a false match.

Here's what I'd suggest:

+       Darwin/[0156]\..*) DYNLOADFILE="dynload_next.o";;

Meanwhile, your patch seems to work for me on an Intel iMac, but I haven't 
tried anything too sophisticated.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1454844&group_id=5470