Building Cygwin-Python extension modules - export strategies
I have posted the following on comp.lang.python. At Jason Tishler's suggestion I am now also sending it to this SIG. In keeping with the dignity of this forum I have cut some of the excessively flippant bits....
On Friday I posted a message to comp.lang.idl-pvwave entitled "Building Numeric package on Cygwin-Python". I reported that the Numpy module library could not be built with Cygwin Python 2.1 using the distutils-based setup script. With the help of people who responded to this message, and to another one on the Cygwin mailing list, I can now build Numpy. But to get it to work out-of-the-box requires changes either to the C code or to distutils (see below). I would like to trigger a discussion on this list and ultimately a ruling from the Python gods about the one-true-Pythonic-way to write and install Python extension modules for the Cygwin version of Python.
To recap, I have the Numpy source distribution, Numeric-20.0.0b2.tar.gz (but I think these considerations apply to any Python module library that uses C code) and I have Python 2.1 installed by Cygwin's setup program (but the results should be the same if I had built it myself using the Cygwin gcc compiler). I unpack the source and cd to the directory created. According to the "Installing Python Modules" manual, I should be able to build & install this by typing
python setup.py build python setup.py install
However the build step fails at the first link command, with messages about undefined symbols. The problem is (and I'm a bit hazy here) that to build a DLL on the Windows platform, the compiler needs a list of symbols to be exported. The above command invokes a Unix-like build process that doesn't know about this.
There are two solutions:
1. (Thanks to Robert Yodlowski): In each C file foomodule.c, replace
void initfoo ()
DL_EXPORT(void) initfoo ()
where DL_EXPORT is a preprocessor macro.
2. (Thanks to Norman Vine): Invoke the setup script's "cygwin" compiler option:
python setup.py build compiler=cygwin python setup.py install
after first applying a patch (after sig) that modifies a couple of distutils files. The compiler then creates and uses the necessary .def files to specify the exports. (Note that the Numeric source does include several .def files but these are ignored.)
This is all fine, and it's not that much of a hardship to make one or other of these changes. But obviously it would be good if Cygwin Python users could build extension modules out-of-the-box, so I would like to see one of these changes (or something like them) incorporated into Python and its documentation. So which one? Jason Tishler and Norman Vine have told me that these things have been discussed before on python-dev and distutils-sig but never carried through to a resolution
Option 1 does have the blessing of Tim Peters
The downside is that it requires changes (albeit minor) to the C code of the Numeric package and, I presume, most other Python extension packages. Not to mention modifications to the "Extending and Embedding" manual.
Option 2 was raised by Jason Tishler on distutils-sig
but there wasn't any enthusiasm for it so he dropped it (to paraphrase Jason's words). If this is to be applied I would prefer that it not be necessary to specify the "cygwin" compiler option explicitly. (According to the "Installing Python Modules" manual this should only be necessary when using Cygwin to build extensions for native win32 Python.) Yes, I know I can achieve this myself via the distutils configuration file.
On the newsgroup Jason has pointed out that Option 1 works with the traditional Misc/Makefile.pre.in, whereas Option 2 does not.
--- Mark Hadfield email@example.com http://katipo.niwa.cri.nz/~hadfield National Institute for Water and Atmospheric Research
? cygwinccompiler.py.patch ? diffs ? my.diffs cvs server: Diffing . Index: cygwinccompiler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/cygwinccompiler.py,v retrieving revision 1.12 diff -r1.12 cygwinccompiler.py 99,103c99,103 < self.set_executables(compiler='gcc -mcygwin -O -Wall', < compiler_so='gcc -mcygwin -mdll -O -Wall', < linker_exe='gcc -mcygwin', < linker_so=('%s -mcygwin -mdll -static' % < self.linker_dll)) ---
# self.set_executables(compiler='gcc -mcygwin -O -Wall', # compiler_so='gcc -mcygwin -mdll -O -Wall', # linker_exe='gcc -mcygwin', # linker_so=('%s -mcygwin -mdll -static' % # self.linker_dll))
Index: sysconfig.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v retrieving revision 1.34 diff -r1.34 sysconfig.py 120c120,121 < if compiler.compiler_type == "unix": ---
# if compiler.compiler_type == "unix": if compiler.compiler_type in ["unix", "cygwin"]:
cvs server: Diffing command