[Distutils] Building Cygwin-Python extension modules - export strategies

[Mark Hadfield]

I have posted the following on comp.lang.python. At Jason Tishler's
suggestion I am now also sending it to this SIG. In keeping with the dignity
of this forum I have cut some of the excessively flippant bits....

On Friday I posted a message to comp.lang.idl-pvwave entitled
"Building Numeric package on Cygwin-Python". I reported that the Numpy
module library could not be built with Cygwin Python 2.1 using the
distutils-based setup script. With the help of people who responded to
this message, and to another one on the Cygwin mailing list, I can now
build Numpy. But to get it to work out-of-the-box requires changes
either to the C code or to distutils (see below). I would like to
trigger a discussion on this list and ultimately a ruling from the
Python gods about the one-true-Pythonic-way to write and install
Python extension modules for the Cygwin version of Python.

To recap, I have the Numpy source distribution, Numeric-20.0.0b2.tar.gz (but
I think these considerations apply to any Python module library that uses C
code) and I have Python 2.1 installed by Cygwin's setup program (but the
results should be the same if I had built it myself using the Cygwin gcc
compiler). I unpack the source and cd to the directory created. According to
the "Installing Python Modules" manual, I should be able to build & install
this by typing

    python setup.py build
    python setup.py install

However the build step fails at the first link command, with messages about
undefined symbols. The problem is (and I'm a bit hazy here) that to build a
DLL on the Windows platform, the compiler needs a list of symbols to be
exported. The above command invokes a Unix-like build process that doesn't
know about this.

There are two solutions:

1. (Thanks to Robert Yodlowski): In each C file foomodule.c, replace

    void initfoo ()

with

    DL_EXPORT(void) initfoo ()

where DL_EXPORT is a preprocessor macro.

2. (Thanks to Norman Vine): Invoke the setup script's "cygwin" compiler
option:

    python setup.py build compiler=cygwin
    python setup.py install

after first applying a patch (after sig) that modifies a couple of distutils
files. The compiler then creates and uses the necessary .def files to
specify the exports. (Note that the Numeric source does include several .def
files but these are ignored.)

This is all fine, and it's not that much of a hardship to make one or other
of these changes. But obviously it would be good if Cygwin Python users
could build extension modules out-of-the-box, so I would like to see one of
these changes (or something like them) incorporated into Python and its
documentation. So which one? Jason Tishler and Norman Vine have told me that
these things have been discussed before on python-dev and distutils-sig but
never carried through to a resolution

Option 1 does have the blessing of Tim Peters

    http://mail.python.org/pipermail/python-dev/2001-February/013292.html

The downside is that it requires changes (albeit minor) to the C code of the
Numeric package and, I presume, most other Python extension packages. Not to
mention modifications to the "Extending and Embedding" manual.

Option 2 was raised by Jason Tishler on distutils-sig

    http://mail.python.org/pipermail/distutils-sig/2001-February/002003.html

but there wasn't any enthusiasm for it so he dropped it (to paraphrase
Jason's words). If this is to be applied I would prefer that it not be
necessary to specify the "cygwin" compiler option explicitly. (According to
the "Installing Python Modules" manual this should only be necessary when
using Cygwin to build extensions for native win32 Python.) Yes, I know I can
achieve this myself via the distutils configuration file.

On the newsgroup Jason has pointed out that Option 1 works with the
traditional Misc/Makefile.pre.in, whereas Option 2 does not.

Opinions?

---
Mark Hadfield
m.hadfield@niwa.cri.nz  http://katipo.niwa.cri.nz/~hadfield
National Institute for Water and Atmospheric Research

? cygwinccompiler.py.patch
? diffs
? my.diffs
cvs server: Diffing .
Index: cygwinccompiler.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/distutils/cygwinccompiler.py,v
retrieving revision 1.12
diff -r1.12 cygwinccompiler.py
99,103c99,103
<         self.set_executables(compiler='gcc -mcygwin -O -Wall',
<                              compiler_so='gcc -mcygwin -mdll -O -Wall',
<                              linker_exe='gcc -mcygwin',
<                              linker_so=('%s -mcygwin -mdll -static' %
<                                         self.linker_dll))
---
> #        self.set_executables(compiler='gcc -mcygwin -O -Wall',
> #                             compiler_so='gcc -mcygwin -mdll -O -Wall',
> #                             linker_exe='gcc -mcygwin',
> #                             linker_so=('%s -mcygwin -mdll -static' %
> #                                        self.linker_dll))
Index: sysconfig.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v
retrieving revision 1.34
diff -r1.34 sysconfig.py
120c120,121
<     if compiler.compiler_type == "unix":
---
> #    if compiler.compiler_type == "unix":
>     if compiler.compiler_type in ["unix", "cygwin"]:
cvs server: Diffing command