Well, I was inspired by Thomas' patch for SWIG support to come up with my own patch. Here it is: completely untested, so I haven't checked it in yet. (I don't have SWIG installed at home, so I figured I'd let someone Out There who actually uses it test this for me. Remember: laziness is a virtue!)
Note that Thomas' patch was Windows/C++-centric; mine tries to be OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i file whether it is destined to be C or C++? I can't think of a good way to do it, but then my knowledge if SWIG is roughly epsilon. (Out of curiosity, I took the manual home and read it about three years ago. It struck me as being *way* to easy, and I wanted to write Perl extensions like a *real* man. Ahh, the folly of youth...)
Here's da patch:
=================================================================== RCS file: /cvsroot/python/distutils/distutils/command/build_ext.py,v retrieving revision 1.43 diff -u -r1.43 build_ext.py --- build_ext.py 2000/06/17 23:04:31 1.43 +++ build_ext.py 2000/06/22 01:30:49 @@ -367,12 +367,13 @@ else: self.announce ("building '%s' extension" % ext.name)
- # First step: compile the source code to object files. This - # drops the object files in the current directory, regardless - # of where the source is (may be a bad thing, but that's how a - # Makefile.pre.in-based system does it, so at least there's a - # precedent!) + # First, scan the sources for SWIG definition files (.i), run + # SWIG on 'em to create .c files, and modify the sources list + # accordingly. + sources = self.swig_sources(sources)
+ # Next, compile the source code to object files. + # XXX not honouring 'define_macros' or 'undef_macros' -- the # CCompiler API needs to change to accomodate this, and I # want to do one thing at a time! @@ -428,6 +429,74 @@
# build_extensions ()
+ + def swig_sources (self, sources): + + """Walk the list of source files in 'sources', looking for SWIG + interface (.i) files. Run SWIG on all that are found, and + return a modified 'sources' list with SWIG source files replaced + by the generated C (or C++) files. + """ + + new_sources = [] + swig_sources = [] + swig_targets = {} + + # XXX this drops generated C files into the source tree, which + # is fine for developers who want to distribute the generated + # source -- but there should be an option to put SWIG output in + # the temp dir. + + for source in sources: + (base, ext) = os.path.splitext(source) + if ext in self.swig_ext(): + new_sources.append(base + ".c") # umm, what if it's C++? + swig_files.append(source) + swig_targets[source] = new_sources[-1] + else: + new_sources.append(source) + + if not swig_files: + return new_sources + + swig = self.find_swig() + swig_cmd = [swig, "-python", "-dnone", "-ISWIG"] # again, C++?!? + + for source in swig_sources: + self.announce ("swigging %s to %s" % (src, obj)) + self.spawn(swig_cmd + ["-o", swig_targets[source], source]) + + return new_sources + + # swig_sources () + + def find_swig (self): + """Return the name of the SWIG executable. On Unix, this is + just "swig" -- it should be in the PATH. Tries a bit harder on + Windows. + """ + + if os.name == "posix": + return "swig" + elif os.name == "nt": + + # Look for SWIG in its standard installation directory on + # Windows (or so I presume!). If we find it there, great; + # if not, act like Unix and assume it's in the PATH. + for vers in ("1.3", "1.2", "1.1"): + fn = os.path.join("c:\swig%s" % vers, "swig.exe") + if os.path.isfile (fn): + return fn + else: + return "swig.exe" + + else: + raise DistutilsPlatformError, \ + ("I don't know how to find (much less run) SWIG " + "on platform '%s'") % os.name + + # find_swig () +
# -- Hooks ---------------------------------------------------------
========================================================================
Greg
On Wed, Jun 21, 2000 at 09:39:08PM -0400, Greg Ward wrote:
Note that Thomas' patch was Windows/C++-centric; mine tries to be OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i file whether it is destined to be C or C++? I can't think of a good way to do it, but then my knowledge if SWIG is roughly epsilon. (Out of curiosity, I took the manual home and read it about three years ago. It struck me as being *way* to easy, and I wanted to write Perl extensions like a *real* man. Ahh, the folly of youth...)
The only way I can think of is to analyze the file in some way, SWIG does handle C and C++ differently, but this is specified on the command line not in the interface file. I don't know C++ so I do not know what to look for in a file that would definitively identify it as C++. Can `file` recognize the difference? If it can then maybe the 'magic' file could be examined to duplicate whatever `file` does (since `file` is not available on all systems I guess Distutils would have to duplicate it.)
Well, I was inspired by Thomas' patch for SWIG support to come up with my own patch. Here it is: completely untested, so I haven't checked it in yet. (I don't have SWIG installed at home, so I figured I'd let someone Out There who actually uses it test this for me. Remember: laziness is a virtue!)
Note that Thomas' patch was Windows/C++-centric; mine tries to be OS-agnostic, but is C-centric. Is it possible to tell from a SWIG .i file whether it is destined to be C or C++? I can't think of a good way to do it, but then my knowledge if SWIG is roughly epsilon. (Out of curiosity, I took the manual home and read it about three years ago. It struck me as being *way* to easy, and I wanted to write Perl extensions like a *real* man. Ahh, the folly of youth...)
I don't like this patch.
While I won't argue whether ccompiler is an appropriate superclass for swig, swig shares some important properties:
Swig has command line flags to specify include directories -I<dir>, libraries -l<ifile>, define symbols -Dsymbol like C-compilers. Other options which are maybe usefull for python extensions are -t <typemap file>, -shadow (create shadow classes), -module name (set module name) and maybe more.
On 23 June 2000, Thomas Heller said:
I don't like this patch.
Well, at least you're up-front and honest. ;-)
While I won't argue whether ccompiler is an appropriate superclass for swig, swig shares some important properties:
Swig has command line flags to specify include directories -I<dir>, libraries -l<ifile>, define symbols -Dsymbol like C-compilers. Other options which are maybe usefull for python extensions are -t <typemap file>, -shadow (create shadow classes), -module name (set module name) and maybe more.
All true. *BUT* SWIG doesn't vary across platforms, there aren't multiple SWIG implementations to worry about, and therefore there's no need for an "abstract SWIG interface".
From this I have the impression that swig should be a separate class implemented in swig.py, and there should be distutils command line flags for setting the above options (and also the -c++ option). Although I did not implement them so far.
I'll buy that. I make no claim that my SWIG patch has anywhere near enough support for running SWIG in the real world; I suspect that adding such support will make it grow up into a real class. But that class really has no need to implement the CCompiler interface. Nor should it, since SWIG is not a C compiler: it's a SWIG interface file compiler.
(Sidenote: I also needed a swig-object in build_ext for my setup-script for win32all. I had to extend distutils in the setup-script because there are also exe-files to be built from C++ and swig sources).
I thought I mentioned this recently: there *is* a 'link_executable()' method in MSVCCompiler. If it works, why not use it? If it doesn't work, could you submit a patch instead of implementing your own? (And if the interface specified by CCompiler is insufficient, I'd like to find out *now*!)
As I pointed out in a separate post, there should also be an option to specify C or C++ processing.
Agreed -- yet another good argument for a SWIG class.
If I find time, I will submit a new patch.
Cool. My time is probably better spent learning SWIG. No wait, I mean documenting and testing the Distutils... sigh...
Greg
I wrote:
(Sidenote: I also needed a swig-object in build_ext for my setup-script for win32all. I had to extend distutils in the setup-script because there are also exe-files to be built from C++ and swig sources).
but this is of course nonsense. Swig is used to build python-extensions, not exe-files. Sorry for that.
Greg:
I thought I mentioned this recently: there *is* a 'link_executable()' method in MSVCCompiler. If it works, why not use it? If it doesn't work, could you submit a patch instead of implementing your own? (And if the interface specified by CCompiler is insufficient, I'd like to find out *now*!)
Yes, I used compile() and link_shared_object() in my win32 setup script. No need to worry about it.
As I pointed out in a separate post, there should also be an option to specify C or C++ processing.
Agreed -- yet another good argument for a SWIG class.
If I find time, I will submit a new patch.
Cool. My time is probably better spent learning SWIG. No wait, I mean documenting and testing the Distutils... sigh...
Well, actually I also don't use swig normally. I was forced to use it for Marks win32 stuff. He lets swig generate c++ code, so...
Appended is a patch for your swig patches (;-) which adds an --swig-cpp command line option to build_ext, so that I can build the win32 stuff again. There should probably better be an option for the Extension() class, but (hopefully) someone else can do this. This is in chunks 1, 2, 4, 5.
Chunks 3 and 6 contain a change for building extensions on NT: MSVC generates .lib and .exp files when building shared libraries. The .exp files are usually unneeded, the .lib files are import libraries, which may be needed for (other) dlls linking to the first one. Up to now these files were created in the build/lib.win32/Release or build/lib.win32/Debug directories. Sometimes python extensions are dependend on each other. One example is the ODBC stuff included in Marks code: The odbc extension links dynamically to the dbi extension. So I changed build_ext to create the .lib (and .exp) files in the build/lib.win32 directory. Since they have different names for debug and release builds, no conficts occur: dbi.lib, dbi_d.lib.
Thomas
On 26 June 2000, Thomas Heller said:
but this is of course nonsense. Swig is used to build python-extensions, not exe-files. Sorry for that.
Yeah, but I'd still like to know if 'link_executable()' works on Windows! I think it'd be really cool if part of building Distutils on Windows was (optionally!) to compile your wininst stub, all using the Distutils compiler interface...
(Yeah, I know, the Distutils shouldn't require a compiler or anything funky. Still...)
Appended is a patch for your swig patches (;-) which adds an --swig-cpp command line option to build_ext, so that I can build the win32 stuff again. There should probably better be an option for the Extension() class, but (hopefully) someone else can do this. This is in chunks 1, 2, 4, 5.
OK, the SWIG stuff is fine -- checked it in.
Chunks 3 and 6 contain a change for building extensions on NT: MSVC generates .lib and .exp files when building shared libraries. The .exp files are usually unneeded, the .lib files are import libraries, which may be needed for (other) dlls linking to the first one. Up to now these files were created in the build/lib.win32/Release or build/lib.win32/Debug directories. Sometimes python extensions are dependend on each other. One example is the ODBC stuff included in Marks code: The odbc extension links dynamically to the dbi extension. So I changed build_ext to create the .lib (and .exp) files in the build/lib.win32 directory. Since they have different names for debug and release builds, no conficts occur: dbi.lib, dbi_d.lib.
OK, that's in too, although it took a while to sink in through my thick skull -- so the checkin messages are a bit confused.
Hey, waitasec: you're referring to "build/lib.win32/Release" -- but the code in front me of says
self.implib_dir = self.build_temp
which to me eyes (scanning build_ext.py and build.py) resolves to "build/temp.win32", hence the Debug and Release directories are under build/temp.win32, not build/lib.win32.
What's going on here? I'm confused... can someone explain my code to me? ;-) (Well, let's face it, the Windows stuff in build_ext.py really isn't my code...)
Greg
On 26 June 2000, Thomas Heller said:
but this is of course nonsense. Swig is used to build python-extensions, not exe-files. Sorry for that.
Yeah, but I'd still like to know if 'link_executable()' works on Windows! I think it'd be really cool if part of building Distutils on Windows was (optionally!) to compile your wininst stub, all using the Distutils compiler interface...
Yes, it works. To illustrate: this is an excerpt of my setup.py for win32all (building wininst.exe inside Distutils is reserved for later):
class my_build_ext (build_ext): def run (self): build_ext.run (self) self.build_dlls() self.build_exes()
def build_dlls (self): for target in dll_files: sources = target.sources ext_filename = os.path.join (self.build_lib, target.name + '.dll')
if not (self.force or newer_group(sources, ext_filename, 'newer')): self.announce ("skipping '%s' (up-to-date)" % target.name) return else: self.announce ("building '%s'" % target.name)
objects = self.compiler.compile (target.sources, output_dir=self.build_temp,
include_dirs=target.include_dirs, debug=self.debug,
extra_postargs=target.extra_compile_args)
self.compiler.link_shared_object( objects, ext_filename, libraries=target.libraries, library_dirs=target.library_dirs, extra_postargs=target.extra_args, debug=self.debug)
def build_exes (self): for target in exe_files: sources = target.sources ext_filename = os.path.join (self.build_lib, target.name)
if not (self.force or newer_group(sources, ext_filename + '.exe', 'newer')): self.announce ("skipping '%s' (up-to-date)" % target.name) return else: self.announce ("building '%s'" % target.name)
extra_args = target.extra_compile_args objects = self.compiler.compile (sources, output_dir=self.build_temp,
include_dirs=target.include_dirs, debug=self.debug, extra_postargs=extra_args)
extra_args = target.extra_link_args
self.compiler.link_executable( objects, ext_filename, libraries=target.libraries, library_dirs=target.library_dirs, extra_postargs=extra_args, debug=self.debug)
[about the MSVC compiler patches] Hey, waitasec: you're referring to "build/lib.win32/Release" -- but the code in front me of says
self.implib_dir = self.build_tempwhich to me eyes (scanning build_ext.py and build.py) resolves to "build/temp.win32", hence the Debug and Release directories are under build/temp.win32, not build/lib.win32.
Sorry, a simple typo. You are correct: What I meant is build/temp.win32.
Thomas
participants (3)
-
Greg Ward -
Harry Henry Gebel -
Thomas Heller