CCompiler interfaces
Last weekend I tried to use Distutils' compiler classes to build an executable on Win32. Beside some problems with the compilers, I found that parameters of some methods of CCompiler are a little bit confusing. Output filenames: * create_static_lib needs a library name, which becomes intern a real file name. (Is anyone using this method?) * link_shared_library is similar to create_static_lib. * link_shared_object needs a real file name. * link_executable wants the basename of its output file. We have two methods shared_object_filename() and library_filename() (for static and shared libraries). Wouldn't it be better to add a method executable_filename() and use for all of the linker methods the real filename as parameter? For link_executable() we also need a parameter export_symbols. (An executable could export symbols to be able to be extended by plug-ins.) If we would make both changes I mentioned above, the interfaces of the three link_* methods would be the same. Also the implementation is almost the same. ( link_shared_library() calls link_shared_object(), and link_executable() differs only in one line in UnixCCompiler. The other compilers are similar, they only change some options to compile an executable instead a shared library.) The best way would to merge these three methods into one new method and remove the old ones. (Or let the old ones call the new and give a warning the user should use the new method.) This new method would have following parameters: (It is better to see a shared libray as an half-build executable than to compare it with static libraries.) def link (self, target, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None): target would be one of 'shared_object', 'shared_library' or 'executable'. The compiler classes would then be much simpler and smaller. For example UnixCCompiler. * Remove link_shared_library() and link_executable() * Rename link_shared_object() to link() and add target to the parameter list. * Change the spawn() call in it to the following piece of code. try: if target == 'executable': self.spawn (self.linker_exe + ld_args) else: self.spawn (self.linker_so + ld_args) except DistutilsExecError, msg: raise LinkError, msg This saves about 70 lines of redundant code. For BCPPCompiler and MSVCCompiler it is similar. Are there any objections? If not, I would send Greg a patch for it. ########################################################### And now a short piece of Changes.txt:
Release 0.9 (29 June, 2000): ---------------------------- ...
* added the ability to compile and link in resource files with Visual C++ on Windows (Thomas Heller)
However, I can't find the place where it is done. Maybe it wasn't checked in ? I can do the same using BCPPCompiler and CygwinCCompiler, and will send a patch after reorganizing the compiler classes. Kind regards Rene Liebscher
Rene Liebscher wrote:
Last weekend I tried to use Distutils' compiler classes to build an executable on Win32. Beside some problems with the compilers, I found that parameters of some methods of CCompiler are a little bit confusing.
Output filenames:
* create_static_lib needs a library name, which becomes intern a real file name. (Is anyone using this method?)
* link_shared_library is similar to create_static_lib.
* link_shared_object needs a real file name.
* link_executable wants the basename of its output file.
We have two methods shared_object_filename() and library_filename() (for static and shared libraries). Wouldn't it be better to add a method executable_filename() and use for all of the linker methods the real filename as parameter?
For link_executable() we also need a parameter export_symbols. (An executable could export symbols to be able to be extended by plug-ins.)
If we would make both changes I mentioned above, the interfaces of the three link_* methods would be the same. Also the implementation is almost the same. ( link_shared_library() calls link_shared_object(), and link_executable() differs only in one line in UnixCCompiler. The other compilers are similar, they only change some options to compile an executable instead a shared library.)
The best way would to merge these three methods into one new method and remove the old ones. (Or let the old ones call the new and give a warning the user should use the new method.)
This new method would have following parameters: (It is better to see a shared libray as an half-build executable than to compare it with static libraries.)
def link (self, target, objects, output_libname, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None):
target would be one of 'shared_object', 'shared_library' or 'executable'.
The compiler classes would then be much simpler and smaller.
For example UnixCCompiler. * Remove link_shared_library() and link_executable() * Rename link_shared_object() to link() and add target to the parameter list.
* Change the spawn() call in it to the following piece of code.
try: if target == 'executable': self.spawn (self.linker_exe + ld_args) else: self.spawn (self.linker_so + ld_args) except DistutilsExecError, msg: raise LinkError, msg
This saves about 70 lines of redundant code. For BCPPCompiler and MSVCCompiler it is similar.
Are there any objections? If not, I would send Greg a patch for it.
########################################################### And now a short piece of Changes.txt:
Release 0.9 (29 June, 2000): ---------------------------- ...
* added the ability to compile and link in resource files with Visual C++ on Windows (Thomas Heller)
However, I can't find the place where it is done. Maybe it wasn't checked in ? I can do the same using BCPPCompiler and CygwinCCompiler, and will send a patch after reorganizing the compiler classes.
Here is now the patch for both, reorganzing the compiler classes and adding the ability to use resource files with Borland C++ and Cygwin. (file: complier.link.patch) It is a huge patch (43k) because it has to delete a lot of old redundant code. unixccompiler.py 12926 -> 10407 Bytes msvccompiler.py 17251 -> 14321 # next both can now compile resources bcppcompiler.py 14992 -> 14237 cygwinccompiler.py 13294 -> 17178 (this is the new code to compile resources.) ccompiler.py 40680 -> 42175 (Without old methods link_shared_*() it would be 39508. These old methods are still there for compatibility with existing external code.) Kind regards Rene Liebscher diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/bcppcompiler.py distutils.patched/distutils/bcppcompiler.py --- distutils.orig/distutils/bcppcompiler.py Mon Sep 4 11:58:57 2000 +++ distutils.patched/distutils/bcppcompiler.py Wed Sep 13 15:09:52 2000 @@ -63,7 +63,7 @@ # indicate their installation locations. self.cc = "bcc32.exe" - self.link = "ilink32.exe" + self.linker = "ilink32.exe" self.lib = "tlib.exe" self.preprocess_options = None @@ -73,6 +73,8 @@ self.ldflags_shared = ['/Tpd', '/Gn', '/q', '/x'] self.ldflags_shared_debug = ['/Tpd', '/Gn', '/q', '/x'] self.ldflags_static = [] + self.ldflags_exe = ['/Gn', '/q', '/x'] + self.ldflags_exe_debug = ['/Gn', '/q', '/x','/r'] # -- Worker methods ------------------------------------------------ @@ -108,16 +110,32 @@ if skip_sources[src]: self.announce ("skipping %s (%s up-to-date)" % (src, obj)) else: + src = os.path.normpath(src) + obj = os.path.normpath(obj) + self.mkpath(os.path.dirname(obj)) + + if ext == '.res': + # this is already a binary file + continue # the 'for' loop + if ext == '.rc': + # this needs to be compiled to a .res file + try: + self.spawn (["brcc32"] + ["-fo"]+ [obj] + [src]) + except DistutilsExecError, msg: + raise CompileError, msg + continue # the 'for' loop + + # next both are for the real compiler if ext in self._c_extensions: input_opt = "" elif ext in self._cpp_extensions: input_opt = "-P" + else: + # set default. it is an unknown extension, + # probably the compiler will fail + input_opt = "" - src = os.path.normpath(src) - obj = os.path.normpath(obj) - output_opt = "-o" + obj - self.mkpath(os.path.dirname(obj)) # Compiler command line syntax is: "bcc32 [options] file(s)". # Note that the source file names must appear at the end of @@ -163,45 +181,20 @@ # create_static_lib () - - def link_shared_lib (self, - objects, - output_libname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): - - self.link_shared_object (objects, - self.shared_library_name(output_libname), - output_dir=output_dir, - libraries=libraries, - library_dirs=library_dirs, - runtime_library_dirs=runtime_library_dirs, - export_symbols=export_symbols, - debug=debug, - extra_preargs=extra_preargs, - extra_postargs=extra_postargs, - build_temp=build_temp) - - def link_shared_object (self, - objects, - output_filename, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): + def link (self, + target, + objects, + output_filename, + output_dir=None, + libraries=None, + library_dirs=None, + runtime_library_dirs=None, + export_symbols=None, + debug=0, + extra_preargs=None, + extra_postargs=None, + build_temp=None): # XXX this ignores 'build_temp'! should follow the lead of # msvccompiler.py @@ -213,45 +206,62 @@ if runtime_library_dirs: self.warn ("I don't know what to do with 'runtime_library_dirs': " + str (runtime_library_dirs)) - + + if output_dir is not None: output_filename = os.path.join (output_dir, output_filename) if self._need_link (objects, output_filename): - if debug: - ld_args = self.ldflags_shared_debug[:] + # target specific code + if target == CCompiler.EXECUTABLE: + startup_obj = 'c0w32' + if debug: + ld_args = self.ldflags_exe_debug[:] + else: + ld_args = self.ldflags_exe[:] else: - ld_args = self.ldflags_shared[:] + startup_obj = 'c0d32' + if debug: + ld_args = self.ldflags_shared_debug[:] + else: + ld_args = self.ldflags_shared[:] + # Create a temporary exports file for use by the linker - head, tail = os.path.split (output_filename) - modname, ext = os.path.splitext (tail) - temp_dir = os.path.dirname(objects[0]) # preserve tree structure - def_file = os.path.join (temp_dir, '%s.def' % modname) - contents = ['EXPORTS'] - for sym in (export_symbols or []): - contents.append(' %s=_%s' % (sym, sym)) - self.execute(write_file, (def_file, contents), - "writing %s" % def_file) + if export_symbols is None: + def_file = '' + else: + head, tail = os.path.split (output_filename) + modname, ext = os.path.splitext (tail) + temp_dir = os.path.dirname(objects[0]) # preserve tree structure + def_file = os.path.join (temp_dir, '%s.def' % modname) + contents = ['EXPORTS'] + for sym in (export_symbols or []): + contents.append(' %s=_%s' % (sym, sym)) + self.execute(write_file, (def_file, contents), + "writing %s" % def_file) # Borland C++ has problems with '/' in paths - objects = map(os.path.normpath, objects) - startup_obj = 'c0d32' - objects.insert(0, startup_obj) - - # either exchange python15.lib in the python libs directory against - # a Borland-like one, or create one with name bcpp_python15.lib - # there and remove the pragmas from config.h - libraries.append ('import32') - libraries.append ('cw32mt') - - # Start building command line flags and options. - + objects2 = map(os.path.normpath, objects) + # split objects in .obj and .res files + # Borland C++ needs them at different positions in the command line + objects = [startup_obj] + resources = [] + for file in objects2: + (base, ext) = os.path.splitext(os.path.normcase(file)) + if ext == '.res': + resources.append(file) + else: + objects.append(file) + + for l in library_dirs: ld_args.append("/L%s" % os.path.normpath(l)) - - ld_args.extend(objects) # list of object files + ld_args.append("/L.") # we use sometimes relative paths + + # list of object files + ld_args.extend(objects) # XXX the command-line syntax for Borland C++ is a bit wonky; # certain filenames are jammed together in one big string, but @@ -263,14 +273,14 @@ # because 'spawn()' would quote any filenames with spaces in # them. Arghghh!. Apparently it works fine as coded... - # name of dll file + # name of dll/exe file ld_args.extend([',',output_filename]) # no map file and start libraries ld_args.append(',,') for lib in libraries: # see if we find it and if there is a bcpp specific lib - # (bcpp_xxx.lib) + # (xxx_bcpp.lib) libfile = self.find_library_file(library_dirs, lib, debug) if libfile is None: ld_args.append(lib) @@ -279,8 +289,17 @@ else: # full name which prefers bcpp_xxx.lib over xxx.lib ld_args.append(libfile) + + # some default libraries + ld_args.append ('import32') + ld_args.append ('cw32mt') + # def file for export symbols ld_args.extend([',',def_file]) + # add resource files + ld_args.append(',') + ld_args.extend(resources) + if extra_preargs: ld_args[:0] = extra_preargs @@ -289,88 +308,24 @@ self.mkpath (os.path.dirname (output_filename)) try: - self.spawn ([self.link] + ld_args) + self.spawn ([self.linker] + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: self.announce ("skipping %s (up-to-date)" % output_filename) - # link_shared_object () - - - def link_executable (self, - objects, - output_progname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - debug=0, - extra_preargs=None, - extra_postargs=None): - - (objects, output_dir) = self._fix_object_args (objects, output_dir) - (libraries, library_dirs, runtime_library_dirs) = \ - self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) - - if runtime_library_dirs: - self.warn ("I don't know what to do with 'runtime_library_dirs': " - + str (runtime_library_dirs)) - - lib_opts = gen_lib_options (self, - library_dirs, runtime_library_dirs, - libraries) - output_filename = output_progname + self.exe_extension - if output_dir is not None: - output_filename = os.path.join (output_dir, output_filename) - - if self._need_link (objects, output_filename): - - if debug: - ldflags = self.ldflags_shared_debug[1:] - else: - ldflags = self.ldflags_shared[1:] - - ld_args = ldflags + lib_opts + \ - objects + ['/OUT:' + output_filename] - - if extra_preargs: - ld_args[:0] = extra_preargs - if extra_postargs: - ld_args.extend (extra_postargs) - - self.mkpath (os.path.dirname (output_filename)) - try: - self.spawn ([self.link] + ld_args) - except DistutilsExecError, msg: - raise LinkError, msg - else: - self.announce ("skipping %s (up-to-date)" % output_filename) - + # link () # -- Miscellaneous methods ----------------------------------------- - # These are all used by the 'gen_lib_options() function, in - # ccompiler.py. - - def library_dir_option (self, dir): - return "-L" + dir - - def runtime_library_dir_option (self, dir): - raise DistutilsPlatformError, \ - ("don't know how to set runtime library search path " - "for Borland C++") - - def library_option (self, lib): - return self.library_filename (lib) def find_library_file (self, dirs, lib, debug=0): # List of effective library names to try, in order of preference: - # bcpp_xxx.lib is better than xxx.lib + # xxx_bcpp.lib is better than xxx.lib # and xxx_d.lib is better than xxx.lib if debug is set # - # The "bcpp_" prefix is to handle a Python installation for people + # The "_bcpp" suffix is to handle a Python installation for people # with multiple compilers (primarily Distutils hackers, I suspect # ;-). The idea is they'd have one static library for each # compiler they care about, since (almost?) every Windows compiler @@ -390,3 +345,31 @@ # Oops, didn't find it in *any* of 'dirs' return None + # overwrite the one from CCompiler to support rc and res-files + def object_filenames (self, + source_filenames, + strip_dir=0, + output_dir=''): + if output_dir is None: output_dir = '' + obj_names = [] + for src_name in source_filenames: + # use normcase to make sure '.rc' is really '.rc' and not '.RC' + (base, ext) = os.path.splitext (os.path.normcase(src_name)) + if ext not in (self.src_extensions + ['.rc','.res']): + raise UnknownFileError, \ + "unknown file type '%s' (from '%s')" % \ + (ext, src_name) + if strip_dir: + base = os.path.basename (base) + if ext == '.res': + # these can go unchanged + obj_names.append (os.path.join (output_dir, base + ext)) + elif ext == '.rc': + # these need to be compiled to .res-files + obj_names.append (os.path.join (output_dir, base + '.res')) + else: + obj_names.append (os.path.join (output_dir, + base + self.obj_extension)) + return obj_names + + # object_filenames () \ No newline at end of file diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/ccompiler.py distutils.patched/distutils/ccompiler.py --- distutils.orig/distutils/ccompiler.py Tue Aug 8 13:18:37 2000 +++ distutils.patched/distutils/ccompiler.py Wed Sep 13 15:00:54 2000 @@ -561,9 +561,15 @@ pass - def link_shared_lib (self, + # values for target parameter in link() + SHARED_OBJECT = "shared_object" + SHARED_LIBRARY = "shared_library" + EXECUTABLE = "executable" + + def link (self, + target, objects, - output_libname, + output_filename, output_dir=None, libraries=None, library_dirs=None, @@ -573,12 +579,15 @@ extra_preargs=None, extra_postargs=None, build_temp=None): - """Link a bunch of stuff together to create a shared library file. - Similar semantics to 'create_static_lib()', with the addition of - other libraries to link against and directories to search for them. - Also, of course, the type and name of the generated file will - almost certainly be different, as will the program used to create - it. + """Link a bunch of stuff together to create an executable or + shared library file. + + The "bunch of stuff" consists of the list of object files supplied + as 'objects'. + 'output_filename' should be a filename. + If 'output_dir' is supplied, 'output_filename' is relative to it + (i.e. 'output_filename' can provide directory components if + needed). 'libraries' is a list of libraries to link against. These are library names, not filenames, since they're translated into @@ -610,7 +619,30 @@ Raises LinkError on failure. """ - pass + raise NotImplementedError + + + # old methods, rewritten to use the new link() method. + + def link_shared_lib (self, + objects, + output_libname, + output_dir=None, + libraries=None, + library_dirs=None, + runtime_library_dirs=None, + export_symbols=None, + debug=0, + extra_preargs=None, + extra_postargs=None, + build_temp=None): + self.warn("link_shared_lib(...) is depreciated, " + "use link(CCompiler.SHARED_LIBRARY,...) instead.") + self.link (CCompiler.SHARED_LIBRARY, objects, + self.library_filename (output_libname, lib_type='shared'), + output_dir, + libraries, library_dirs, runtime_library_dirs, export_symbols, + debug, extra_preargs, extra_postargs, build_temp) def link_shared_object (self, @@ -625,16 +657,11 @@ extra_preargs=None, extra_postargs=None, build_temp=None): - """Link a bunch of stuff together to create a shared object file. - Much like 'link_shared_lib()', except the output filename is - explicitly supplied as 'output_filename'. If 'output_dir' is - supplied, 'output_filename' is relative to it - (i.e. 'output_filename' can provide directory components if - needed). - - Raises LinkError on failure. - """ - pass + self.warn("link_shared_object(...) is depreciated, " + "use link(CCompiler.SHARED_OBJECT,...) instead.") + self.link (CCompiler.SHARED_OBJECT, objects, output_filename, output_dir, + libraries, library_dirs, runtime_library_dirs, export_symbols, + debug, extra_preargs, extra_postargs, build_temp) def link_executable (self, @@ -647,15 +674,12 @@ debug=0, extra_preargs=None, extra_postargs=None): - """Link a bunch of stuff together to create a binary executable - file. The "bunch of stuff" is as for 'link_shared_lib()'. - 'output_progname' should be the base name of the executable - program--e.g. on Unix the same as the output filename, but on - DOS/Windows ".exe" will be appended. - - Raises LinkError on failure. - """ - pass + self.warn("link_executable(...) is depreciated, " + "use link(CCompiler.EXECUTABLE,...) instead.") + self.link (CCompiler.EXECUTABLE, objects, + self.executable_filename(output_progname), output_dir, + libraries, library_dirs, runtime_library_dirs, None, + debug, extra_preargs, extra_postargs, None) @@ -756,6 +780,14 @@ basename = os.path.basename (basename) return os.path.join (output_dir, basename + self.shared_lib_extension) + def executable_filename (self, + basename, + strip_dir=0, + output_dir=''): + if output_dir is None: output_dir = '' + if strip_dir: + basename = os.path.basename (basename) + return os.path.join (output_dir, basename + (self.exe_extension or '')) def library_filename (self, libname, diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/build_ext.py distutils.patched/distutils/command/build_ext.py --- distutils.orig/distutils/command/build_ext.py Thu Sep 7 11:51:25 2000 +++ distutils.patched/distutils/command/build_ext.py Tue Sep 12 16:08:32 2000 @@ -425,7 +425,7 @@ extra_args = ext.extra_link_args or [] - self.compiler.link_shared_object ( + self.compiler.link (self.compiler.SHARED_OBJECT, objects, ext_filename, libraries=self.get_libraries(ext), library_dirs=ext.library_dirs, diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/cygwinccompiler.py distutils.patched/distutils/cygwinccompiler.py --- distutils.orig/distutils/cygwinccompiler.py Mon Sep 4 11:58:57 2000 +++ distutils.patched/distutils/cygwinccompiler.py Wed Sep 13 14:55:09 2000 @@ -45,8 +45,10 @@ __revision__ = "$Id: cygwinccompiler.py,v 1.7 2000/09/01 01:24:31 gward Exp $" import os,sys,copy +from distutils.ccompiler import gen_preprocess_options, gen_lib_options from distutils.unixccompiler import UnixCCompiler from distutils.file_util import write_file +from distutils.errors import DistutilsExecError,CompileError,UnknownFileError class CygwinCCompiler (UnixCCompiler): @@ -111,18 +113,77 @@ # __init__ () - def link_shared_object (self, - objects, - output_filename, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): + # not much different of the compile method in UnixCCompiler, + # but we have to insert some lines in the middle of it, so + # we put here a adapted version of it. + # (If we would call compile() in the base class, it would do some + # initializations a second time, this is why all is done here.) + def compile (self, + sources, + output_dir=None, + macros=None, + include_dirs=None, + debug=0, + extra_preargs=None, + extra_postargs=None): + + (output_dir, macros, include_dirs) = \ + self._fix_compile_args (output_dir, macros, include_dirs) + (objects, skip_sources) = self._prep_compile (sources, output_dir) + + # Figure out the options for the compiler command line. + pp_opts = gen_preprocess_options (macros, include_dirs) + cc_args = pp_opts + ['-c'] + if debug: + cc_args[:0] = ['-g'] + if extra_preargs: + cc_args[:0] = extra_preargs + if extra_postargs is None: + extra_postargs = [] + + # Compile all source files that weren't eliminated by + # '_prep_compile()'. + for i in range (len (sources)): + src = sources[i] ; obj = objects[i] + ext = (os.path.splitext (src))[1] + if skip_sources[src]: + self.announce ("skipping %s (%s up-to-date)" % (src, obj)) + else: + self.mkpath (os.path.dirname (obj)) + if ext == '.rc' or ext == '.res': + # gcc needs '.res' and '.rc' compiled to object files !!! + try: + self.spawn (["windres","-i",src,"-o",obj]) + except DistutilsExecError, msg: + raise CompileError, msg + else: # for other files use the C-compiler + try: + self.spawn (self.compiler_so + cc_args + + [src, '-o', obj] + + extra_postargs) + except DistutilsExecError, msg: + raise CompileError, msg + + # Return *all* object filenames, not just the ones we just built. + return objects + + # compile () + + + # XXXX needs to be checked with executables + def link (self, + target, + objects, + output_filename, + output_dir=None, + libraries=None, + library_dirs=None, + runtime_library_dirs=None, + export_symbols=None, + debug=0, + extra_preargs=None, + extra_postargs=None, + build_temp=None): # use separate copies, so we can modify the lists extra_preargs = copy.copy(extra_preargs or []) @@ -137,46 +198,49 @@ # where are the object files temp_dir = os.path.dirname(objects[0]) - # name of dll to give the helper files (def, lib, exp) the same name - (dll_name, dll_extension) = os.path.splitext( - os.path.basename(output_filename)) + # using export symbols with executables is not tested yet + if target <> UnixCCompiler.EXECUTABLE: + # name of dll to give the helper files (def, lib, exp) the same name + (dll_name, dll_extension) = os.path.splitext( + os.path.basename(output_filename)) - # generate the filenames for these files - def_file = None # this will be done later, if necessary - exp_file = os.path.join(temp_dir, dll_name + ".exp") - lib_file = os.path.join(temp_dir, 'lib' + dll_name + ".a") + # generate the filenames for these files + def_file = None # this will be done later, if necessary + exp_file = os.path.join(temp_dir, dll_name + ".exp") + lib_file = os.path.join(temp_dir, 'lib' + dll_name + ".a") - #extra_preargs.append("--verbose") - if self.linker == "dllwrap": - extra_preargs.extend([#"--output-exp",exp_file, - "--output-lib",lib_file, - ]) - else: - # doesn't work: bfd_close build\...\libfoo.a: Invalid operation - extra_preargs.extend([#"-Wl,--out-implib,%s" % lib_file, - ]) + if self.linker == "dllwrap": + extra_preargs.extend([#"--output-exp",exp_file, + "--output-lib",lib_file, + ]) + else: + # doesn't work: bfd_close build\...\libfoo.a: Invalid operation + extra_preargs.extend([#"-Wl,--out-implib,%s" % lib_file, + ]) - # check what we got in export_symbols - if export_symbols is not None: - # Make .def file - # (It would probably better to check if we really need this, - # but for this we had to insert some unchanged parts of - # UnixCCompiler, and this is not what we want.) - def_file = os.path.join(temp_dir, dll_name + ".def") - contents = [ - "LIBRARY %s" % os.path.basename(output_filename), - "EXPORTS"] - for sym in export_symbols: - contents.append(sym) - self.execute(write_file, (def_file, contents), - "writing %s" % def_file) + # check what we got in export_symbols + if export_symbols is not None: + # Make .def file + # (It would probably better to check if we really need this, + # but for this we had to insert some unchanged parts of + # UnixCCompiler, and this is not what we want.) + def_file = os.path.join(temp_dir, dll_name + ".def") + contents = [ + "LIBRARY %s" % os.path.basename(output_filename), + "EXPORTS"] + for sym in export_symbols: + contents.append(sym) + self.execute(write_file, (def_file, contents), + "writing %s" % def_file) - if def_file: - if self.linker == "dllwrap": - # for dllwrap we have to use a special option - extra_preargs.append("--def") - # for gcc/ld it is specified as any other object file - extra_preargs.append(def_file) + if def_file: + if self.linker == "dllwrap": + # for dllwrap we have to use a special option + extra_preargs.append("--def") + # for gcc/ld it is specified as any other object file + extra_preargs.append(def_file) + + # end: if target <> UnixCCompiler.EXECUTABLE # who wants symbols and a many times larger output file # should explicitly switch the debug mode on @@ -186,7 +250,8 @@ if not debug: extra_preargs.append("-s") - UnixCCompiler.link_shared_object(self, + UnixCCompiler.link(self, + target, objects, output_filename, output_dir, @@ -200,6 +265,34 @@ build_temp) # link_shared_object () + + # -- Miscellaneous methods ----------------------------------------- + + # overwrite the one from CCompiler to support rc and res-files + def object_filenames (self, + source_filenames, + strip_dir=0, + output_dir=''): + if output_dir is None: output_dir = '' + obj_names = [] + for src_name in source_filenames: + # use normcase to make sure '.rc' is really '.rc' and not '.RC' + (base, ext) = os.path.splitext (os.path.normcase(src_name)) + if ext not in (self.src_extensions + ['.rc','.res']): + raise UnknownFileError, \ + "unknown file type '%s' (from '%s')" % \ + (ext, src_name) + if strip_dir: + base = os.path.basename (base) + if ext == '.res' or ext == '.rc': + # these need to be compiled to object files + obj_names.append (os.path.join (output_dir, base + ext + self.obj_extension)) + else: + obj_names.append (os.path.join (output_dir, + base + self.obj_extension)) + return obj_names + + # object_filenames () # class CygwinCCompiler diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/msvccompiler.py distutils.patched/distutils/msvccompiler.py --- distutils.orig/distutils/msvccompiler.py Mon Sep 4 11:58:57 2000 +++ distutils.patched/distutils/msvccompiler.py Wed Sep 13 13:52:26 2000 @@ -201,7 +201,7 @@ version = versions[0] # highest version self.cc = find_exe("cl.exe", version) - self.link = find_exe("link.exe", version) + self.linker = find_exe("link.exe", version) self.lib = find_exe("lib.exe", version) set_path_env_var ('lib', version) set_path_env_var ('include', version) @@ -215,7 +215,7 @@ else: # devstudio not found in the registry self.cc = "cl.exe" - self.link = "link.exe" + self.linker = "link.exe" self.lib = "lib.exe" self.preprocess_options = None @@ -313,45 +313,19 @@ # create_static_lib () - - def link_shared_lib (self, - objects, - output_libname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): - - self.link_shared_object (objects, - self.shared_library_name(output_libname), - output_dir=output_dir, - libraries=libraries, - library_dirs=library_dirs, - runtime_library_dirs=runtime_library_dirs, - export_symbols=export_symbols, - debug=debug, - extra_preargs=extra_preargs, - extra_postargs=extra_postargs, - build_temp=build_temp) - - - def link_shared_object (self, - objects, - output_filename, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): + def link (self, + target, + objects, + output_filename, + output_dir=None, + libraries=None, + library_dirs=None, + runtime_library_dirs=None, + export_symbols=None, + debug=0, + extra_preargs=None, + extra_postargs=None, + build_temp=None): (objects, output_dir) = self._fix_object_args (objects, output_dir) (libraries, library_dirs, runtime_library_dirs) = \ @@ -369,10 +343,16 @@ if self._need_link (objects, output_filename): - if debug: - ldflags = self.ldflags_shared_debug + if target == CCompiler.EXECUTABLE: + if debug: + ldflags = self.ldflags_shared_debug[1:] + else: + ldflags = self.ldflags_shared[1:] else: - ldflags = self.ldflags_shared + if debug: + ldflags = self.ldflags_shared_debug + else: + ldflags = self.ldflags_shared export_opts = [] for sym in (export_symbols or []): @@ -386,12 +366,13 @@ # needed! Make sure they are generated in the temporary build # directory. Since they have different names for debug and release # builds, they can go into the same directory. - (dll_name, dll_ext) = os.path.splitext( - os.path.basename(output_filename)) - implib_file = os.path.join( - os.path.dirname(objects[0]), - self.library_filename(dll_name)) - ld_args.append ('/IMPLIB:' + implib_file) + if export_symbols is not None: + (dll_name, dll_ext) = os.path.splitext( + os.path.basename(output_filename)) + implib_file = os.path.join( + os.path.dirname(objects[0]), + self.library_filename(dll_name)) + ld_args.append ('/IMPLIB:' + implib_file) if extra_preargs: ld_args[:0] = extra_preargs @@ -400,65 +381,15 @@ self.mkpath (os.path.dirname (output_filename)) try: - self.spawn ([self.link] + ld_args) + self.spawn ([self.linker] + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: self.announce ("skipping %s (up-to-date)" % output_filename) - # link_shared_object () - - - def link_executable (self, - objects, - output_progname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - debug=0, - extra_preargs=None, - extra_postargs=None): - - (objects, output_dir) = self._fix_object_args (objects, output_dir) - (libraries, library_dirs, runtime_library_dirs) = \ - self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) - - if runtime_library_dirs: - self.warn ("I don't know what to do with 'runtime_library_dirs': " - + str (runtime_library_dirs)) - - lib_opts = gen_lib_options (self, - library_dirs, runtime_library_dirs, - libraries) - output_filename = output_progname + self.exe_extension - if output_dir is not None: - output_filename = os.path.join (output_dir, output_filename) - - if self._need_link (objects, output_filename): - - if debug: - ldflags = self.ldflags_shared_debug[1:] - else: - ldflags = self.ldflags_shared[1:] - - ld_args = ldflags + lib_opts + \ - objects + ['/OUT:' + output_filename] - - if extra_preargs: - ld_args[:0] = extra_preargs - if extra_postargs: - ld_args.extend (extra_postargs) + # link () - self.mkpath (os.path.dirname (output_filename)) - try: - self.spawn ([self.link] + ld_args) - except DistutilsExecError, msg: - raise LinkError, msg - else: - self.announce ("skipping %s (up-to-date)" % output_filename) - # -- Miscellaneous methods ----------------------------------------- # These are all used by the 'gen_lib_options() function, in diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/unixccompiler.py distutils.patched/distutils/unixccompiler.py --- distutils.orig/distutils/unixccompiler.py Tue Aug 8 13:18:38 2000 +++ distutils.patched/distutils/unixccompiler.py Tue Sep 12 16:06:19 2000 @@ -190,45 +190,19 @@ # create_static_lib () - def link_shared_lib (self, - objects, - output_libname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): - - self.link_shared_object ( - objects, - self.library_filename (output_libname, lib_type='shared'), - output_dir, - libraries, - library_dirs, - runtime_library_dirs, - export_symbols, - debug, - extra_preargs, - extra_postargs, - build_temp) - - - def link_shared_object (self, - objects, - output_filename, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - export_symbols=None, - debug=0, - extra_preargs=None, - extra_postargs=None, - build_temp=None): + def link (self, + target, + objects, + output_filename, + output_dir=None, + libraries=None, + library_dirs=None, + runtime_library_dirs=None, + export_symbols=None, + debug=0, + extra_preargs=None, + extra_postargs=None, + build_temp=None): (objects, output_dir) = self._fix_object_args (objects, output_dir) (libraries, library_dirs, runtime_library_dirs) = \ @@ -253,54 +227,16 @@ ld_args.extend (extra_postargs) self.mkpath (os.path.dirname (output_filename)) try: - self.spawn (self.linker_so + ld_args) - except DistutilsExecError, msg: - raise LinkError, msg - else: - self.announce ("skipping %s (up-to-date)" % output_filename) - - # link_shared_object () - - - def link_executable (self, - objects, - output_progname, - output_dir=None, - libraries=None, - library_dirs=None, - runtime_library_dirs=None, - debug=0, - extra_preargs=None, - extra_postargs=None): - - (objects, output_dir) = self._fix_object_args (objects, output_dir) - (libraries, library_dirs, runtime_library_dirs) = \ - self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) - - lib_opts = gen_lib_options (self, - library_dirs, runtime_library_dirs, - libraries) - output_filename = output_progname # Unix-ism! - if output_dir is not None: - output_filename = os.path.join (output_dir, output_filename) - - if self._need_link (objects, output_filename): - ld_args = objects + self.objects + lib_opts + ['-o', output_filename] - if debug: - ld_args[:0] = ['-g'] - if extra_preargs: - ld_args[:0] = extra_preargs - if extra_postargs: - ld_args.extend (extra_postargs) - self.mkpath (os.path.dirname (output_filename)) - try: - self.spawn (self.linker_exe + ld_args) + if target == CCompiler.EXECUTABLE: + self.spawn (self.linker_exe + ld_args) + else: + self.spawn (self.linker_so + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: self.announce ("skipping %s (up-to-date)" % output_filename) - # link_executable () + # link () # -- Miscellaneous methods -----------------------------------------
On 12 September 2000, Rene Liebscher said:
Output filenames:
* create_static_lib needs a library name, which becomes intern a real file name. (Is anyone using this method?)
Yes, this is used by the "build_clib" command. It's important functionality (IMHO).
* link_shared_library is similar to create_static_lib.
Right, because shared libraries and static libraries are (at least conceptually) similar. In implementation terms -- how they're created, what you can do with them, how they're formatted -- they are indeed quite different, as you observed later.
* link_shared_object needs a real file name.
* link_executable wants the basename of its output file.
'link_shared_object()' is the odd man out here: the other three all want an "abstract" name -- library or executable name -- rather than a concrete filename. 'link_shared_object()' was mainly conceived as a back-door to let you generate whatever filename you please, if the high-level interface ('link_shared_library()') doesn't cut it.
We have two methods shared_object_filename() and library_filename() (for static and shared libraries). Wouldn't it be better to add a method executable_filename() and use for all of the linker methods the real filename as parameter?
'shared_object_filename()' is a relic -- I just checked, and it's only defined by CCompiler, and not used anywhere. Fine to delete it. I agree about 'executable_filename()' -- good idea to add this for consistency.
For link_executable() we also need a parameter export_symbols. (An executable could export symbols to be able to be extended by plug-ins.)
OK, I didn't know that. Sounds like a reasonable change.
The best way would to merge these three methods into one new method and remove the old ones. (Or let the old ones call the new and give a warning the user should use the new method.)
Agreed. The backwards compatibility is a pain because these methods take so many parameters, but I think the 'link_this()' and 'link_that()' interface is nice to support. I wouldn't bother with a warning, though.
This new method would have following parameters: (It is better to see a shared libray as an half-build executable than to compare it with static libraries.)
def link (self, target, [...]
target would be one of 'shared_object', 'shared_library' or 'executable'.
Then it should be called 'target_desc' -- when I see "target", I think "filename".
The compiler classes would then be much simpler and smaller.
Good -- they need it.
This saves about 70 lines of redundant code. For BCPPCompiler and MSVCCompiler it is similar.
Excellent!
Are there any objections? If not, I would send Greg a patch for it.
OK -- I'll take a look at the patch.
Release 0.9 (29 June, 2000): ---------------------------- ...
* added the ability to compile and link in resource files with Visual C++ on Windows (Thomas Heller)
However, I can't find the place where it is done. Maybe it wasn't checked in ?
I think this was a change to msvccompiler.py. Can't remember exactly what the change was -- was there anything in the CVS log? Greg
On 17 September 2000, I said:
* added the ability to compile and link in resource files with Visual C++ on Windows (Thomas Heller)
However, I can't find the place where it is done. Maybe it wasn't checked in ?
I think this was a change to msvccompiler.py. Can't remember exactly what the change was -- was there anything in the CVS log?
Aiee! I goofed -- looks like that patch was never applied and checked in; it's still sitting in my patches directory. Arghgh! I'll attach the patch: can someone look it over, bring it up-to-date, test it, and send me a new version? Otherwise I'll have to do it myself, and who knows what damage I'll cause... ;-) Greg
Thomas Heller wrote:
On 12 September 2000, Rene Liebscher said:
For link_executable() we also need a parameter export_symbols. (An executable could export symbols to be able to be extended by plug-ins.)
But not on windows... (?)
Have you ever tried to use 'quikview' (German: Schnellansicht) with a executable or a dll? It shows you all exported symbols. For example most Borland C++ compiled executable export a hook for the debugger. netscape.exe exports about 8000 symbols. (see attached file for some examples) To programm plugins, which call functions in the main program, you link against the main program as you would do to link against a dll. (file: netscape_exports.txt) Kind regards Rene Liebscher ??0CPrefStore@@QAE@ABV0@@Z ??0CPrefStoreLocal@@QAE@ABV0@@Z ??0CPrefStoreLocalLI@@QAE@ABV0@@Z ??0CPrefStoreLockFile@@QAE@ABV0@@Z ??0CProfile@@QAE@ABV0@@Z ??0CProfileAuthKey@@QAE@ABV0@@Z ??0CProfileManager@@QAE@ABV0@@Z ??1CPrefStore@@UAE@XZ ??1CPrefStoreLocalLI@@UAE@XZ ??1CPrefStoreLockFile@@UAE@XZ ??4CPrefStore@@QAEAAV0@ABV0@@Z ??4CPrefStoreLocal@@QAEAAV0@ABV0@@Z ??4CPrefStoreLocalLI@@QAEAAV0@ABV0@@Z ??4CPrefStoreLockFile@@QAEAAV0@ABV0@@Z ??4CProfile@@QAEAAV0@ABV0@@Z ??4CProfileAuthKey@@QAEAAV0@ABV0@@Z ??4CProfileManager@@QAEAAV0@ABV0@@Z ??_7CPrefStore@@6B@ ??_7CPrefStoreLocal@@6B@ ??_7CPrefStoreLocalLI@@6B@ ??_7CPrefStoreLockFile@@6B@ ??_7CProfile@@6B@ ??_7CProfileAuthKey@@6B@ ??_7CProfileManager@@6B@ ??_ECPrefStore@@UAEPAXI@Z ??_ECPrefStoreLocal@@UAEPAXI@Z ??_ECPrefStoreLocalLI@@UAEPAXI@Z ??_ECPrefStoreLockFile@@UAEPAXI@Z ??_ECProfile@@UAEPAXI@Z ??_ECProfileAuthKey@@UAEPAXI@Z ??_ECProfileManager@@UAEPAXI@Z ??_GCPrefStore@@UAEPAXI@Z ??_GCPrefStoreLocal@@UAEPAXI@Z ??_GCPrefStoreLocalLI@@UAEPAXI@Z ??_GCPrefStoreLockFile@@UAEPAXI@Z ??_GCProfile@@UAEPAXI@Z ??_GCProfileAuthKey@@UAEPAXI@Z ??_GCProfileManager@@UAEPAXI@Z ?AttachCurrentThread@JavaVM_@@QAGJPAPAUJNIEnv_@@PAX@Z ?DestroyJavaVM@JavaVM_@@QAGJXZ ?DetachCurrentThread@JavaVM_@@QAGJXZ ?GetAuthLogin@CProfileAuthKey@@QAEPBDXZ ?GetAuthToken@CProfileAuthKey@@QAEPBDXZ ?GetConnectionInfo@CProfile@@UAEPAUProfileConnectionInfo@@XZ ... NET_ParseDate NET_ParseHTMLHelpFree NET_ParseHTMLHelpInit NET_ParseHTMLHelpLoadHelpDoc NET_ParseMimeHeader NET_ParseNetHelpURL NET_ParseURL NET_ParseUploadURL NET_PlainTextConverter NET_PlusToSpace NET_Pop3Load NET_PrefChangedFunc NET_PrintDirectory NET_PrintFileType NET_PrintNetlibStatus NET_PrintRawToDisk NET_ProcessAddressBook NET_ProcessCallbackURL NET_ProcessDataURL NET_ProcessFTP NET_ProcessFile ... xp_MakeHTMLDialogPass1 xp_MakeHTMLDialogPass2 xp_MakeHTMLDialogWindow zip_close zip_get zip_get_no_of_elements zip_list_files zip_lock zip_open zip_stat zip_unlock zlibVersion
participants (3)
-
Greg Ward
-
Rene Liebscher
-
Thomas Heller