Hi all -- as promised earlier in the week, I have completed the beginning steps along the road to a 'build_ext' command that will work under Unix. The legions of eager developers who watch every movement on the distutils-checkins list will know that I've just added two modules, ccompiler and unixccompiler, which provide the classes CCompiler and UnixCCompiler. The basic idea is this: CCompiler defines the interface to a generic C/C++ compiler, and UnixCCompiler implements an interface to the traditional Unix "cc -Dmacro -Iincludedir -Umacro -c foo.c -o foo.o" compiler invocation (and -l/-L linker invocation). So far all it does is generate and print out command lines, but that's enough to convince me that it works on my Linux/gcc system, i.e. it generates the command lines I intended it to generate, and gets the right preprocessor/compiler/linker flags from Python's Makefile (so that the basic compile/link steps are essentially the same as would be done by a Makefile.pre.in-generated Makefile). Please, take a look at the code. To encourage this, you'll find the bulk of ccompiler.py below: it's mostly comments and docstrings, since after all it mostly exists to define an interface. It's crucial that this interface be capable of what we need to build Python extensions on Unix, Windows, and Mac OS, and I'm relying on you folks to tell me what needs to be added to support Windows and Mac OS. (Well, if I missed something for Unix, be sure to tell me about that too. That's less likely, though, and I'll have a clue what you're talking about when politely inform me of my errors.) In particular: is this interface sufficient to handle Windows .def files? Is it enough for the weird case of using Oracle's libraries that Greg Stein mentioned (or other libraries with hairy interdependencies)? What Mac C compiler is supported, and is there a way to drive it programmatically? (Ie. is this even *possible* on the Mac?) Oh, if you're looking for some example code: see test/test_cc.py. Gives whatever CCompiler class applies on your platform a run for its money. (Currently only works when os.name == 'posix', since so far only UnixCCompiler is implemented.) Anyways, here's that hunk of ccompiler.py: class CCompiler: """Abstract base class to define the interface that must be implemented by real compiler abstraction classes. Might have some use as a place for shared code, but it's not yet clear what code can be shared between compiler abstraction models for different platforms. The basic idea behind a compiler abstraction class is that each instance can be used for all the compile/link steps in building a single project. Thus, attributes common to all of those compile and link steps -- include directories, macros to define, libraries to link against, etc. -- are attributes of the compiler instance. To allow for variability in how individual files are treated, most (all?) of those attributes may be varied on a per-compilation or per-link basis.""" # XXX things not handled by this compiler abstraction model: # * client can't provide additional options for a compiler, # e.g. warning, optimization, debugging flags. Perhaps this # should be the domain of concrete compiler abstraction classes # (UnixCCompiler, MSVCCompiler, etc.) -- or perhaps the base # class should have methods for the common ones. # * can't put output files (object files, libraries, whatever) # into a separate directory from their inputs. Should this be # handled by an 'output_dir' attribute of the whole object, or a # parameter to the compile/link_* methods, or both? # * can't completely override the include or library searchg # path, ie. no "cc -I -Idir1 -Idir2" or "cc -L -Ldir1 -Ldir2". # I'm not sure how widely supported this is even by POSIX # compilers, much less on other platforms. And I'm even less # sure how useful it is; probably for cross-compiling, but I # have no intention of supporting that. # * can't do really freaky things with the library list/library # dirs, e.g. "-Ldir1 -lfoo -Ldir2 -lfoo" to link against # different versions of libfoo.a in different locations. I # think this is useless without the ability to null out the # library search path anyways. # * don't deal with verbose and dry-run flags -- probably a # CCompiler object should just drag them around the way the # Distribution object does (either that or we have to drag # around a Distribution object, which is what Command objects # do... but might be kind of annoying) [...] # -- Bookkeeping methods ------------------------------------------- def define_macro (self, name, value=None): """Define a preprocessor macro for all compilations driven by this compiler object. The optional parameter 'value' should be a string; if it is not supplied, then the macro will be defined without an explicit value and the exact outcome depends on the compiler used (XXX true? does ANSI say anything about this?)""" def undefine_macro (self, name): """Undefine a preprocessor macro for all compilations driven by this compiler object. If the same macro is defined by 'define_macro()' and undefined by 'undefine_macro()' the last call takes precedence (including multiple redefinitions or undefinitions). If the macro is redefined/undefined on a per-compilation basis (ie. in the call to 'compile()'), then that takes precedence.""" def add_include_dir (self, dir): """Add 'dir' to the list of directories that will be searched for header files. The compiler is instructed to search directories in the order in which they are supplied by successive calls to 'add_include_dir()'.""" def set_include_dirs (self, dirs): """Set the list of directories that will be searched to 'dirs' (a list of strings). Overrides any preceding calls to 'add_include_dir()'; subsequence calls to 'add_include_dir()' add to the list passed to 'set_include_dirs()'. This does not affect any list of standard include directories that the compiler may search by default.""" def add_library (self, libname): """Add 'libname' to the list of libraries that will be included in all links driven by this compiler object. Note that 'libname' should *not* be the name of a file containing a library, but the name of the library itself: the actual filename will be inferred by the linker, the compiler, or the compiler abstraction class (depending on the platform). The linker will be instructed to link against libraries in the order they were supplied to 'add_library()' and/or 'set_libraries()'. It is perfectly valid to duplicate library names; the linker will be instructed to link against libraries as many times as they are mentioned.""" def set_libraries (self, libnames): """Set the list of libraries to be included in all links driven by this compiler object to 'libnames' (a list of strings). This does not affect any standard system libraries that the linker may include by default.""" def add_library_dir (self, dir): """Add 'dir' to the list of directories that will be searched for libraries specified to 'add_library()' and 'set_libraries()'. The linker will be instructed to search for libraries in the order they are supplied to 'add_library_dir()' and/or 'set_library_dirs()'.""" def set_library_dirs (self, dirs): """Set the list of library search directories to 'dirs' (a list of strings). This does not affect any standard library search path that the linker may search by default.""" def add_link_object (self, object): """Add 'object' to the list of object files (or analogues, such as explictly named library files or the output of "resource compilers") to be included in every link driven by this compiler object.""" def set_link_objects (self, objects): """Set the list of object files (or analogues) to be included in every link to 'objects'. This does not affect any standard object files that the linker may include by default (such as system libraries).""" # -- Worker methods ------------------------------------------------ # (must be implemented by subclasses) def compile (self, sources, macros=None, includes=None): """Compile one or more C/C++ source files. 'sources' must be a list of strings, each one the name of a C/C++ source file. Return a list of the object filenames generated (one for each source filename in 'sources'). 'macros', if given, must be a list of macro definitions. A macro definition is either a (name, value) 2-tuple or a (name,) 1-tuple. The former defines a macro; if the value is None, the macro is defined without an explicit value. The 1-tuple case undefines a macro. Later definitions/redefinitions/ undefinitions take precedence. 'includes', if given, must be a list of strings, the directories to add to the default include file search path for this compilation only.""" pass # XXX this is kind of useless without 'link_binary()' or # 'link_executable()' or something -- or maybe 'link_static_lib()' # should not exist at all, and we just have 'link_binary()'? def link_static_lib (self, objects, output_libname, libraries=None, library_dirs=None): """Link a bunch of stuff together to create a static library file. The "bunch of stuff" consists of the list of object files supplied as 'objects', the extra object files supplied to 'add_link_object()' and/or 'set_link_objects()', the libraries supplied to 'add_library()' and/or 'set_libraries()', and the libraries supplied as 'libraries' (if any). 'output_libname' should be a library name, not a filename; the filename will be inferred from the library name. 'library_dirs', if supplied, should be a list of additional directories to search on top of the system default and those supplied to 'add_library_dir()' and/or 'set_library_dirs()'.""" pass # XXX what's better/more consistent/more universally understood # terminology: "shared library" or "dynamic library"? def link_shared_lib (self, objects, output_libname, libraries=None, library_dirs=None): """Link a bunch of stuff together to create a shared library file. Has the same effect as 'link_static_lib()' except that the filename inferred from 'output_libname' will most likely be different, and the type of file generated will almost certainly be different.""" pass def link_shared_object (self, objects, output_filename, libraries=None, library_dirs=None): """Link a bunch of stuff together to create a shared object file. Much like 'link_shared_lib()', except the output filename is explicitly supplied as 'output_filename'.""" pass # class CCompiler Hope you enjoyed that as much as I did. ;-) Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913