Hi all --
I've finally done some thinking and scribbling on how to build extensions -- well, C/C++ extensions for CPython. Java extensions for JPython will have to wait, but they are definitely looming on the horizon as something Distutils will have to handle.
Anyways, here are the conclusions I've arrived at.
* Stick with C/C++ for now; don't worry about other languages (yet). That way we can be smart about C/C++ things like preprocessor tokens and macros, include directories, shared vs static libraries, source and object files, etc.
* At the highest level, we should just be able to say "I know nothing, just give me a compiler object". This implies a factory function returning instances of concrete classes derived from an abstract CCompiler class. These compiler objects must know how to: - compile .c -> .o (or local equivalent) - compile multiple .c's to matching .o's - be able to define/undefine preprocessor macros/tokens - be able to supply preprocessor search directories - link multiple .o's to static library (libfoo.a, or local equiv.) - link multiple .o's to shared library (libfoo.so, or local equiv.) - link multiple .o's to shared object (foo.so, or local equiv.) - for all link steps: + be able to supply explicit libraries (/foo/bar/libbaz.a) + be able to supply implicit libraries (-lbaz) + be able to supply search directories for implicit libraries - do all this with timestamp-based dependency analysis (non-trivial because it requires analyzing header dependencies!)
Linking to static/shared libraries and dependency analysis are optional for now; everything else is required to build C/C++ extensions for Python. (At least that's my impression!)
"Local equivalent" is meant to encompass different filenames for C++ (eg. .C -> .o) and different operating systems/compilers (eg. .c -> .obj, multiple .obj's to foo.dll or foo.lib)
BIG QUESTION: I know this will work on Unix, and from my distant recollections of past work on other systems, it should work on MS-DOS and VMS too. I gather that Windows is pretty derivative of MS-DOS, so will this model work for Windows compilers too? Do we have to worry about Windows compilers other than VC++? But I have *no clue* about Macintosh compilers -- presumably somebody "out there" (not necessarily on this SIG, but I hope so!) knows how to compile Python on the Mac, so hopefully it's possible to compile Python extensions on the Mac. But will this compiler abstraction model work there?
Brushing that moment of self-doubt aside, here's a proposed interface for CCompiler and derived classes.
define_macro (name [, value]) define a preprocessor macro or token; this will affect all invocations of the 'compile()' method undefine_macro (name) undefine a preprocessor macro or token
add_include_dir (dir) add 'dir' to the list of directories that will be searched by the preprocessor for header files set_include_dir ([dirs]) reset the list of preprocessor search directories; 'dirs' should be a list or tuple of directory names; if not supplied, the list is cleared
compile (source, define=macro_list, undef=names, include_dirs=dirs) compile source file(s). 'source' may be a sequence of source filenames, all of which will be compiled, or a single filename to compile. The optional 'define', 'undef', and 'include_dirs' named parameters all augment the lists setup by the above four methods. 'macro_list' is a list of either 2-tuples (macro_name, value) or bare macro names. 'names' is a list of macro names, and 'dirs' a list of directories.
add_lib (libname) add a library name to the list of implicit libraries ("-lfoo") to link with set_libs ([libnames]) reset the list of implicit libraries (or clear if 'libnames' not supplied)
add_lib_dir (dir) add a directory to the list of library search directories ("-L/foo/bar/baz") used when we link set_lib_dirs ([dirs]) reset (or clear) the list of library search directorie
link_shared_object (objects, shared_object, libs=libnames, lib_dirs=dirs) link a set of object files together to create a shared object file. The optional 'libs' and 'lib_dirs' parameters only augment the lists setup by the previous four methods.
Things to think about: should there be explicit support for "explicit libraries" (eg. where you put "/foo/bar/libbaz.a" on the command line instead of trusting "-lbaz" to figure it out)? I don't think we can expect the caller to put them in the 'objects' list, because the filenames are too system-dependent. My inclination, as you could probably guess, would be to add methods 'add_explicit_lib()' and 'set_explicit_libs()', and a named parameter 'explicit_libs' to 'link_shared_objects()'.
Also, there would have to be methods to support creating static and shared libraries: I would call them 'link_static_lib()' and 'link_shared_lib()'. They would have the same interface as 'link_shared_object()', except the output filename would of course have to be handled differently. (To illustrate: on Unix-y systems, passing shared_object='foo' to 'link_shared_object()' would result in an output file 'foo.so'. But passing output_lib='foo' to 'link_shared_lib()' would result in 'libfoo.so', and passing it to 'link_static_lib()' would result in 'libfoo.a'.
So, to all the Windows and Mac experts out there: will this cover it? Can the variations in filename conventions and compilation/link schemes all be shoved under this umbrella? Or is it back to the drawing board?
Thanks for your comments!