Hi all --
I've finally done some thinking and scribbling on how to build
extensions -- well, C/C++ extensions for CPython. Java extensions for
JPython will have to wait, but they are definitely looming on the
horizon as something Distutils will have to handle.
Anyways, here are the conclusions I've arrived at.
* Stick with C/C++ for now; don't worry about other languages (yet).
That way we can be smart about C/C++ things like preprocessor
tokens and macros, include directories, shared vs static
libraries, source and object files, etc.
* At the highest level, we should just be able to say "I know nothing,
just give me a compiler object". This implies a factory function
returning instances of concrete classes derived from an abstract
CCompiler class. These compiler objects must know how to:
- compile .c -> .o (or local equivalent)
- compile multiple .c's to matching .o's
- be able to define/undefine preprocessor macros/tokens
- be able to supply preprocessor search directories
- link multiple .o's to static library (libfoo.a, or local equiv.)
- link multiple .o's to shared library (libfoo.so, or local equiv.)
- link multiple .o's to shared object (foo.so, or local equiv.)
- for all link steps:
+ be able to supply explicit libraries (/foo/bar/libbaz.a)
+ be able to supply implicit libraries (-lbaz)
+ be able to supply search directories for implicit libraries
- do all this with timestamp-based dependency analysis
(non-trivial because it requires analyzing header dependencies!)
Linking to static/shared libraries and dependency analysis are
optional for now; everything else is required to build C/C++
extensions for Python. (At least that's my impression!)
"Local equivalent" is meant to encompass different filenames for C++
(eg. .C -> .o) and different operating systems/compilers (eg. .c ->
.obj, multiple .obj's to foo.dll or foo.lib)
BIG QUESTION: I know this will work on Unix, and from my distant
recollections of past work on other systems, it should work on MS-DOS
and VMS too. I gather that Windows is pretty derivative of MS-DOS, so
will this model work for Windows compilers too? Do we have to worry
about Windows compilers other than VC++? But I have *no clue* about
Macintosh compilers -- presumably somebody "out there" (not necessarily
on this SIG, but I hope so!) knows how to compile Python on the Mac, so
hopefully it's possible to compile Python extensions on the Mac. But
will this compiler abstraction model work there?
Brushing that moment of self-doubt aside, here's a proposed interface
for CCompiler and derived classes.
define_macro (name [, value])
define a preprocessor macro or token; this will affect all
invocations of the 'compile()' method
undefine_macro (name)
undefine a preprocessor macro or token
add_include_dir (dir)
add 'dir' to the list of directories that will be searched by
the preprocessor for header files
set_include_dir ([dirs])
reset the list of preprocessor search directories; 'dirs' should
be a list or tuple of directory names; if not supplied, the list
is cleared
compile (source, define=macro_list, undef=names, include_dirs=dirs)
compile source file(s). 'source' may be a sequence of source
filenames, all of which will be compiled, or a single filename to
compile. The optional 'define', 'undef', and 'include_dirs'
named parameters all augment the lists setup by the above four
methods. 'macro_list' is a list of either 2-tuples
(macro_name, value) or bare macro names. 'names' is a list of
macro names, and 'dirs' a list of directories.
add_lib (libname)
add a library name to the list of implicit libraries ("-lfoo")
to link with
set_libs ([libnames])
reset the list of implicit libraries (or clear if 'libnames'
not supplied)
add_lib_dir (dir)
add a directory to the list of library search directories
("-L/foo/bar/baz") used when we link
set_lib_dirs ([dirs])
reset (or clear) the list of library search directorie
link_shared_object (objects, shared_object,
libs=libnames, lib_dirs=dirs)
link a set of object files together to create a shared object file.
The optional 'libs' and 'lib_dirs' parameters only augment the
lists setup by the previous four methods.
Things to think about: should there be explicit support for "explicit
libraries" (eg. where you put "/foo/bar/libbaz.a" on the command line
instead of trusting "-lbaz" to figure it out)? I don't think we can
expect the caller to put them in the 'objects' list, because the
filenames are too system-dependent. My inclination, as you could
probably guess, would be to add methods 'add_explicit_lib()' and
'set_explicit_libs()', and a named parameter 'explicit_libs' to
'link_shared_objects()'.
Also, there would have to be methods to support creating static and
shared libraries: I would call them 'link_static_lib()' and
'link_shared_lib()'. They would have the same interface as
'link_shared_object()', except the output filename would of course have
to be handled differently. (To illustrate: on Unix-y systems,
passing shared_object='foo' to 'link_shared_object()' would result in an
output file 'foo.so'. But passing output_lib='foo' to
'link_shared_lib()' would result in 'libfoo.so', and passing it to
'link_static_lib()' would result in 'libfoo.a'.
So, to all the Windows and Mac experts out there: will this cover it?
Can the variations in filename conventions and compilation/link schemes
all be shoved under this umbrella? Or is it back to the drawing board?
Thanks for your comments!
Greg
--
Greg Ward - software developer gward(a)cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990 x287
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
