[Distutils] distutils - locating include files, specifying C libraries
Greg Ward
gward@python.net
Wed, 8 Mar 2000 22:08:03 -0500
On 07 March 2000, Joe Van Andel said:
> I'm trying to use distutils (CVS from 3/6/2000) to build C++ extensions
> that use the Numeric library. I'm using Python 1.5.2, Redhat 6.1,
> Solaris 2.6 and Gcc 2.95.2
Great -- first successful build of C++ extensions, AFAIK. I hope the
compiler interface is holding up under the strain. Planning to try it
under Windows?
> I'm pleased that I can build and install my extensions for both Solaris
> and Linux, now that I've built workarounds for the two problems I
> describe below. Given that distutils automatically builds libraries and
> objects in machine-dependent temporary directories, I find it much
> easier to use than the corresponding Makefiles.
Oh good, another completely untested feature getting a workout. The
intention of having per-architecture build directories was for
installers and/or developers to be able to do this:
[on arch. A, eg. Linux]
python setup.py build
[on arch. B, eg. Solaris]
python setup.py build
and have *no* interference between the two builds, ie. everything would
be in the build/temp.<plat> directory. You should be able to build on
A, build on B, install on A, install on B. Whatever. I have not tested
any of this for real, though, and it's only been in the source since the
weekend: so, does it work for you?
> Here are two problems that should be addressed.
>
> Problem 1 - locating Python extension include files.
Hmm, yeah, I worried about this a bit when hacking Numeric's setup
script to install header files. Didn't have an answer then.
> I want to specify the location of the Numeric include files in an
> installation/machine independent way. I added the following into my
> setup.py file:
[...]
> config_h = sysconfig.get_config_h_filename()
> py_inc = os.path.dirname(config_h)
> NUM_INC = os.path.join(py_inc, 'Numeric')
Nice kludge, but subtly wrong: the config.h directory is for
platform-specific header files. You most likely want the
non-platform-specific header file directory, ie. where Python.h and
friends live.
I think your patch (add 'get_python_inc()') is 95% exactly right.
Here's my adaptation of it, with Mac support based on Corran Webster's
'_init_mac()' (checked in last night -- thanks Corran!):
def get_python_inc (plat_specific=0):
"""Return the directory containing installed Python header files.
If 'plat_specific' is false (the default), this is the path to the
non-platform-specific header files, i.e. Python.h and so on;
otherwise, this is the path to platform-specific header files
(namely config.h)."""
the_prefix = (plat_specific and exec_prefix or prefix)
if os.name == "posix":
return os.path.join (the_prefix,
"include", "python" + sys.version[:3])
elif os.name == "nt":
return os.path.join (the_prefix, "Include") # include or Include?
elif os.name == "mac":
return os.path.join (the_prefix, "Include")
else:
raise DistutilsPlatformError, \
("I don't know where Python installs its C header files " +
"on platform '%s'") % os.name
Ummm, should that be "Include" on Windows for consistency with "Lib"?
(Yes yes, I know it doesn't strictly matter, but I'm a stickler for
aesthetic consistency, even in a tty-style UI.)
'get_config_h_filename()' should change to use the new
function... there, done. And come to think of it, why not add
'get_python_lib()'? The 'install_py' command should probably use this
instead of some ugly GLOBAL exported by sysconfig, and it could be handy
for setup scripts that want to do naughty things but still play nice
(ie. use portable directory names).
Here's 'get_python_lib()':
def get_python_lib (plat_specific=0, standard_lib=0):
"""Return the directory containing the Python library (standard or
site additions). If 'plat_specific' is true, return the directory
containing platform-specific modules, i.e. any module from a
non-pure-Python module distribution; otherwise, return the
platform-shared library directory. If 'standard_lib' is true,
return the directory containing standard Python library modules;
otherwise, return the directory for site-specific modules."""
the_prefix = (plat_specific and exec_prefix or prefix)
if os.name == "posix":
libpython = os.path.join (the_prefix,
"lib", "python" + sys.version[:3])
if standard_lib:
return libpython
else:
return os.path.join (libpython, "site-packages")
elif os.name == "nt":
if standard_lib:
return os.path.join (the_prefix, "Lib")
else:
return the_prefix
elif os.name == "mac":
if platform_specific:
if standard_lib:
return os.path.join (exec_prefix, "Mac", "Plugins")
else:
raise DistutilsPlatformError, \
"OK, where DO site-specific extensions go on the Mac?"
else:
if standard_lib:
return os.path.join (prefix, "Lib")
else:
raise DistutilsPlatformError, \
"OK, where DO site-specific modules go on the Mac?"
else:
raise DistutilsPlatformError, \
("I don't know where Python installs its library " +
"on platform '%s'") % os.name
# get_python_lib ()
Hint hint: I need help with the Mac support!
> Problem 2 - specifying architecture/compiler specific libraries to
> setup.py
>
> When I link my extension on Linux (RH 6.1), I need to specify
> 'libraries': ['gcc', 'stdc++','pthread'],
Nope, I still don't have an answer to this one.
> (At this point, we start approaching the complex world of GNU
> 'autoconf', that dynamically checks for the existence of specified
> libraries! )
Well, except for this non-answer. If I start rewriting Autoconf in
Python for Distutils 1.0, though, Guido will kill me (there's already
too damn much code in this thing!).
> Here's a solution, is there a better one?
>
> from distutils.util import get_platform
> plat = get_platform()
>
> # machine dependent C/C++ support libs
> if plat == 'linux-i586' :
> CLIBS=['gcc', 'stdc++','pthread']
> else:
> CLIBS=[]
That would be the "everybody-has-a-personal-imake-database-in-their-
setup-script" non-solution. Ick. See also the pil_setup.py example in
the distutils source, which goes for the "everyone-knows-what-libraries-
they-have-installed-and-edits-the-setup-script" non-solution, which is
approximately as nasty.
So far, I have yet to be convinced that there is *any* long-term
solution short of rewriting Autoconf in Python. (Distutils 2.0,
anyone?) And oh yeah, this has to be an Autoconf that runs on Unix,
Windows, and Mac OS. Good thing there's a portable C/C++ compiler
framework!
Greg
--
Greg Ward - just another /P(erl|ython)/ hacker gward@python.net
http://starship.python.net/~gward/
God is real, unless declared integer.