[Distutils] Unravelling installation schemes
Phillip J. Eby
pje at telecommunity.com
Tue Mar 9 18:57:49 EST 2004
I'm working on implementing distutils dependencies, as described at:
and I've run into a bit of a hitch. In order to run dependencies' setup
scripts, I need to be able to pass along command line options from the
parent setup script.
But, it's not quite that simple. The distutils framework allows people to
specify quite a few options at multiple levels. For example, you can
specify a different C compiler to the build_lib and build_ext commands, and
a third one to the build command itself!
Next, there's path munging and alternate installation schemes, along with
the 'extra_path' support, install-lib overriding purelib and platlib, the
distribution name embedded in the install-headers path, and so on. My
brain is approaching critical mass for explosion. :)
Now, add to this some additional requirements that are specific to
dependency management. I need to know where stuff is being installed, so
that when I check for the existence of a dependency, I'm not fooled by it
not being on sys.path currently. And, what if a dependency exists on
extra_path? [sob, sniff]
I guess I'm beginning to see why nobody else has tackled this up to now. :)
Of course, for maybe half the installs it would suffice to just run
'setup.py install' on the dependencies. For about half of the remaining
situations, it would probably suffice to just pass along whatever options
were specified on the parent setup script's command line, verbatim.
It's the remaining scenarios that are tough:
* Some joker uses a relative path on the command line (could fix, if we
know which arguments are directories)
* Non-build/non-install commands (could maybe fix by only copying options
for build, install, and subcommands thereof)
* Somebody uses an explicit --install-lib or install_lib --install-dir
(fixable, but it could screw things up if they really should've been
specifying platlib or purelib or one of the higher-level options)
* Somebody uses an explicit install --install-headers or install_headers
So far, I'm thinking that about the only thing to do is:
1. Go through all the command-line options for install, build, and their
2. If *any* '--install-X' option exists apart from 'install-base' or
'install-platbase', raise holy heck and stop the gravy train when trying to
install dependencies. Ask the user to either change their options, or to
manually run the child setup with appropriate options, specifying where it
is and what to do.
3. Only pass recognized-to-be safe options from build and its
subcommands. (Which is probably going to equate to just the compiler
option, include-) Fix up directory options from the install command to be
absolute. Barf if an inplace build_ext was specified.
4. Don't use any options that came from config files. If it was a global
config, it'll apply to the child setup automatically. If it came from the
local setup.cfg, it *shouldn't* apply to the child.
Whew. So here's the thing... am I making this too hard? Is there a
better way to deal with options to child setups? I look at all the
zillions of options to the individual commands, and I think about when and
why I might specify those options, I think I mostly *wouldn't* want them
passing through to the child setups.
Maybe the simplest thing to do would be to make a list of "safe"
command-line options that will be passed to children, and issue warnings
for the rest? Or maybe stop the build if any options would be ignored,
unless a "go ahead anyway" option is given?
What do y'all think? I guess I'm now leaning towards that last
option... simply dump out on-screen all the command-line arguments that
aren't safe, and show what *would* be passed to the child setup, and show
what arguments would be needed to provide a command line that will force
the build to proceed anyway, and also how to run the child setup manually
(since the target will have already been downloaded and extracted to the
This won't address complex installation options, but then, if somebody has
complex installation options, they can probably deal with a little
complexity for installing the dependencies, I guess.
There are a few other issues floating about, mostly to do with sys.path,
.pth files, and installation to locations other than site-packages. But I
think I'll leave those thoughts to another post. :)
More information about the Distutils-SIG