[Distutils] Unravelling installation schemes

[Phillip J. Eby]

I'm working on implementing distutils dependencies, as described at:

http://www.python.org/cgi-bin/moinmoin/DistutilsDependencies

and I've run into a bit of a hitch.  In order to run dependencies' setup 
scripts, I need to be able to pass along command line options from the 
parent setup script.

But, it's not quite that simple.  The distutils framework allows people to 
specify quite a few options at multiple levels.  For example, you can 
specify a different C compiler to the build_lib and build_ext commands, and 
a third one to the build command itself!

Next, there's path munging and alternate installation schemes, along with 
the 'extra_path' support, install-lib overriding purelib and platlib, the 
distribution name embedded in the install-headers path, and so on.  My 
brain is approaching critical mass for explosion.  :)

Now, add to this some additional requirements that are specific to 
dependency management.  I need to know where stuff is being installed, so 
that when I check for the existence of a dependency, I'm not fooled by it 
not being on sys.path currently.  And, what if a dependency exists on 
extra_path?  [sob, sniff]

I guess I'm beginning to see why nobody else has tackled this up to now.  :)

Of course, for maybe half the installs it would suffice to just run 
'setup.py install' on the dependencies.  For about half of the remaining 
situations, it would probably suffice to just pass along whatever options 
were specified on the parent setup script's command line, verbatim.

It's the remaining scenarios that are tough:

* Some joker uses a relative path on the command line (could fix, if we 
know which arguments are directories)

* Non-build/non-install commands (could maybe fix by only copying options 
for build, install, and subcommands thereof)

* Somebody uses an explicit --install-lib or install_lib --install-dir 
(fixable, but it could screw things up if they really should've been 
specifying platlib or purelib or one of the higher-level options)

* Somebody uses an explicit install --install-headers or install_headers 
--install-dir (aaagh!)

So far, I'm thinking that about the only thing to do is:

1. Go through all the command-line options for install, build, and their 
subcommands

2. If *any* '--install-X' option exists apart from 'install-base' or 
'install-platbase', raise holy heck and stop the gravy train when trying to 
install dependencies.  Ask the user to either change their options, or to 
manually run the child setup with appropriate options, specifying where it 
is and what to do.

3. Only pass recognized-to-be safe options from build and its 
subcommands.  (Which is probably going to equate to just the compiler 
option, include-)  Fix up directory options from the install command to be 
absolute.  Barf if an inplace build_ext was specified.

4. Don't use any options that came from config files.  If it was a global 
config, it'll apply to the child setup automatically.  If it came from the 
local setup.cfg, it *shouldn't* apply to the child.

Whew.  So here's the thing...  am I making this too hard?  Is there a 
better way to deal with options to child setups?  I look at all the 
zillions of options to the individual commands, and I think about when and 
why I might specify those options, I think I mostly *wouldn't* want them 
passing through to the child setups.

Maybe the simplest thing to do would be to make a list of "safe" 
command-line options that will be passed to children, and issue warnings 
for the rest?  Or maybe stop the build if any options would be ignored, 
unless a "go ahead anyway" option is given?

What do y'all think?  I guess I'm now leaning towards that last 
option...  simply dump out on-screen all the command-line arguments that 
aren't safe, and show what *would* be passed to the child setup, and show 
what arguments would be needed to provide a command line that will force 
the build to proceed anyway, and also how to run the child setup manually 
(since the target will have already been downloaded and extracted to the 
temporary directory).

This won't address complex installation options, but then, if somebody has 
complex installation options, they can probably deal with a little 
complexity for installing the dependencies, I guess.

There are a few other issues floating about, mostly to do with sys.path, 
.pth files, and installation to locations other than site-packages.  But I 
think I'll leave those thoughts to another post.  :)