[Distutils] Pondering multi-package packages

[Greg Ward]

On 26 May 2000, M.-A. Lemburg said:
> Hmm, the pre/post-install hooks are definitely an install thing.

Again, there are two meanings to install: install from source and
install from a built distribution.  Hooks at install-from-source time
should be doable with the Distutils' existing extension mechanism (if a
bit cumbersome -- you have to know how to write a Distutils command
class, which is a tad idiosyncratic).

> We'd also need a pre-build, though, for the things I mentioned
> below, e.g. finding header files and libs, checking the compiler,
> etc.

In principle, that should all be doable with the Distutils' existing
facilities.  I'd be willing to grease the wheels by adding a standard
"prebuild" or "configure" command that runs before "build", but I'd
leave it empty and open to subclassing -- there's just too many things
that you might want to do there!

So eg. you might do this in your setup script:

class configure (Command):
    user_options = [('foo-inc', None,
                    ("where to search for foo headers"),
                    ('foo-lib', None,
                     "where to search for foo library"),
                   ]

    def initialize_options (self):
        self.foo_inc = None
        self.foo_lib = None

    def finalize_options (self):
        # if user doesn't define foo_inc and foo_lib, leave them
        # alone (we will search for foo.h and libfoo)
        pass

    def run (self):
        if self.foo_inc is None:
            for dir in (...):      # list would vary by platform
                # try to compile "#include <foo.h>" with -Idir
                # break if success
            else:
                # die, couldn't find foo.h

        # similar loop, trying to link with -lfoo and -Ldir

and then a little later:
    setup (...,
           cmdclass = {'configure': configure},
           ...,
          )

Now, what the hell do we do with 'foo_inc' and 'foo_lib' -- as written,
the 'configure' command finds the foo header and library paths, and then 
quits without doing anything with that information.

If you happen to know the guts of the Distutils, you could be evil and
sneaky and do something like this:

    build_ext_opts = self.distribution.get_option_dict('build_ext')
    include_dirs = build_ext_opts.get('include_dirs')
    if include_dirs is None:
        include_dirs = build_ext_opts['include_dirs'] = []
    include_dirs.append(self.foo_inc)

...and then similar code to modify build_ext's 'library_dirs' option
from 'self.foo_lib'.

This is nasty, though.  Come to think of it, it's not entirely reliable:
if the "build_ext" command object has already been created by the time
we run "configure", then it's too late to go frobbing the option dict
owned by the Distribution object -- you'd want to frob the "build_ext"
object directly.

Well, it's a common idiom to *fetch* options from another command
object.  And, oh yeah, I decided many months ago to stick with this
"pull" model -- if command X needs option Y from command Z, then it's
X's responsibility to dig up a Z object and get attribute Y from it.
Just search the code for 'find_peer' to see how often this happens.
Eg. in bdist.py:
    build_base = self.find_peer('build').build_base
to find the build base directory.

But there's no way the general-purpose "build" command can know what's
defined in your particular "configure" command -- so this is one place
where we seem to need to support "pushing" options.  The problem with
pushing options from one command to another is that option
initialization is *tricky*, because we need to be able to derive default
values in an intelligent way.  See build_ext.py for a rich, meaty, but
comprehensible example; or install.py for an insanely complex example.

I think the difficulty of pushing options boils down to the fact that
'finalize_options()' only expects to be called once, and most commands
are written in such a way that they die horribly if it is called more
than once.  (I have accidentally ventured into option-pushing territory
once or twice in the past, and quickly retreated, licking my wounds.)
This is a design/implementation flaw that I have lived with up to now,
but I might not be able to any longer.

Now do you see why I have avoided a "configure" command?  ;-)

Other little things...

> Why not add some keywords to the constructor ?!
> 
> import mx.ODBC.Misc.DistSupport
> setup(
> 	preinstall = mx.ODBC.Misc.DistSupport.preinstall,
> 	postinstall = ...postinstall,
> 	prebuild = ...prebuild,
> 	postbuild = ...postbuild
> 	)

I realize that the OO write-your-own-class alternative is a little more
clunky, but I don't think it's clunky enough to mandate a
function-passing interface.  Can you buy that?

> Naa... no need to rewrite Autoconf in Python: the simple tests
> can easily be done using a few lines of Python provided that
> the compiler classes allow these trial-and-error approaches.

You may be right, based on the above hypothetical configure command.
Abstracting some of these common functions away shouldn't be too hard.

        Greg
-- 
Greg Ward - geek-on-the-loose                           gward@python.net
http://starship.python.net/~gward/
Hold the MAYO & pass the COSMIC AWARENESS ...