[Distutils] Pondering multi-package packages
M.-A. Lemburg
mal@lemburg.com
Sat, 27 May 2000 11:20:45 +0200
Greg Ward wrote:
>
> On 26 May 2000, M.-A. Lemburg said:
> > Hmm, the pre/post-install hooks are definitely an install thing.
>
> Again, there are two meanings to install: install from source and
> install from a built distribution. Hooks at install-from-source time
> should be doable with the Distutils' existing extension mechanism (if a
> bit cumbersome -- you have to know how to write a Distutils command
> class, which is a tad idiosyncratic).
I was referring to installing a (pre)built binary -- just before
copying the compiled files to their final install location and
right after that step is done.
"install-from-source" would execute these hooks too: right after
having built the binaries.
> > We'd also need a pre-build, though, for the things I mentioned
> > below, e.g. finding header files and libs, checking the compiler,
> > etc.
>
> In principle, that should all be doable with the Distutils' existing
> facilities. I'd be willing to grease the wheels by adding a standard
> "prebuild" or "configure" command that runs before "build", but I'd
> leave it empty and open to subclassing -- there's just too many things
> that you might want to do there!
Ok... if you beat me to it, I'll do some subclassing then ;-)
> So eg. you might do this in your setup script:
>
> class configure (Command):
> user_options = [('foo-inc', None,
> ("where to search for foo headers"),
> ('foo-lib', None,
> "where to search for foo library"),
> ]
>
> def initialize_options (self):
> self.foo_inc = None
> self.foo_lib = None
>
> def finalize_options (self):
> # if user doesn't define foo_inc and foo_lib, leave them
> # alone (we will search for foo.h and libfoo)
> pass
>
> def run (self):
> if self.foo_inc is None:
> for dir in (...): # list would vary by platform
> # try to compile "#include <foo.h>" with -Idir
> # break if success
> else:
> # die, couldn't find foo.h
>
> # similar loop, trying to link with -lfoo and -Ldir
>
> and then a little later:
> setup (...,
> cmdclass = {'configure': configure},
> ...,
> )
Looks feasable :-)
> Now, what the hell do we do with 'foo_inc' and 'foo_lib' -- as written,
> the 'configure' command finds the foo header and library paths, and then
> quits without doing anything with that information.
>
> If you happen to know the guts of the Distutils, you could be evil and
> sneaky and do something like this:
>
> build_ext_opts = self.distribution.get_option_dict('build_ext')
> include_dirs = build_ext_opts.get('include_dirs')
> if include_dirs is None:
> include_dirs = build_ext_opts['include_dirs'] = []
> include_dirs.append(self.foo_inc)
>
> ...and then similar code to modify build_ext's 'library_dirs' option
> from 'self.foo_lib'.
>
> This is nasty, though. Come to think of it, it's not entirely reliable:
> if the "build_ext" command object has already been created by the time
> we run "configure", then it's too late to go frobbing the option dict
> owned by the Distribution object -- you'd want to frob the "build_ext"
> object directly.
>
> Well, it's a common idiom to *fetch* options from another command
> object. And, oh yeah, I decided many months ago to stick with this
> "pull" model -- if command X needs option Y from command Z, then it's
> X's responsibility to dig up a Z object and get attribute Y from it.
> Just search the code for 'find_peer' to see how often this happens.
> Eg. in bdist.py:
> build_base = self.find_peer('build').build_base
> to find the build base directory.
Wouldn't a method interface be more reliable and provide
better means of extension using subclassing ?
I usually wrap these attributes in .get_foobar(), .set_foobar()
methods -- this also makes it clear which attributes are
read-only, read-write or "better don't touch" :-)
> But there's no way the general-purpose "build" command can know what's
> defined in your particular "configure" command -- so this is one place
> where we seem to need to support "pushing" options. The problem with
> pushing options from one command to another is that option
> initialization is *tricky*, because we need to be able to derive default
> values in an intelligent way. See build_ext.py for a rich, meaty, but
> comprehensible example; or install.py for an insanely complex example.
>
> I think the difficulty of pushing options boils down to the fact that
> 'finalize_options()' only expects to be called once, and most commands
> are written in such a way that they die horribly if it is called more
> than once. (I have accidentally ventured into option-pushing territory
> once or twice in the past, and quickly retreated, licking my wounds.)
> This is a design/implementation flaw that I have lived with up to now,
> but I might not be able to any longer.
>
> Now do you see why I have avoided a "configure" command? ;-)
Ehm, yes... but I can't really follow here: ok, I don't know
much about the internals of distutils, but wouldn't passing
a (more-or-less) intelligent context object around solve
the problem ? The context object would know which parts are
readable, changeable or write-once, etc.
(I've been doing this in an 55k LOC application server and it works
great.)
> Other little things...
>
> > Why not add some keywords to the constructor ?!
> >
> > import mx.ODBC.Misc.DistSupport
> > setup(
> > preinstall = mx.ODBC.Misc.DistSupport.preinstall,
> > postinstall = ...postinstall,
> > prebuild = ...prebuild,
> > postbuild = ...postbuild
> > )
>
> I realize that the OO write-your-own-class alternative is a little more
> clunky, but I don't think it's clunky enough to mandate a
> function-passing interface. Can you buy that?
Ok.
> > Naa... no need to rewrite Autoconf in Python: the simple tests
> > can easily be done using a few lines of Python provided that
> > the compiler classes allow these trial-and-error approaches.
>
> You may be right, based on the above hypothetical configure command.
> Abstracting some of these common functions away shouldn't be too hard.
Basically, I need:
rc = compile("""#include "foobar.h"\nmain(){}""", output="delete.me")
if rc != 0:
HAVE_FOOBAR_H = 0
else:
HAVE_FOOBAR_H = 1
os.unlink("delete.me")
and sometimes:
rc = compile("""#include "foobar.h"
main()
{
int x = frobnicate();
exit(x);
}""", output="run-and-then-delete.me")
if rc != 0:
HAVE_FROBNICATE = 0
else:
HAVE_FROBNICATE = 1
# Run and get
rc = os.system("run-and-then-delete.me")
FROBINATE_VALUE = rc
os.unlink("run-and-then-delete.me")
--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/