[Distutils] Unravelling installation schemes

[Mark W. Alexander]

On Tue, Mar 09, 2004 at 06:57:49PM -0500, Phillip J. Eby wrote:
> I guess I'm beginning to see why nobody else has tackled this up to now.  :)

My $.02 is, Distutils can take one of 2 forms. It can be
automake/autoconf for Python and everyone can install from source, or it
can be a package manager. The battle you're fighting seems to me to be
from trying to make it do both.

If Distutils can make binary packages for most package managers, and it
can be extended to include "Provides" and "Requires" in the meta-data,
then any OS's native package manager should have all the information it
needs to do the right thing at install time.

> Of course, for maybe half the installs it would suffice to just run 
> 'setup.py install' on the dependencies.  For about half of the remaining 
> situations, it would probably suffice to just pass along whatever options 
> were specified on the parent setup script's command line, verbatim.

Why does everyone seem to think that the prefered method of Distutils
package installations is "setup.py install"? What's the point of
providing bdist_* commands if we're going to perpetualy re-inforce the
"setup.py install" mindset? I'm not trying to be confrontational here,
but do we seriously expect everyone who wants to install Distutils
packages to have a compiler suite in case a package has C extensions?
Even without C extensions, I suspect you can rule out most Windows users
by just telling them to "setup.py install". You'll be lucky if you can
get them to listen long enough when they ask you to explain how to pass
"install" as an option when they click the setup.py icon.

I really, really, think Distutils should _produce_binary_packages. For
as many platforms and package managers as possible, with as much
meta-data as each package manager supports. Then we can just let the
package managers worry about dependencies and not re-invent another
wheel.

> It's the remaining scenarios that are tough:
> 
> * Some joker uses a relative path on the command line (could fix, if we 
> know which arguments are directories)
> 
> * Non-build/non-install commands (could maybe fix by only copying options 
> for build, install, and subcommands thereof)
> 
> * Somebody uses an explicit --install-lib or install_lib --install-dir 
> (fixable, but it could screw things up if they really should've been 
> specifying platlib or purelib or one of the higher-level options)
> 
> * Somebody uses an explicit install --install-headers or install_headers 
> --install-dir (aaagh!)

No matter what you do, there will be people who are simply not good at
packaging software. Where Distutils shines is with a minimal amount of
configuration "setup.py bdist_whatever" will produce a binary usable by
anyone with the whatever package manager, even if the developer created
their setup.py on a completely different platform. This is the real
magic. It's what distinguishes Distutils from CPAN.*

You are taking quite a leap when you talk about supporting multiple
binary packages from a single source package. I started by abstracting a
generic bdist_packager that simply managed the PEP 241 meta-data so
bdist_commands could rely on it for metadata information. Subpackages
was an extension I never got around to.

My point is, _nothing_ you can code will save a bad packager from
themself. The penalty for producing bad packages is that no one uses
them because they're a pain to install. OTOH, Distutils should do
everything it can to make it simple to package a module directories into
multiple inter-dependent components so it takes reall effort to screw up
the packaging.

> There are a few other issues floating about, mostly to do with sys.path, 
> .pth files, and installation to locations other than site-packages.  But I 
> think I'll leave those thoughts to another post.  :)

Personal whine :( I solved these issues on both Solaris and HP-UX by
making bdist_commands that produced packages that were relocatable by
the native package manager based on where the python installation was
installed on the target and any other criteria required. I'm sorry, I
_still_ waiting on my employer's approval to (re)submit them, but I
strongly believe _that's_ the approach to take. Using native package
managers intelligience and pre and post install scripts lets the person
_installing_ the package make these decisions. That person is the REAL
Distutils "customer". Helping developers make better packages is simply
the most effective way to improve the end user (package installer)
experience.

When Distutils generated packages are universally available via apt/yum
repositories, on the sunfreeware.com site and the HP Porting Archive
Center, _that_ will be day I consider Distutils a true success.

*I know... People point to the popularity of CPAN, which has NO
integration with package managers. My point exactly! If you rebuild a
box, your package manager knows everything that needs to be installed
_except_ stuff CPAN brought down. Please, let's not do this with
Distutils....

<end rant, sorry...>

mwa
-- 
Mark W. Alexander
slash at dotnetslash.net

The contents of this message authored by Mark W. Alexander are
released under the Creative Commons Attribution-NonCommercial license.
Copyright of quoted materials are retained by the original author(s).

http://creativecommons.org/licenses/by-nc/1.0/