[Distutils] PEP 426: proposed metadata caching convention

[Nick Coghlan]

On Wed, Feb 27, 2013 at 8:52 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Just to clarify: when I say "hooks", what I mean is "setuptools-style entry
> points that the installer looks for, which are used to customise the installation
> process".

The command to create a wheel from a source archive is currently still
"./setup.py bdist_wheel". This may be executed on an appropriate build
system rather than the target system, but aside from that everything
in setup.py should still execute normally. This is the major
difference between the current attempt and distutils2: du2 made moving
from setup.py to setup.cfg a requirement to generate the new metadata
format. By contrast, I want at least distribute, as well as the Python
3.4 distutils, to be able to generate wheels (including the new
metadata) from current setup.py files.

> I believe it is possible to provide limited extensibility using hooks
> without it leading to the complete ad-hocery that setup.py entails.

For version 1.0, the only install-time modification that all wheel
installers must implement is fanning files out to their target
locations based on sysconfig directories and rewriting script shebang
lines (they may also want to generate parallel Windows executables,
but with the Windows launcher, that's less necessary).

If a project needs more than that, they cannot ship wheels at this
time, and will need to continue shipping source distributions that can
execute arbitrary code at install time. Alternatively (and
preferably), such a project could split out a support library that is
wheel compatible, and have a separate component that must be installed
from source and is able to make arbitrary changes to the target
system.

*Incremental* change, and explicitly leaving some use cases to source
distribution and ./setup.py for the moment is the key to creating a
distribution format that is as simple as we can make it while still
supporting a wide variety of use cases. Will we eventually get
pre-install and post-install hooks ala RPM and other platform specific
systems? Quite possibly. But let's see how far we can get without them
first - in particular, I want to focus people's initial efforts on
putting the smarts into the wheel *creation* process rather than
delaying decisions until install time.

The initial problem I believe we need to solve is the one of arcane
build systems for key dependencies, and the simple fact that most
Windows users aren't equipped to build software written in C in the
first place. Eggs tried to tackle that problem years ago, but ignored
things like the Filesystem Hierarchy Standard and the interests of OS
distributions and system administrators, limiting its adoption to
those developers that were happy with the idea of storing *everything*
inside a single directory (the various legitimate concerns with the
default behaviour of easy_install also didn't help). Wheel is designed
to integrate more cleanly with platform specific conventions,
hopefully overcoming some of those past objections to the egg format.

This preliminary approach also integrates well with centralised system
management tools like Puppet, Chef and Salt - for those, the states
and configurations of services and other components are handled
through the management infrastructure, and the language specific
package management tools are just a way to get the application code
onto the target systems in a controlled fashion.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia