[Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Paul Moore]

On 12 February 2013 12:54, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I've learned a lot about packaging in the last year and a half, mostly
> through learning how to deal with the SRPM/RPM toolchain after
> starting with RH. That's actually where I took the
> Archiver->Builder->Installer model from, except that in RPM the spec
> file is still the "one metadata file to rule them all".
[...]
> We could then take a decent run at creating pydist (aka "pysetup v2")
> for Python 3.4 (probably under the distlib banner), which would aim to
> handle the following tasks:

I would very much like to *not* get bogged down in "grand plans" - as
you say, that way lies failure and burnout.

My assumptions:

1. We will have to support projects using setup.py for a long time to
come (probably till Python 4...)
2. Setuptools/distribute is, and will remain, external to the core.
And yet, it offers a short-term way forward (e.g., the wheel project
already provides a bdist_wheel command under setuptools). And external
tools like wheel and pip can provide installers. So the external scene
isn't really a problem.
3. Core support is necessary to give any particular new format
traction and momentum. (That assumption is a bit shaky - eggs imply
otherwise, but sadly wheel isn't getting much take-up on its own).
4. The core is the place for enabling technology/libraries, and
reference implementations of standards.
5. The wheel binary format is a key piece in any new infrastructure
(it's key for me, anyway)

Based on this, the things I consider important are:

1. Getting something like distlib into the core, to provide the
canonical implementation of the key things that packaging tools
require. [1]
2. Providing tools in the core to take an existing, functional,
distribution (using setup.py) and build a wheel. (This is a
"political" goal more than anything - distribute/wheel is the
practical solution right now, but having wheel support in the core
sends a specific message that it *is* the standard format). I see no
value in this being spelled anything other than "python setup.py
bdist_wheel". [2]
3. Providing a minimal wheel installer in the core. Same logic. I have
one already that I would be happy to contribute. [3]
4. A PEP376 based uninstaller in the core. Not so much because we need
an uninstaller, but again simply to assert clearly that the Python
core supports the PEP. In particular, as an encouragement for external
tools like distribute and pip to adopt the dist-info format over the
older egg-info one.
5. I hadn't thought of this one myself, but I'd like to see the sdist
format documented and standardised as Nick suggested. It goes without
saying that "python setup.py sdist" should produce standard format
sdists.

[1] Whether those tools consider it worth the effort to remove their
own code and use the core in its place is not so important, as it
relates to support of older versions which isn't a core concern - but
the tools would have to ensure that they *behave* the same as the core
code.
[2] Whether the distribute/wheel implementation of bdist_wheel clashes
with this is something that needs to be considered, but I doubt it'll
be a big problem in practice.
[3] It does not depend on the wheel project, although I did copy some
code from the wheel project, so we'd need Daniel's agreement to use
it.

Paul

PS Even with all of the above, I don't know if the wheel format will
"take off". I'd like to see the PyPI infrastructure offer a means of
enabling people taking the role of "builder", and taking existing
source packages and uploading binary builds (in wheel format,
naturally :-)) which can be used by the likes of pip. Everyone
building their own local archive is wasteful. But that's a whole other
debate that I have neither the time nor the energy to get involved
in...