[Distutils] PEP 426 updated (with more than you ever wanted to know about version schemes)

[Daniel Holth]

On Tue, Feb 12, 2013 at 9:31 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 12 February 2013 12:54, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > I've learned a lot about packaging in the last year and a half, mostly
> > through learning how to deal with the SRPM/RPM toolchain after
> > starting with RH. That's actually where I took the
> > Archiver->Builder->Installer model from, except that in RPM the spec
> > file is still the "one metadata file to rule them all".
> [...]
> > We could then take a decent run at creating pydist (aka "pysetup v2")
> > for Python 3.4 (probably under the distlib banner), which would aim to
> > handle the following tasks:
>
> I would very much like to *not* get bogged down in "grand plans" - as
> you say, that way lies failure and burnout.
>
> My assumptions:
>
> 1. We will have to support projects using setup.py for a long time to
> come (probably till Python 4...)
> 2. Setuptools/distribute is, and will remain, external to the core.
> And yet, it offers a short-term way forward (e.g., the wheel project
> already provides a bdist_wheel command under setuptools). And external
> tools like wheel and pip can provide installers. So the external scene
> isn't really a problem.
> 3. Core support is necessary to give any particular new format
> traction and momentum. (That assumption is a bit shaky - eggs imply
> otherwise, but sadly wheel isn't getting much take-up on its own).
> 4. The core is the place for enabling technology/libraries, and
> reference implementations of standards.
> 5. The wheel binary format is a key piece in any new infrastructure
> (it's key for me, anyway)
>
> Based on this, the things I consider important are:
>
> 1. Getting something like distlib into the core, to provide the
> canonical implementation of the key things that packaging tools
> require. [1]
> 2. Providing tools in the core to take an existing, functional,
> distribution (using setup.py) and build a wheel. (This is a
> "political" goal more than anything - distribute/wheel is the
> practical solution right now, but having wheel support in the core
> sends a specific message that it *is* the standard format). I see no
> value in this being spelled anything other than "python setup.py
> bdist_wheel". [2]
> 3. Providing a minimal wheel installer in the core. Same logic. I have
> one already that I would be happy to contribute. [3]
> 4. A PEP376 based uninstaller in the core. Not so much because we need
> an uninstaller, but again simply to assert clearly that the Python
> core supports the PEP. In particular, as an encouragement for external
> tools like distribute and pip to adopt the dist-info format over the
> older egg-info one.
> 5. I hadn't thought of this one myself, but I'd like to see the sdist
> format documented and standardised as Nick suggested. It goes without
> saying that "python setup.py sdist" should produce standard format
> sdists.
>
> [1] Whether those tools consider it worth the effort to remove their
> own code and use the core in its place is not so important, as it
> relates to support of older versions which isn't a core concern - but
> the tools would have to ensure that they *behave* the same as the core
> code.
> [2] Whether the distribute/wheel implementation of bdist_wheel clashes
> with this is something that needs to be considered, but I doubt it'll
> be a big problem in practice.
> [3] It does not depend on the wheel project, although I did copy some
> code from the wheel project, so we'd need Daniel's agreement to use
> it.
>
> Paul
>
> PS Even with all of the above, I don't know if the wheel format will
> "take off". I'd like to see the PyPI infrastructure offer a means of
> enabling people taking the role of "builder", and taking existing
> source packages and uploading binary builds (in wheel format,
> naturally :-)) which can be used by the likes of pip. Everyone
> building their own local archive is wasteful. But that's a whole other
> debate that I have neither the time nor the energy to get involved
> in...
>

Yes you have my permission to put wheel infrastructure in core. Just do a
good job at it :-)

I don't know whether wheel is "taking off" or if it has to. Jannis has
asked me to PEP-ify it in order to get the installer support merged into
pip.  Once that happens I suppose the format will become more popular.

The existing wheel users are getting along fine without a large number of
pre-built wheels. The pip implementation is designed to make it very easy
to cache a virtualenv's worth of packages as wheels, and when you want to
duplicate the virtualenv you don't have to continually re-build.
The format makes new-virtualenv-per-deploy workflows much, much faster and
more predictable.

# Build a directory of wheels for pyramid and all its dependencies
pip wheel --wheel-dir=/tmp/wheelhouse pyramid

# Install from cached wheels
pip install --use-wheel --no-index --find-links=/tmp/wheelhouse pyramid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130212/e92a654b/attachment-0001.html>