[Distutils] draft PEP: manylinux2

[Nathaniel Smith]

On Wed, Jan 31, 2018 at 4:01 PM, Mark Williams <mrw at enotuniq.org> wrote:
> Hi everyone!
>
> The manylinux1 platform tag has been tremendously useful, but unfortunately it's showing its age:
>
> https://mail.python.org/pipermail/distutils-sig/2017-April/030360.html
> https://mail.python.org/pipermail/wheel-builders/2016-December/000239.html
>
> Nathaniel identified a list of things to do for its successor, manylinux2:
>
> https://mail.python.org/pipermail/distutils-sig/2017-April/030361.html
>
> Please find below a draft PEP for manylinux2 that attempts to address these issues.  I've also opened a PR against python/peps:
>
> https://github.com/python/peps/pull/565
>
> Docker images for x86_64 (and soon i686) are available to test drive:
>
> https://hub.docker.com/r/markrwilliams/manylinux2/tags/

Huzzah! This is an amazing bit of work, and I'm glad you got that
weird email problem sorted out :-).

I have a few minor comments below, but overall this all looks fine and
sensible to me. Also, I think we should try to move quickly on this if
we can, because the manylinux1 images are currently in the process of
collapsing into unmaintainability. (For example: the openssl that
CentOS 5 ships with is now so old that you can no longer use it to
connect to the openssl web site to download a newer version.)

> 4. If a wheel is built for any version of CPython 2 or CPython
>    versions 3.0 up to and including 3.2, it *must* include a CPython
>    ABI tag indicating its Unicode ABI.  A ``manylinux2`` wheel built
>    against Python 2, then, must include either the ``cpy27mu`` tag
>    indicating it was built against an interpreter with the UCS-4 ABI
>    or the ``cpy27m`` tag indicating an interpeter with the UCS-2
>    ABI. *[Citation for UCS ABI tags?]*

For the citation: maybe PEP 3149? Or just https://github.com/pypa/pip/pull/3075

> Compilation of Compliant Wheels
> ===============================
>
> Like ``manylinux1``, the ``auditwheel`` tool adds ```manylinux2``
> platform tags to ``linux`` wheels built by ``pip wheel`` or
> ``bdist_wheel`` a ``manylinux2`` Docker container.

Missing word: "*in* a"

> Docker Images
> -------------
>
> ``manylinux2`` Docker images based on CentOS 6.9 x86_64 and i686 are
> provided for building binary ``linux`` wheels that can reliably be
> converted to ``manylinux2`` wheels.  [8]_ These images come with a
> full compiler suite installed (``gcc``, ``g++``, and ``gfortran``
> 4.8.2) as well as the latest releases of Python and  ``pip``.

We can and should use newer compiler versions than that, and probably
upgrade them again over the course of the image's lifespan, so let's
just drop the version numbers from the PEP entirely. (Maybe s/6.9/6/
as well for the same reason.)

> Compatibility with kernels that lack ``vsyscall``
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section is maybe not *strictly* necessary in the PEP but I think
we might as well keep it; maybe someone will find it useful.

> Backwards compatibility with ``manylinux1`` wheels
> ==================================================
>
> As explained in PEP 513, the specified symbol versions for
> ``manylinux1`` whitelisted libraries constitute an *upper bound*.  The
> same is true for the symbol versions defined for ``manylinux2`` in
> this PEP.  As a result, ``manylinux1`` wheels are considered
> ``manylinux2`` wheels.  A ``pip`` that recognizes the ``manylinux2``
> platform tag will thus install ``manylinux1`` wheels for
> ``manylinux2`` platforms -- even when explicitly set -- when no
> ``manylinux2`` wheels are available. [20]_

I'm a little confused about what this section is trying to say
(especially the words "even when explicitly set"). Should we maybe
just say something like:

In general, systems that can use manylinux2 wheels can also use
manylinux1 wheels; pip and similar installers should prefer manylinux2
wheels where available.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org