[Distutils] draft PEP: manylinux2

[Nick Coghlan]

On 6 February 2018 at 16:05, Mark Williams <mrw at twistedmatrix.com> wrote:
> As a counter point: presumably a `manylinux` standard that supports
> those architectures will require a PEP, in which case the author(s)
> will have read the preceding `manylinx` PEPs, either to actively
> borrow as much as possible or to understand arcane but necessary
> details at the behest of this list.  In that case the PEPs' numbers
> will determine the next `manylinux`: `manylinux1` was PEP 513; if it's
> accepted, this will be PEP N > 513; and so a `manylinux` that supports
> ppc64le or aarch64 will be PEP M > N.  This doesn't account for the
> simultaneous proposal of independent PEPs for ppc64le and aarch64, but
> in that case I imagine they'd be merged into a single PEP.

The CalVer idea first came up in the context of skipping ahead in the
numbering sequence to go straight to a baseline that supported ppc64le
and/or aarch64. Even 2014 would likely be too old for that, since
CentOS 7 didn't support those at launch, and neither did Ubuntu 14.04.
While such a PEP hasn't actually been written yet, the kinds of
numbers we were looking at for a suitable baseline year were around
2015 or 2016, as that's when support for them started showing up in
mainline Linux distros.

> Given that `manylinux` PEP numbers determine their sequence number, I
> don't see how CalVer would change the situation.

It lets us deterministically skip numbers if we decide we want to
enable access to things that older platforms just straight up don't
support (like new instruction set architectures).

> A bigger issue is that `manylinux` isn't really one dimensional.  Lots
> of things happened in 2014; for example, IBM shipped the first POWER8
> systems and glibc 2.19 and 2.20 were released.  But RHEL 7 and thus
> CentOS ship glibc 2.17.  Why should `manylinux2014` support ppc64le
> but not glibc 2.19?

Mainly because we aim for "oldest version still used in new releases
that year", but it's also why each version still needs a PEP that maps
out the actual platform ABI as specific library versions.

> Since the definition of `manylinux` depends on the state of RHEL and
> CentOS, maybe we should change the sequence number to match the
> underlying major release of RHEL/CentOS.  That would have `manylinux2`
> become `manylinux6`, and its successor `manylinux7`.  If we require
> that each `manylinux` support all the platforms its RHEL/CentOS
> supports, implementers and users could simply refer to that release to
> know what they're in for.

We discussed that too, and one key reason for not doing it is that we
only build off Red Hat's platform definitions as a matter of
convenience, and because they currently have the longest support
lifecycles.

In the future, we could instead decide that a particular version of
Ubuntu LTS or Debian stable (or even some other LTS distro) was a more
suitable baseline for a given manylinux version, depending on how the
relative timing works out.

For non-RHEL/CentOS users, the RHEL/CentOS version is also just as
arbitrary a sequence number as 1-based indexing.

By contrast, year-based CalVer maintains distro-neutrality, while also
giving a good sense of the maximum age of compatible target platforms.
(e.g. given "manylinux2010", it's a pretty safe guess that Ubuntu
12.04, 14.04 and 16.04 are all expected to be compatible, while that
isn't as clear given "manylinux2" or "manylinux6")

> I think I'm +1 for `manylinux6`, +0 for `manylinux2010`, and -1 for
> `manylinux2`, which now seems to be the worst alternative.
>
>> Would a manylinux2 -> manylinux2010 version numbering switch
>> significantly complicate implementation of the PEP?
>
> Certainly not!  It'll take a little bit more time to adjust
> `auditwheel` and `pip`, but I don't consider that to be onerous.  Once
> we settle on the appropriate versioning scheme I'll be happy to update
> everything!
>
> If I may, a quick question about procedure: do I continue to included
> updates to the PEP in my responses here?  Or do I link to my branch on
> GitHub?

Ah, thanks for the reminder - I'll get your intial PR merged, then we
can track any further changes in the main PEPs repo as additional PRs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia