[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Paul Moore]

On 7 October 2015 at 20:36, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> Currently I can take the code from the numpy release and compile it in
> different incompatible ways. For example I could make a wheel that bundles a
> BLAS library. Or I could make a wheel that expects to use a system BLAS
> library that should be installed separately somehow or I could build a wheel
> against pyopenblas and make a wheel that depends on pyopenblas. Or I could
> link a BLAS library statically into numpy.
>
> A numpy release supports being compiled and linked in many different ways
> and will continue to do so regardless of any decisions made by PYPA. What
> that means is that there is not a one to one correspondence between a numpy
> release and a binary wheel. If there must be a one to one correspondence
> between a source wheel and a binary wheel then it follows that there cannot
> be a one to one correspondence between the source release and a source
> wheel.
>
> Of course numpy could say that they will only upload one particular source
> wheel and binary wheel to PyPI but people need to be able to use the source
> release in many different ways. So only releasing a source wheel that maps
> one to one to a particular way of compiling numpy is not an acceptable way
> for numpy to release its code.

The disconnect here seems to be that I view all of those wheels as
being numpy 1.9.X wheels (or whatever). They differ in terms of
compatibility details, but they are all wheels for the same
project/version. So there's no problem with them all being built from
the same source wheel. I also have no problem with it being possible
to configure the build differently from a single source wheel, to
generate all those wheels. The configuration isn't metadata, it's
"just" settings for the build.

Of course, there *is* an unsolved issue here, which is how we manage
compatibility for wheels at the level needed for numpy. But I thought
the discussion on that was ongoing? I'm concerned that this proposal
is actually about bypassing that discussion, and instead trying to
treat incompatibly linked wheels as "different" in terms of project
metadata, which I think is the wrong way of handling things. I note
that Christoph Gohlke's numpy builds are tagged with a "+mkl" local
version modifier - that's presumably intended to mark the fact that
they are built with an incompatible runtime - but that's a misuse of
local versions (and I've found it causes niggling issues with how pip
recognises upgrades, etc).

So, in summary: Your points above don't seem to me to in any way
preclude having a single numpy source wheel, and a number of (mutually
incompatible, but the same in terms of project and version) binary
wheels.

Paul