[Distutils] Working toward Linux wheel support

[Wes Turner]

On Jul 28, 2015 10:02 AM, "Oscar Benjamin" <oscar.j.benjamin at gmail.com>
wrote:
>
> On Fri, 24 Jul 2015 at 19:53 Chris Barker <chris.barker at noaa.gov> wrote:
>>
>> On Tue, Jul 21, 2015 at 9:38 AM, Oscar Benjamin <
oscar.j.benjamin at gmail.com> wrote:
>>>
>>>
>>> I think it would be great to just package these up as wheels and put
them on PyPI.
>>
>>
>> that's the point -- there is no way with the current spec to specify a
wheel dependency as opposed to a package dependency. i.e this particular
binary numpy wheel depends on this other wheel, whereas the numpy source
pacakge does not have that dependency -- and, indeed, a wheel for one
platform may have different dependencies that\n other platforms.
>
>
> I thought it was possible to do this with wheels. It's already possible
to have wheels or sdists whose dependencies vary by platform I thought.
>
> The BLAS dependency is different. In particular the sdist is compatible
with more cases than a wheel would be so the built wheel would have a more
precise requirement than the sdist. Is that not possible with
pip/wheels/PyPI or is that a limitation of using setuptools to build the
wheel?
>
>>>
>>> So numpy could depend on "blas" and there could be a few different
distributions on PyPI that provide "blas" representing the different
underlying libraries. If I want to install numpy with a particular one I
can just do:
>>>
>>>     pip install gotoblas  # Installs the BLAS library within Python dirs
>>>     pip install numpy
>>
>>
>> well,different implementations of BLAS are theoretically ABI compatible,
but as I understand it, it's not actually that simple, so this is
particularly challenging.
>>
>>
>> But if it were, this would be a particular trick, because then that
numpy wheel would depend on _some_ BLAS wheel, but there may be more than
one option -- how would you express that????
>
>
> I imagined having numpy Require "blas OR openblas". Then openblas package
Provides "blas". Any other BLAS library also provides "blas". If you do
"pip install numpy" and "blas" is already provided then the numpy wheel
installs fine. Otherwise it falls back to installing openblas.
>
> Potentially "blas" is not specific enough so the label could be
"blas-gfortran" to express the ABI.

BLAS may not be the best example, but should we expect such linked
interfaces to change over time? (And e.g. be versioned dependencies with
shim packages that have check functions)?

... How is an ABI constraint different from a package dependency?

iiuc, ABI tags are thus combinatorial with package/wheel dependency strings?

Conda/pycosat solve this with "preprocessing selectors" :
http://conda.pydata.org/docs/building/meta-yaml.html#preprocessing-selectors
:

```
linux True if the platform is Linux
linux32 True if the platform is Linux and the Python architecture is 32-bit
linux64 True if the platform is Linux and the Python architecture is 64-bit
armv6 True if the platform is Linux and the Python architecture is armv6l
osx True if the platform is OS X
unix True if the platform is Unix (OS X or Linux)
win True if the platform is Windows
win32 True if the platform is Windows and the Python architecture is 32-bit
win64 True if the platform is Windows and the Python architecture is 64-bit
py The Python version as a two digit string (like '27'). See also the
CONDA_PY environment variable below.
py3k True if the Python major version is 3
py2k True if the Python major version is 2
py26 True if the Python version is 2.6
py27 True if the Python version is 2.7
py33 True if the Python version is 3.3
py34 True if the Python version is 3.4
np The NumPy version as a two digit string (like '17'). See also the
CONDA_NPY environment variable below.
Because the selector is any valid Python expression, complicated logic is
possible.
```

>
> --
> Oscar
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150728/9a32d9d5/attachment-0001.html>