[Distutils] Working toward Linux wheel support

[Nate Coraor]

On Mon, Aug 24, 2015 at 1:51 PM, Wes Turner <wes.turner at gmail.com> wrote:

>
>
> On Mon, Aug 24, 2015 at 10:03 AM, Nate Coraor <nate at bx.psu.edu> wrote:
>
>> On Fri, Aug 21, 2015 at 2:51 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>>> On 21 August 2015 at 05:58, Robert Collins <robertc at robertcollins.net>
>>> wrote:
>>> > On 21 August 2015 at 07:25, Donald Stufft <donald at stufft.io> wrote:
>>> >>
>>> >> On August 20, 2015 at 3:23:09 PM, Daniel Holth (dholth at gmail.com)
>>> wrote:
>>> >>> If you need that for some reason just put the longer information in
>>> the
>>> >>> metadata, inside the WHEEL file for example. Surely "does it work on
>>> my
>>> >>> system" dominates, as opposed to "I have a wheel with this mnemonic
>>> tag,
>>> >>> now let me install debian 5 so I can get it to run".
>>> >>>
>>> >>>
>>> >>
>>> >> It’s less about “now let me install Debian 5” and more like tooling
>>> that doesn’t run *on* the platform but which needs to make decisions based
>>> on what platform a wheel is built for.
>>> >
>>> > Cramming that into the file name is a mistake IMO.
>>> >
>>> > Make it declarative data, make it indexable, and index it. We can do
>>> > that locally as much as via the REST API.
>>> >
>>> > That btw is why the draft for referencing external dependencies
>>> > specifies file names (because file names give an ABI in the context of
>>> > a platform) - but we do need to identify the platform, and
>>> > platform.distribution should be good enough for that (or perhaps we
>>> > start depending on lsb-release for detection
>>>
>>> LSB has too much stuff in it, so most distros aren't LSB compliant out
>>> of the box - you have to install extra packages.
>>>
>>> /etc/os-release is a better option:
>>> http://www.freedesktop.org/software/systemd/man/os-release.html
>>
>>
>> As per this discussion, and because I've discovered that the entire
>> platform module is deprecated in 3.5 (and other amusements, like a
>> Ubuntu-modified version of platform that ships on Ubuntu - platform as
>> shipped with CPython detects Ubuntu as debian), I'm switching to
>> os-release, but even that is unreliable - the file does not exist in
>> CentOS/RHEL 6, for example. On Debian testing/sid installs, VERSION and
>> VERSION_ID are unset (which is not wrong - there is no release of testing,
>> but it does make identifying the platform more complicated since even the
>> codename is not provided other than at the end of PRETTY_NAME). Regardless
>> of whether a hash or a human-identifiable string is used to identify the
>> platform, there still needs to be a way to reliably detect it.
>>
>> Unless someone tells me not to, I'm going to default to using os-release
>> and then fall back to other methods in the event that os-release isn't
>> available, and this will be in some sort of library alongside pep425tags in
>> wheel/pip.
>>
>> FWIW, os-release's `ID_LIKE` gives us some ability to make assumptions
>> without explicit need for a binary-compatibility.cfg (although not blindly
>> - for example, CentOS sets this to "rhel fedora", but of course RHEL/CentOS
>> and Fedora versions are not congruent).
>>
>
> IIUC, then the value of os-release
> will be used to generalize
> the compatible versions of *.so deps
> of a given distribution at a point in time?
>
> This works for distros that don't change [libc] much during a release,
> but for rolling release models (e.g. arch, gentoo),
> IDK how this simplification will work.
> (This is a graph with nodes and edges (with attributes), and rules).
>

Arch, Gentoo, and other rolling release distributions don't have a stable
ABI, so by definition I don't think we can support redistributable wheels
on them. I'm adding platform detection support for them regardless, but I
don't think there's any way to allow wheels built for these platforms in
PyPI.


> * Keying/namespacing is a simplification which may work.
>   * *conda preprocessing selectors* (and ~LSB-Python-Conda)
>      ~'prune' large parts of the graph
>
> * Someone mentioned LSB[-Python-Base] (again as a simplification)
>   * [[package, [version<=>verstr]]]
>
> Salt
> * __salt__['grains']['os']  = "Fedora" || "Ubuntu"
> * __salt__['grains']['os_family'] = "RedHat" || "Debian"
> * __salt__['grains']['osrelease'] = "22" || "14.04"
> * __salt__['grains']['oscodename'] = "Twenty Two" || "trusty"
> * Docs: http://docs.saltstack.com/en/latest/topics/targeting/grains.html
> * Docs:
> http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.grains.html#salt.modules.grains.get
> * Src:
> https://github.com/saltstack/salt/blob/develop/salt/grains/core.py#L1018
> ("def os_data()")
>
> $ sudo salt-call --local grains.item os_family os osrelease oscodename
> local:
>     ----------
>     os:
>         Fedora
>     os_family:
>         RedHat
>     oscodename:
>         Twenty Two
>     osrelease:
>         22
>
>
>
>> --nate
>>
>>
>>>
>>>
>>> My original concern with using that was that it *over*specifies the
>>> distro (e.g. not only do CentOS and RHEL releases show up as different
>>> platforms, but so do X.Y releases within a series), but the
>>> binary-compatibility.txt idea resolves that issue, since a derived
>>> distro can explicitly identify itself as binary compatible with its
>>> upstream and be able to use the corresponding wheel files.
>>>
>>> Regards,
>>> Nick.
>>>
>>> --
>>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150824/e8bed6ff/attachment.html>