On Thu, Aug 30, 2018 at 6:52 PM, Brett Cannon
On Thu, 30 Aug 2018 at 11:21 Nathaniel Smith
wrote: If we're going to rethink this,
Well, I didn't want to "rethink" so much as "fill in". :)
then I would really like to move away from assigning special meaning to specific combinations of tags. The thing where if you use cp36m as you Python ABI tag then you're forced to use cp36 as your python dialect tag doesn't make sense. And the thing where if you have a wheel with an extension module tagged as cp36-abi3, then that works fine on 3.7,
I wouldn't expect that to if the stable ABI was expanded in Python 3.7 and you used the expanded part.
In my example, you have a wheel using the 3.6 ABI, running on 3.7. That's case that's supposed to work :-).
but if you *remove* the extension module then it *stops* working on 3.7? That's just bizarre...
I don't quite follow what you mean by "remove the extension module then it stops". What stops by removing the extension module(s)?
If I'm reading your proposal right, it says that when running on Python 3.7, pip should be willing to install wheels tagged cp36-abi3, but not wheels tagged cp36-none. But conceptually, if you take the extension modules out of a cp36-abi3 wheel, then you're left with a cp36-none wheel. So this is weird. Anywhere we're willing to install a cp36-abi3 wheel, we should also be willing to install a cp36-none wheel.
So my suggestions:
* Make the 3 tag categories totally independent. Compute a separate set for each, and then take the full cross product.
I think Paul was hinting at this as part of his "wildcard" idea (and I honestly thought of this initially as well as it greatly simplifies things). So what would cp36-cp36m-plat expand to?
I don't know what it means to "expand" a wheel tag. Are you punning the tag as also describing a Python installation ("CPython 3.6, with a certain soabi tag and platform"), and then asking to find all the wheel tags that could go in that installation? You can't actually do this in general – for example, determining whether a target interpreter is compatible with manylinux wheels requires running some special sniffing code on that interpreter.
cp36: cp3N where N is any positive digit(s)? And then toss in py3 and py3N? Prefer exact, then generic '3', then older, and finally newer?
Given that we don't have any real use cases for cpXY and pyXY, I'd rather not expand the options – just preserve what we do now... so for CPython 3.6, I'd say py3, py3N for N <= 6, cp3N for N <= 6, and I don't really care about the exact order – some sort of more-specific-before-less-specific makes sense, and whatever we do now is probably fine. If someone goes wild and starts distributing py35 and py36 wheels for the same package (via a "36to35" tool, I guess?) then that's probably what you want? But I don't imagine this will ever be an important use case. We tried it with 2to3, and everyone decided they'd rather write in the subset language for a decade instead.
cp36m: that, abi3, and then 'none'? Do we care if someone has some crazy ABI that no one understands like 'b' (and if so should it only be applied to 'py' interpreter versions which break your nice cross product simplicity)? plat: depends on platform.
Do you mean, what happens if we find ourselves running on an interpreter we don't recognize (not cpython/pypy/jython/...), and that interpreter returns something wacky from sysconfig.get_config_var("SOABI")? I think there's a reasonable argument that in that case we should only accept 'none' as the tag. (And then maybe whoever invented this new interpreter gets the job of adding sysconfig.get_supported_abi_tags() as a standard stdlib feature :-).)
And the match preference goes to platform, interpreter version, and then ABI? I think that would work in terms of ignoring C ABIs that have very little chance of linking while being the broadest in terms of accepting a wheel that has some semblance of a chance of running.
I don't think the preference order matters that much as long as its well-defined. The only cases where it would affect things are like... Suppose in the future we add support for platform tags based on different ISA levels, like x86_64_avx2 vs x86_64_sse3 vs x86_64. Then you could find yourself in a situation where examplepkg v1.2.3 has the following wheels available: cp36-cp36m-manylinux1_x86_64 cp36-abi3-manylinux1_x86_64_avx2 Our environment is CPython 3.6, running on a manylinux1-compatible OS and we do have AVX2 support available, so either of these wheels could work. The first wheel has a "better" ABI (cp36m > abi3), but a "worse" platform (x86_64 < x86_64_avx2). So which one should we pick? I have zero intuition here. Whoever compiled these wheels is clearly perverse, and they should stop doing silly things like this.
And how would that apply to PyPy3? Same ranging on 'pp36N` and drop the 'abi3' insertion?
* Since the stable ABI actually changes over time, we should define new tags abi35, abi36, etc. that mean "requires the stable ABI as defined by this version of cpython or higher", instead of relying on abi3 + a dialect tag. (Imagine if pypy started implementing the stable ABI –we'd have to start allowing cpXY tags to match PyPy.)
* Plan to move away from the pyXY and cpXY tags over time; they're confusing and not useful. Of course this will have to be a gradual process, but if pip stops requiring them now then in a year or two we could make setuptools stop generating then.
What would you do then about preferred match order for pure Python wheels? E.g. how do you preferably match against Python 3.7 wheels over 3.6 when running a Python 3.7 interpreter? Or are you suggesting equivalent ABI tags to make up for the pyXY tags and the interpreter tag simply gets ignored?
I don't understand your questions, sorry :-(. I'm just saying: we should make the stable-ABI tags self-contained, so that 'py3-abi36' would be sufficient to express "uses the stable abi, as it existed on python 3.6". This would match the same set of interpreters that that 'cp36-abi3' is currently used to target. And also, we should try to gradually decrease the usage of pyXY and cpXY tags (e.g. eventually setuptools should stop generating them by default). -n -- Nathaniel J. Smith -- https://vorpus.org