On Tue, 4 Sep 2018 at 02:15 Paul Moore <p.f.moore@gmail.com> wrote:
On Fri, 31 Aug 2018 at 22:03, Brett Cannon <brett@python.org> wrote:
You can make your pure Python wheel have a py3-cp36m-* wheel name and you get the version specification you want; there is nothing saying a wheel must have extension modules if it specifies an ABI tag.
Currently, that won't work, as pip doesn't consider combinations with py* as the version and an ABI that isn't none as valid:
print("\n".join([f"{p}-{b}-{a}" for p,b,a in pep425tags.get_supported()])) cp37-cp37m-win_amd64 cp37-none-win_amd64 py3-none-win_amd64 cp37-none-any cp3-none-any py37-none-any py3-none-any py36-none-any py35-none-any py34-none-any py33-none-any py32-none-any py31-none-any py30-none-any
If we're talking about what could be made to work, then yes, I guess this is possible. Although it feels like a hack to me
It's a total hack. :) I was just trying to alleviate Xavier's worry about having a directory of wheels where he couldn't choose a specific wheel that was pure Python and tied to a Python version without the interpreter version being specific. Since PyPI doesn't really have projects doing this, moving away from e.g. py35 and to just py3 would still work as you could still say cp35-none-* for the same effect without abusing the ABI tag, but you would have to potentially duplicate the same wheel under different interpreter names if you were supporting both PyPy3 6.0.0 and CPython 3.5.x.
- I consider the ABI to be about the C API, and therefore pure-python code *should* have "none" as the ABI. From PEP 425: "The ABI tag indicates which Python ABI is required by any included extension modules". The semantics of the 3 tags is important, and like it or not they are currently "(interpreter) version", "(extension) ABI" and "(platform) architecture".
The real annoyance are tags that have a sliding scale of support like abi3 and the fact that what Python version an interpreter supports can't be known from a single tag set/triple/whatever-you-call-all-three-tags-together (which comes into play if you tell me you want to download wheels for your cloud host and you want to make it simple for them to tell you what their Python interpreter supports, but I'm slowly giving up on that dream).
I'm not saying we couldn't alter the meanings, but doing so would need an update to the PEP (which is just admin, so not that significant) and a user re-education exercise (which is the non-trivial aspect).
I think the real question is whether you like Nathaniel's clarification suggestion on the tags or if it isn't worth it? -Brett
On Fri, 31 Aug 2018 at 13:33 Xavier Fernandez <xav.fernandez@gmail.com> wrote:
while python-requires is indeed nice, it is only usable by pip to select a wheel file with a fully compliant PEP 503 repository. Otherwise, like in a local wheelhouse (wheel files in a directory), it is invisible to pip until pip tries to install the wheel so a more precise python tag can still be somewhat useful... (I agree that pip could access the wheel metadata to check the python-requires without needing to unzip the whole file).
The point of this is that PEP 503 allows the index to expose Requires-Python as a `data-requires-python` attribute on a file link. So if accessing an index, it's possible to rely on requires-python when selecting what wheels to download or consider further (and pip does that). But when using --find-links to point to a directory full of files (i.e., without a PEP 503 HTML index page) you can't access Requires-Python without opening up the wheel and accessing the metadata - so Requires-Python can't be used to trim the list of potential candidate wheels (or at least, pip doesn't, it is of course theoretically possible).
But more importantly, pip still has to resolve what version of a package to use, so regardless of what's cached in your wheelhouse there will be a check against python-requires somewhere, else you're no worse off than just installing the wheel directly. IOW your wheelhouse isn't going to short-circuit pip's resolver, it's just going to save you from having to download some bits from somewhere else (unless I understand how pip utilizes the wheelhouse on disk).
What you're talking about here is the cache, which is different (and which *is* just about saving downloads).
The details of pip's resolution sequence are not defined by any standard, and they aren't documented in detail by pip either, so we're very much talking about implementation-defined behaviour here. And I haven't checked the source code in detail, so I'm going off memory and assumptions. But basically, pip picks which wheel it's going to install by scanning the filenames (and the data-requires-python tag if it's looking at a PEP 503 index). It doesn't open the wheel and look at the metadata until it's committed to a particular wheel. I *think* that if the chosen package has an incompatible Requires-Python at that point, the resolver simply errors out and doesn't look for alternatives.
The *finder* picks which wheel we're going to use, and does this solely based on the wheel filename (and the data-requires-python tag, for files from a PEP 503 compatible index that supplies it). Once that decision is made, there's no going back - any incompatibilities in metadata after that point simply cause the install to fail. Or in other words, using Requires-Python is not a reliable way to direct pip to choose a particular wheel. That decision is deliberate, for performance reasons. The principle is that installers are able to pick the correct file to install *without* opening up the wheel or otherwise downloading it to check its validity.
Paul.