On 17 Jul 2018, at 5:37 pm, Paul Moore <p.f.moore@gmail.com> wrote:
>
> On 17 July 2018 at 16:59, Cosimo Lupo <cosimo@anthrotype.com> wrote:
>> I would like to revive this 5 year old thread and see if we can stir things
>> up a bit.
>>
>> Basically the problem is that, in the current state of the PEPs and build
>> tools, it is still not possible to build and distribute a Windows wheel that
>> includes an extension module compiled with Py_LIMITED_API.
>> Setuptools can successfully build such extension module on Windows (with
>> `.pyd` file extension and no extra specifiers in the module filename), and
>> these seems to work at least on CPython 3.5 and above. However the
>> `--py-limited-api cpXX` option of bdist_wheel command fails on Windows
>> because it attempts to use the `abi3` tag but the latter is not in the list
>> of compatible tags for that platform.
>> One can work around this by creating a wheel with `none` as the abi tag, and
>> `cp35.cp36.cp37` as the python implementation tag but this feels a bit
>> hackish.
>>
>> Here are some unresolved questions from the old distutils-sig thread,
>> quoting Paul Moore:
>>
>>> 2. How should tools determine which ABIs a given Python supports?
>>> (This is the get_supported function in wheel and distlib). The "Use"
>>> section of the PEP (http://www.python.org/dev/peps/pep-0425/#id1)
>>> gives a Linux-based example, but nothing normative and nothing that is
>>> understandable to a Windows user.
>>
>> And from Vinay Sajip
>>
>>> For Windows, we (eventually) need some low-level sysconfig-supported way
>>> to get the ABI information in an analogous way to how it happens on POSIX:
>>> and
>>> because that's not currently there, distlib doesn't provide any ABI
>>> information
>>> on Windows other than "none".
>>
>> Other related links:
>> https://github.com/pypa/pip/issues/4445
>> https://mail.python.org/pipermail/distutils-sig/2018-January/031856.html
>>
>> So.. what needs to be done here to allow distributing/installing Windows
>> wheels with Py_LIMITED_API support?
>
> IMO, the question I posed back then remains key. Vinay's response is
> fair, but I don't think that waiting for core Python to provide
> something via sysconfig is practical (it's not happened yet, so why
> would we expect things to change?). So I think the next step is
> probably for someone to propose an algorithm that can be used.
> Actually, what I'd like to see is a full end to end proposal of the
> process someone would use to build and install a limited-ABI
> extension. That would probably tease out a number of issues.
>
> I imagine the steps would be something like this:
>
> 1. Create an extension. Presumably you'd need to #define
> PY_LIMITED_ABI in the source.
> 2. Build a wheel from it - how would you do that? I gather it's
> possible to do this with plain setuptools - would it be necessary to
> do this with setuptools/bdist_wheel, or should there be a way to
> request a limited ABI build via pip? If we do want to be able to
> request this from a build tools like pip, do we need something in PEP
> 517? Are we only looking at the prebuilt wheel case, or do we need to
> support building from source?
> 3. What tags would the built wheel have?
> 4. Install that wheel - `pip install xxx`. Pip needs to be able to
> enumerate the full list of valid tags here (cp37-abi3, cp3-abi3, ...)
> There are also questions like - if there's a limited ABI wheel and a
> full ABI (version specific) wheel, which takes precedence?
>
> I don't honestly know how well the limited ABI actually achieves its
> goals - is "cp3-abi3-win_x86_64" a realistic tag to apply? Can limited
> ABI wheels really be used on any version of Python 3? That's a
> question for python-dev, rather than distutils-sig, but if we take the
> position that this is what's promised, and it later turns out not to
> be true, we've got a lot of wheel renaming to do when Python 3.10
> comes out and it doesn't support the same limited ABI as 3.x for x <
> 10...
>
> Also, does the limited ABI work on other platforms? If it does, we
> should ensure that the Windows support works the same. If it doesn't,
> do we want a Windows-only solution (why is that OK?) or should we
> extend to (say) manylinux or OSX (at the risk of making the job too
> big for anyone to actually get anywhere with it).
>
> So to move this forward, I think someone needs to write up the process
> of building and using a limited ABI wheel, as described above, and
> document suggested answers to the various questions that will come out
> in the process of going through the details. Is that something you'd
> be willing to take on?
>
> From that, we'd have something concrete to debate. I'm not sure how
> many people have an interest in this topic, so getting people with the
> necessary knowledge to chime in might take some work (I'm interested,
> but I don't have detail understanding of build options and linking
> conventions). The ultimate goal would be some sort of PEP covering
> handling limited ABI extensions within the packaging infrastructure.
>
> Does that seem reasonable? Is that the sort of guidance you were
> looking for? I doubt anything is going to happen on this subject until
> someone with the interest in moving it forward steps up to do the work
> of making a proposal and collecting community views.

As an example, see...

https://pypi.org/project/PyQt5/#files

These are extension modules that use the limited ABI. As you can see I am anticipating they will also work with Python v3.8. They are created from my own build tools.

Phil
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-leave@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/ZY6SDNMWYNRCZYZSCNWPAKWJFUCKZ6UX/