Re: [Distutils] Should abi3 tag be usable on Windows?

Thank you, Steve. Ok so, setuptools can successfully build extension modules with Py_LIMITED_API on Windows, with a generic *.pyd suffix and no extra tags. However, there are still two problems before I can distribute them: 1) it's not clear how wheels containing such extension modules should be tagged on Windows, given that `abi3` tag is not recognized for that platform. The function imp.get_suffixes() used by both pip and wheel to get the list of supported tags does not return any abi3 suffixes on Windows, and for this reason the bdist_wheel command assumes from this that Windows does not support extensions built with Py_LIMITED_API, and gives an error. Some have resorted to tagging their Windows wheels with abi "none" to work around this, however "none" seems to imply that extension does not use the Python ABI which is not true. Alex Grönholm (current maintainer of pypa/wheel package) pointed me to this other related discussion on distutils-sig (from 2013) where Paul Moore explains what the problem is: https://mail.python.org/pipermail/distutils-sig/2013-February/020022.html Has anything happened since the to address this either in the PEPs and/or the tooling? 2) the current virtualenv module (not the built-in venv) does not copy over the required PYTHON3.DLL and thus importing such extension modules from within the virtual environment fails. The latter issue was already fixed in virtualenv's master branch, but virtualenv package hasn't seen a release since then (the last release 15.10.0 was in November 2016). https://github.com/pypa/virtualenv/pull/1083 I would love to take advantage of CPython limited API feature, and be able to distribute a one/two wheels per platform instead of four (multiply that by two for 32 vs 64 bit arch), and having to rebuild one for each new Python 3.x release. It works fine for Linux and macOS; I hope that Windows python users will also be able to enjoy this soon. I am willing to contribute my time to help solving this issue. Thanks, Cosimo On Mon, Jan 8, 2018 at 1:31 AM Steve Dower wrote:

There is a solution to that problem on the linked issue. Basically, you need to declare Py_LIMITED_API in your code or as an extra preprocessor variable.

Windows doesn’t use a filename suffix for python3.dll linked extensions as it will be handled at load time. The tag for the wheel is outside of my area, so hopefully someone can chime in on that for you.

Cheers,

Steve

Top-posted from my Windows phone

*From: *Cosimo Lupo *Sent: *Monday, January 8, 2018 10:32 *To: *distutils-sig@python.org *Subject: *Re: [Distutils] Should abi3 tag be usable on Windows?

It turns out setuptools is still linking to PYTHON36.DLL instead of PYTHNO3.DLL even when py_limited_api=True

https://github.com/pypa/setuptools/issues/1248

This is a different, though somewhat related, problem from the one concering wheel/pip pep425tags disallowing `abi3` tag on Windows.

On Sun, Jan 7, 2018 at 12:39 PM Cosimo Lupo wrote:

I jut found this related pip issue: https://github.com/pypa/pip/issues/4445

(I myself as @anthrotype commented on that thread back in April last year but then completely forgot...)

After re-reading it now, it's still not clear to me what the resolution on that issue was.

On Sun, Jan 7, 2018 at 10:12 AM Cosimo Lupo wrote:

Hello,

CFFI has recently added support for Py_LIMITED_API extension modules built for CPython 3. The wheel module since version 0.30.0 also supports passing —py-limited-api cp3X to bdist_wheel command to allow the generated .whl to be installed on all CPython versions equal or greater than the one specified.

Yesterday I was trying to apply this on a cffi-built extension module, and it worked for Linux and macOS but failed for Windows:

https://github.com/hynek/argon2_cffi/pull/32 https://github.com/hynek/argon2_cffi/pull/32#issuecomment-355771832 The AssertionError from wheel.pep425tags complains that a tag with abi3 would be unsupported for the target platform.

Alex Gronholm commented

...

imp.get_suffixes() does not seem to contain any ABI3 suffixes, but I'm not sure if this is even applicable on Windows.

Incidentally, I noticed one specific package, PyQt5, that distributes both abi3-tagged wheels for Mac and manylinux and Windows wheels for a range of cp35.cp36.cp37 but with abi tag set as “none”, and they do seem to work.

So, can one make such py_limited_api wheels work on Windows with the current state of the tooling, and if so how?

Thank you in advance

Cosimo Lupo

--

Cosimo Lupo

--

Cosimo Lupo

-- Cosimo Lupo