[Distutils] Should abi3 tag be usable on Windows?

[Cosimo Lupo]

Thank you, Steve.

Ok so, setuptools can successfully build extension modules with
Py_LIMITED_API on Windows, with a generic *.pyd suffix and no extra tags.

However, there are still two problems before I can distribute them:

1) it's not clear how wheels containing such extension modules should be
tagged on Windows, given that `abi3` tag is not recognized for that
platform. The function imp.get_suffixes() used by both pip and wheel to get
the list of supported tags does not return any abi3 suffixes on Windows,
and for this reason the bdist_wheel command assumes from this that Windows
does not support extensions built with Py_LIMITED_API, and gives an error.
Some have resorted to tagging their Windows wheels with abi "none" to work
around this, however "none" seems to imply that extension does not use the
Python ABI which is not true.
Alex Grönholm (current maintainer of pypa/wheel package) pointed me to this
other related discussion on distutils-sig (from 2013) where Paul Moore
explains what the problem is:
https://mail.python.org/pipermail/distutils-sig/2013-February/020022.html
Has anything happened since the to address this either in the PEPs and/or
the tooling?

2) the current virtualenv module (not the built-in venv) does not copy over
the required PYTHON3.DLL and thus importing such extension modules from
within the virtual environment fails. The latter issue was already fixed in
virtualenv's master branch, but virtualenv package hasn't seen a release
since then (the last release 15.10.0 was in November 2016).
https://github.com/pypa/virtualenv/pull/1083

I would love to take advantage of CPython limited API feature, and be able
to distribute a one/two wheels per platform instead of four (multiply that
by two for 32 vs 64 bit arch), and having to rebuild one for each new
Python 3.x release.

It works fine for Linux and macOS; I hope that Windows python users will
also be able to enjoy this soon.
I am willing to contribute my time to help solving this issue.

Thanks,

Cosimo

On Mon, Jan 8, 2018 at 1:31 AM Steve Dower <steve.dower at python.org> wrote:

> There is a solution to that problem on the linked issue. Basically, you
> need to declare Py_LIMITED_API in your code or as an extra preprocessor
> variable.
>
>
>
> Windows doesn’t use a filename suffix for python3.dll linked extensions as
> it will be handled at load time. The tag for the wheel is outside of my
> area, so hopefully someone can chime in on that for you.
>
>
>
> Cheers,
>
> Steve
>
>
>
> Top-posted from my Windows phone
>
>
>
> *From: *Cosimo Lupo <cosimo at anthrotype.com>
> *Sent: *Monday, January 8, 2018 10:32
> *To: *distutils-sig at python.org
> *Subject: *Re: [Distutils] Should abi3 tag be usable on Windows?
>
>
>
> It turns out setuptools is still linking to PYTHON36.DLL instead of
> PYTHNO3.DLL even when py_limited_api=True
>
> https://github.com/pypa/setuptools/issues/1248
>
>
>
> This is a different, though somewhat related, problem from the one
> concering wheel/pip pep425tags disallowing `abi3` tag on Windows.
>
>
>
>
>
> On Sun, Jan 7, 2018 at 12:39 PM Cosimo Lupo <cosimo at anthrotype.com> wrote:
>
> I jut found this related pip issue:
> https://github.com/pypa/pip/issues/4445
>
>
>
> (I myself as @anthrotype commented on that thread back in April last year
> but then completely forgot...)
>
>
>
> After re-reading it now, it's still not clear to me what the resolution on
> that issue was.
>
>
>
> On Sun, Jan 7, 2018 at 10:12 AM Cosimo Lupo <cosimo at anthrotype.com> wrote:
>
> Hello,
>
> CFFI has recently added support for Py_LIMITED_API extension modules built
> for CPython 3.
> The wheel module since version 0.30.0 also supports passing
> —py-limited-api cp3X to bdist_wheel command to allow the generated .whl to
> be installed on all CPython versions equal or greater than the one
> specified.
>
> Yesterday I was trying to apply this on a cffi-built extension module, and
> it worked for Linux and macOS but failed for Windows:
>
> https://github.com/hynek/argon2_cffi/pull/32
> <https://github.com/hynek/argon2_cffi/pull/32#issuecomment-355771832>
> The AssertionError from wheel.pep425tags complains that a tag with abi3
> would be unsupported for the target platform.
>
> Alex Gronholm commented
>
> > imp.get_suffixes() does not seem to contain any ABI3 suffixes, but I'm
> not sure if this is even applicable on Windows.
>
> Incidentally, I noticed one specific package, PyQt5, that distributes both
> abi3-tagged wheels for Mac and manylinux and Windows wheels for a range of
> cp35.cp36.cp37 but with abi tag set as “none”, and they do seem to work.
>
> So, can one make such py_limited_api wheels work on Windows with the
> current state of the tooling, and if so how?
>
> Thank you in advance
>
>
>
> Cosimo Lupo
>
>
>
> --
>
> Cosimo Lupo
>
>
>
> --
>
> Cosimo Lupo
>
>
>


-- 
Cosimo Lupo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20180113/41f41647/attachment.html>