[Distutils] cffi & Py_LIMITED_API

[Nick Coghlan]

On 1 August 2016 at 23:36, Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Aug 1, 2016 at 1:42 AM Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Status quo, on publication side:
>>
>> - require minimum cffi version 1.8
>> - build with setuptools
>> - postprocessing step to rename shared library/DLL
>> - postprocessing step to regenerate renamed whl file with renamed SO/DLL
>
> There wouldn't necessarily need to be renaming.

Aye, that description was for the process if there weren't any changes
to the toolchain.

> build_ext command determines
> the DLL extension. It could be patched or modified to read an "I'm ABI3"
> flag on the Extension() object.
>
> We could pass an ABI3 flag to bdist_wheel in the same way we ask for
> universal 'py2.py3-none-any'. To be set if the wheel contained only ABI3
> extensions, and ignored on py2.

The general idea sounds good to me, but as a slight bikeshed on the
flag name, perhaps "cpabi3"?

That's a mash-up of the 'cp' interpreter code for CPython, with the
'abi3' stable ABI tag.

Longer term, we may want to allow people to version that (as new APIs
may sometimes be added to the stable ABI, which you gain access to at
the C level by setting Py_LIMITED_API to the corresponding CPython hex
version rather than just defining it [1]), but as a starting point
enabling access to the initial 3.2 stable ABI used by cffi should be
sufficient.

>> Of those changes, only the "somehow be Py_LIMITED_API aware" sounds
>> potentially tricky to me, as I'd be surprised if there was any way
>> around requiring a new explicit setting in either setup.py or
>> setup.cfg.
>
>
> Any alternative to an explicit flag that I can think of would be too magical
> to consider, as poor as trying to inspect library symbols for manylinux1
> compat in bdist_wheel itself.

Aye, that was my conclusion as well. Once my brain started going
"Well, we could scan the source for..." I just went "Ewww, no, we can
just ask people to set a new flag" :)

However, would it make sense for the new flag to also implicitly
define Py_LIMITED_API in the compile flags when building the
extension, even if it isn't otherwise specified in the extension's
source code? That way folks would still only need to define their
intent to use the flag in one place, it's just that that place would
be their build instructions rather than the extension module source
code.

Cheers,
Nick.

[1] https://docs.python.org/3/c-api/stable.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia