[Python-Dev] Stable ABI

[Ronald Oussoren]

> On 4 Jun 2018, at 08:35, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> 
> 
> 
>> On 3 Jun 2018, at 17:04, Eric V. Smith <eric at trueblade.com <mailto:eric at trueblade.com>> wrote:
>> 
>> On 6/3/2018 10:55 AM, Christian Tismer wrote:
>>> On 03.06.18 13:18, Ronald Oussoren wrote:
>>>> 
>>>> 
>>>>> On 3 Jun 2018, at 12:03, Christian Tismer <tismer at stackless.com <mailto:tismer at stackless.com>> wrote:
>>> ...
>>>>> 
>>>>> I have written a script that scans all relevant header files
>>>>> and analyses all sections which are reachable in the limited API
>>>>> context.
>>>>> All macros that don't begin with an underscore which contain
>>>>> a "->tp_" string are the locations which will break.
>>>>> 
>>>>> I found exactly 7 locations where this is the case.
>>>>> 
>>>>> My PR will contain the 7 fixes plus the analysis script
>>>>> to go into tools. Preparind that in the evening.
>>>> 
>>>> Having tests would still be nice to detect changes to the stable ABI when they are made.
>>>> 
>>>> Writing those tests is quite some work though, especially if those at least smoke test the limited ABI by compiling snippets the use all symbols that should be exposed by the limited ABI. Writing those tests should be fairly simple for someone that knows how to write C extensions, but is some work.
>>>> 
>>>> Writing a tests that complain when the headers expose symbols that shouldn’t be exposed is harder, due to the need to parse headers (either by hacking something together using regular expressions, or by using tools like gccxml or clang’s C API).
>>> What do you mean?
>>> My script does that with all "tp_*" type fields.
>>> What else would you want to check?
>> 
>> I think Ronald is saying we're trying to answer a few questions:
>> 
>> 1. Did we accidentally drop anything from the stable ABI?
>> 
>> 2. Did we add anything to the stable ABI that we didn't mean to?
>> 
>> 3. (and one of mine): Does the stable ABI already contain things that we don't expect it to?
> 
> That’s correct.  There have been instances of the second item over the year, and not all of them have been caught before releases.  What doesn’t help for all of these is that the stable ABI documentation says that every documented symbol is part of the stable ABI unless there’s explicit documentation to the contrary. This makes researching if functions are intended to be part of the stable ABI harder.
> 
> And also:
> 
> 4. Does the stable ABI actually work?
> 
> Christian’s script finds cases where exposed names don’t actually work when you try to use them.

To reply to myself, the gist below is a very crude version of what I was trying to suggest:

https://gist.github.com/ronaldoussoren/fe4f80351a7ee72c245025df7b2ef1ed#file-gistfile1-txt <https://gist.github.com/ronaldoussoren/fe4f80351a7ee72c245025df7b2ef1ed#file-gistfile1-txt>

The gist is far from usable, but shows some tests that check that symbols in the stable ABI can be used, and tests that everything exported in the stable ABI is actually tested. 

Again, the code in the gist is a crude hack and I have currently no plans to turn this into something that could be added to the testsuite.

Ronald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180604/ab0d7067/attachment.html>