On 13Apr2020 2308, André Malo wrote:
For one thing, if you open up APIs for Cython, they're open for everybody (Cython being "just" another C extension). More to the point: The ABIs have the same problem as they have now, regardless how responsive the Cython developers are. Once you compiled the extension, you're using the ABI and are supposedly not required to recompile to stay compatible.
So, where I'm getting at is: Either you open up to everybody or nobody. In C there's not really an in-between.
On a technical level, you are correct.
On a policy level, we don't make changes that would break users of the C API. Because we can't track everyone who's using it, we have to assume that everything is used and any change will cause breakage.
To make sure it's possible to keep developing CPython, we declare parts of the API off limits (typically by prepending them with an underscore). If you use these, and you break, we're sorry but we aren't going to fix it.
This line of discussion is basically saying that we would designate a broader section of the API that is off limits, most likely the parts that are only useful for increased performance (rather than increased functionality). We would then specifically include the Cython team/volunteers in discussions about how to manage changes to these parts of the API to avoid breaking them, and possibly do simultaneous releases to account for changes so that their users have more time to rebuild.
Effectively, when we change our APIs, we would break everyone except Cython because we've worked with them to avoid the breakage. Anyone else using it has to make their own effort to follow CPython development and detect any breakage themselves (just like today).
So probably the part you're missing is where we would give ourselves permission to break more APIs in a release, while simultaneously encouraging people to use Cython as an isolation layer from those breaks.
(Cython is still just a placeholder name here, btw. There are 1-2 other projects that could be considered instead, though I think Cython is the only one that also provides a usability improvement as well as API stability.)