[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Paul Moore]

On Tue, 13 Nov 2018 at 21:02, Victor Stinner <vstinner at redhat.com> wrote:

> My plan is to select something like the top five most popular C
> extensions based on PyPI download statistics. I cannot test
> everything, I have to put practical limits.

You should probably also consider embedding applications - these have
the potential to be adversely affected too. One example would be vim,
which embeds Python, and makes fairly heavy use of the API (in some
relatively nonstandard ways, for better or worse).

Paul

PS What percentage does "top 5" translate to? In terms of both
downloads and actual numbers of extensions? With only 5, it would be
very easy (I suspect) to get only scientific packages, and (for
example) miss out totally on database APIs, or web helpers. You'll
likely get a broader sense of where issues lie if you cover a wide
range of application domains.

PPS I'd like to see a summary of your backward compatibility plan.
I've not been following this thread so maybe I missed it (if so, a
pointer would be appreciated), but I'd expect as a user that
extensions and embedding applications would *not* need a major rewrite
to work with Python 3.8 - that being the implication of "opt in". I'd
also expect that to remain true for any future version of Python -
assuming the experiment is successful, forced (as opposed to opt-in)
migration to the new API would be handled in a gradual, backward
compatibility respecting manner, exactly as any other changes to the C
API are. A hard break like Python 3, even if limited to the C API,
would be bad for users (for example, preventing adoption of Python 3.X
until the scientific stack migrates to the new API and works out how
to handle supporting old-API versions of Python...)