On 9 Jun 2021, at 11:13, Victor Stinner <vstinner@python.org> wrote:

On Wed, Jun 9, 2021 at 10:32 AM Ronald Oussoren via Python-Dev
<python-dev@python.org> wrote:
Its a bit late to complain (and I’m not affected by this myself), but those functions are part of the stable ABI. The change in 3.10 will break any extensions that use the stable ABI, use these functions and don’t use PY_SSIZE_T_CLEAN.  I have no idea how many of those exist, especially given that the stable ABI doesn’t seem to be used a lot.

Require PY_SSIZE_T_CLEAN macro to be defined in an incompatible *API*
change. At the ABI level, what changed if that C extensions built
(with Python 3.9 and older) without PY_SSIZE_T_CLEAN now raise an
error on Python 3.10 (for a few specific argument formats using "#").
Ah you are right, it's an incompatible ABI change.



It might be possible to keep the backward compatibility at the ABI
level by adding a 3rd flavor of "parse" functions:

* parse with size_t: no change
* parse without size_t: stable ABI
* parse without size_t which raises an exception on "#" formats: new
Python 3.10 functions

It's already painful to have 2 flavors of each functions. Adding a 3rd
flavor would make the maintenance burden even worse, whereas Inada-san
wants to opposite (remove the 2nd flavor to only have one!).

I don’t think it is necessary to introduce a 3th variant, for 3.11+ we could do something like this:

* [3.11] Add deprecation warnings in the C headers to the few functions with a “PY_SSIZE_T_CLEAN” variant when “PY_SSIZE_T_CLEAN” is not defined
* [3.12+] Change the headers to behave as if “PY_SSIZE_T_CLEAN” is defined, and only keep the non-PY_SSIZE_T_CLEAN variants for the stable ABI (which would include dropping non-PY_SSIZE_T_CLEAN variants for private functions).  The PY_SSIZE_T_CLEAN variants would keep their "_PysizeT” suffix in the (stable or unstable) ABI. 

This wouldn’t allow dropping the non-PY_SSIZE_T_CLEAN variants entirely, at least not until we’re fine with breaking the stable ABI.  Another disadvantage is that this might require changes in code that doesn’t even use “#” in format strings in 3.11. 

A more general question is: do we still want to keep backward
compatibility with Python 3.2 released 10 years ago, or is it ok to
start with a new stable ABI which drops backward compatibility with
Python 3.5 (for example)?

It's maybe time to replace "abi3" with "abi4" in Python 3.10?

Personally I’d say it is too soon for that, especially when the CPython speedup project (Guido, Mark, et.al.) is just started and HPy is far from finished.  Either project might teach us what changes are needed for a long term stable ABI.


Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/