On 22. 02. 22 15:10, Victor Stinner wrote:
Hi,
Thanks for looking into my PEP 674!
I don't understand well why Py_SIZE() cannot be changed until Py_SET_SIZE() is available on all supported Python versions (so affected projects don't have to add a compatibility code), whereas it's ok to require a compatibility code to keep support for Python 3.8 and older for the approved Py_TYPE() change (Py_SET_TYPE was added to Python 3.9).
See the exception for Py_Type, https://github.com/python/steering-council/issues/79#issuecomment-982702725 :
This decision only applies to Py_SET_TYPE
We mean that in general (not for this change as we have made an exception per our previous argument), there should be a deprecation notice in some way or form (in these cases must be in the docs/what's new as there is no way to make the preprocessor warn only when we need it). We just wanted to reiterate that in general, we still want two releases with some deprecation notice for any deprecation, unless exempt.
The SC didn't want to revert the earlier SC's decision. But this PEP lists the dozens of new macros, and no clear reason why the general policy shouldn't apply.
Many projects affected by the Py_SIZE() change are already affected by the Py_TYPE() change, and so need a compatibility code anyway: 33% (5/15) of affected projects are affected by Py_SIZE() and Py_TYPE() changes (see below).
Not changing Py_SIZE() now avoids adding a compatibility code to the 33% (5/15) of affected projects only affected by the Py_SIZE() change.
Either changing Py_TYPE() *and* Py_SIZE(), or none, would make more sense to me. Well, I would prefer to change both, since the work is already done. The change is already part of Python 3.11 and all projects known to be afffected are already fixed. And the Py_TYPE() change was already approved.
IMO it's still possible to change none of them *now* and go through the a deprecation-in-docs period for everything, if you want that.
--
The Py_TYPE() change requires a compatibility code to get Py_SET_TYPE() on Python 3.8 and older, use pythoncapi_compat.h or add:
#if PY_VERSION_HEX < 0x030900A4 # define Py_SET_TYPE(obj, type) ((Py_TYPE(obj) = (type)), (void)0) #endif
The Py_SIZE() change requires a similar compatibility code. For example, boost defines Py_SET_TYPE() *and* Py_SET_SIZE():
#if PY_VERSION_HEX < 0x030900A4 # define Py_SET_TYPE(obj, type) ((Py_TYPE(obj) = (type)), (void)0) # define Py_SET_SIZE(obj, size) ((Py_SIZE(obj) = (size)), (void)0) #endif
Right. But on 3.9+ you don't need any of that, and as since there is no need to do the change *right* now, it can wait until 3.8 becomes history.
--
Affected projects from PEP 674.
Projects affected by Py_SIZE() and Py_TYPE() changes (5):
* guppy3: Py_SET_TYPE(), Py_SET_SIZE(), Py_SET_REFCNT(), use pythoncapi_compat.h * bitarray: Py_SET_TYPE(), Py_SET_SIZE(), use pythoncapi_compat.h * mypy: Py_SET_TYPE(), Py_SET_SIZE(), use pythoncapi_compat.h * numpy: Py_SET_TYPE(), Py_SET_SIZE(), custom compatibility code * boost: Py_SET_TYPE(), Py_SET_SIZE(), custom compatibility code
Projects only affected by the Py_SIZE() change (5):
* python-snappy: Py_SET_SIZE(), use pythoncapi_compat.h * recordclass: use custom py_refcnt() and py_set_size() macros * Cython: Py_SET_SIZE(), Py_SET_REFCNT(), custom compatibility code * immutables: Py_SET_SIZE(), use pythoncapi_compat.h * zstd: Py_SET_SIZE(), use pythoncapi_compat.h
Projects only affected by Py_TYPE() change (5):
* datatable: Py_SET_TYPE(), Py_SET_REFCNT(), use pythoncapi_compat.h * mercurial: Py_SET_TYPE(), use pythoncapi_compat.h * pycurl: Py_SET_TYPE(), custom compatibility code * duplicity: Py_SET_TYPE(), test PY_MAJOR_VERSION and PY_MINOR_VERSION, or use Py_TYPE() as l-value * gobject-introspection: Py_SET_TYPE(), custom compatibility code
These examples don't count the larger number of affected projects using Cython which only need to re-run Cython to use Py_SET_REFCNT(), Py_SET_TYPE() and Py_SET_SIZE().
I would like to add that 100% of the top 5000 PyPI projects are already fixed for PEP 674, but 26 projects need a release including a fix (which will likely happend before Python 3.11 final release).
Thank you for surveying and fixing popular public projects. As you know, there are many more that are not public, popular, nor fixed.