On Thu, Sep 9, 2021 at 5:05 PM Marc-Andre Lemburg <mal@egenix.com> wrote:
Since this ML is not really on the radar of many extension writers, how about creating a C API change announcement list where breaking changes are posted or, even better:
Analyze the top PyPI packages, identify the ones which would break and send the maintainers an email explaining the change.
Sure, I did that whenever I could.
First, my conclusion: it's kind of sad that there is no "reliable" and "automated" way to rebuild *all* PyPI projects to count how many projects would be broken by an incompatible change, when working on a change. I do this work manually.
Because this task is manual, I'm more in favor of doing a quick estimate of the risk first, then push the change, and consider reverting if too many problems are discovered during the development phase.
I know that it's not perfect.
I also know that some people consider that the C API is perfect and that it must not be changed :-) But there are *reasons* to "fix" it :-)
--
I copied Inada-san tool to download PyPI top (5000) projects: https://github.com/vstinner/misc/blob/main/cpython/download_pypi_top.py
You can then use tools like "rg" to grep regex patterns directly in tarballs and ZIP archives. See Inada-san notes about that: https://github.com/methane/notes/tree/master/2020/wchar-cache
I usually use it as an estimation on how many projects will be impacted outside the PyPI most popular projects.
--
Moreover, at Red Hat, we are rebuilding the whole Fedora operating system (technically, we try to only rebuild Python packages) with newer Python during alpha and then beta releases.
We are usually the first ones to detect Python packages impacted by incompatible changes. That's how we decided to revert a few incompatible changes in Python 3.9 for example: https://lwn.net/Articles/811369/
That's also how we decided to revert the Py_TYPE() change in Python 3.10.
--
There is also the code search on GitHub, but it's not easy to skip the big number of copies of the CPython source code in results.
Victor
Night gathers, and now my watch begins. It shall not end until my death.