On Thu, Sep 9, 2021 at 12:30 PM Victor Stinner vstinner@python.org wrote:
We cannot evolve Python (fix bugs, optimize it) without this annoying work of fixing the C API: add more abstractions on top of C structures.
I agree with that, but this argument works for *any* breaking change. For each change there should be an independent cost/benefit analysis, and this argument isn't helping there. I don't think "evolve Python" is specific enough to be a goal.
On 09. 09. 21 16:18, Antonio Cuni wrote:
I agree with this and I think that it's the core of the problem. Currently, the C API leaks too many implementation details and makes it almost impossible to change anything in CPython.
The first thing to acknowledge is that CPython is stuck with design decisions which were taken in 1990 and that cannot be changed without breaking (some) backwards compatibility. That's a fact.
Another fact is that there is no single breaking change and/or abstraction which by itself enables big refactorings/experiments/optimizations, because many of them have multiple blockers. So if you take any breaking change on its own, it's never worthwhile because it breaks compatibility for no immediate gain. On the other hand, every such small change decreases a tiny bit of the technical debt which has been accumulated over the years.
And on the third hand, every small change is disruptive. Perhaps in ways we don't hear about: people stick to old versions, abandon their projects, or switch to other languages.
IMHO, the correct question that the core CPython developers need to ask themselves is how much technical debt they are willing to carry over forever for the sake of backwards compatibility at all costs. I think that the tone of my email makes it clear what my personal opinion is :), but again this is something which should be decided by CPython developers.
It would also be fair if we could ask the projects that use the API, and see if the proposed improvements are actually worth the churn for them. For this particular change, I haven't actually seen the proposed improvements -- just vague ideas. Why should vague ideas justify the maintenance burden for existing, working projects?