[New-bugs-announce] [issue39573] Make PyObject an opaque structure in the limited C API

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

Today, CPython is leaking too many implementation through its public C API. We cannot easily change the "default" C API, but we can enhance the "limited" C API (when Py_LIMITED_API macro is defined). Example of leaking implementation details: memory allocator, garbage collector, structure layouts, etc.

Making PyObject an opaque structure would allow in the long term of modify structures to implement more efficient types (ex: list specialized for small integers), and it can prepare CPython to experiment tagged pointers.

Longer rationale:

* https://pythoncapi.readthedocs.io/
* https://pythoncapi.readthedocs.io/bad_api.html
* https://pythoncapi.readthedocs.io/optimization_ideas.html

I propose to incremental evolve the existing limited C API towards opaque PyObject, by trying to reduce the risk of breakage.

We may test changes on PyQt which uses the limited C API.

Another idea would be to convert some C extensions of the standard library to the limited C API. It would ensure that the limited C API contains enough functions to be useful, but would also notify us directly if the API is broken.

----------
components: C API
messages: 361513
nosy: vstinner
priority: normal
severity: normal
status: open
title: Make PyObject an opaque structure in the limited C API
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39573>
_______________________________________