On Tue, Mar 16, 2021 at 1:27 PM Petr Viktorin encukou@gmail.com wrote:
On 16. 03. 21 17:38, Eric Snow wrote:
...
The limited API is a subset of the full C-API. I don't understand how you want to limit the limited API but allow the full API.
Right. This would require that the limited API and the full API diverge. The limited API would stay as-is. In the public API we would introduce a per-interpreter lookup function (and perhaps replace the public PyObject variables with macros that call lookup functions, more or less).
Again, this is viable only if limited API extensions are only allowed to be imported in the main interpreter. The benefit is that we can move forward toward a per-interpreter GIL while preserving the limited API without a disruptive/broad change in the repo and without breaking existing extensions. The main downside is that the limited API diverges, which I consider okay in this limited and focused case.
Also, extensions using the limited API can currently be loaded in subinterpreters. Disallowing this would break backwards compatibility.
The big question is if there are extensions out there that both define Py_LIMITED_API *and* are used in subinterpreters. It's worth checking but I expect the answer is that there are none.
If that's the case then we're fine. Otherwise there's a viable solution if we want to move ahead with the restriction: add a "legacy" mode for subinterpreters that preserves the current status quo (much like you suggest in https://github.com/ericsnowcurrently/multi-core-python/issues/71).
- introduce a "limited API 2.0"
Yes, please. I would very much prefer this (except in my head it's not 2.0 but "Stable ABI 4.0", as the current versions follow Python's).
- specify that a subset of the limited API that extensions must target
in order to support for subinterpreters (i.e. "Py_LIMITED_PLUS_SUBINTERPRETERS_API")
Right, and let's call it "Stable ABI 4.0" :)
Before that, we can even deprecate (and eventually remove) problematic API from the limited set -- as long as we don't break ABI before 4.0. For details see PEP 652 (currently waiting for SC approval).
There are two distinct things involved here: ABI compatibility and API compatibility. I've started another thread about how those relate to the limited API. [1] Basically, I think such an updated limited API should not be tied to the stable ABI like the current one is (and deserves a different name than Py_LIMITED_API). See the thread for more discussion.
(I'm also wondering if we even need the stable ABI any more. [2] Many points in PEP 652 would still apply though.)
Anyway, your thoughts on the proposal would be helpful.
Please also look at my idea here: https://github.com/ericsnowcurrently/multi-core-python/issues/71
Ah, I missed this. I agree (though have a different idea on some of the details).
-eric
[1] https://mail.python.org/archives/list/capi-sig@python.org/thread/DX2HNPMJLIF... [2] https://mail.python.org/archives/list/capi-sig@python.org/thread/ZMNNLKK5NUW...