On 17. 03. 21 19:12, Eric Snow wrote:
On Wed, Mar 17, 2021 at 12:11 PM Eric Snow ericsnowcurrently@gmail.com wrote:
If that's the case then we're fine. Otherwise there's a viable solution if we want to move ahead with the restriction: add a "legacy" mode for subinterpreters that preserves the current status quo (much like you suggest in https://github.com/ericsnowcurrently/multi-core-python/issues/71).
To elaborate:
- introduce a field to PyConfig (or maybe PyInterpreterState directly) that indicates the interpreter should operate in "legacy" mode
- this could be a boolean or an int flag, a la PyTypeObject.tp_flags (or maybe an enum field, with members INTERP_LEGACY and INTERP_RETRICTED)
- if on PyConfig it would surface as PyInterpreterState.config.legacy (or "mode" or whatever)
- legacy mode would preserve the current status quo:
- each subinterpreter in legacy mode would use the main interpreter's GIL instead of its own
- extensions built with the limited API would be allowed in subinterpreters
- without legacy mode, importing a limited API extension would result in an ImportError
- there would be a new _PyInterpreterState_NewWIthConfig(PyConfig *) (or similar; internal-only for now) that would allow setting the mode
- the existing PyInterpreterState_New() would wrap it, using legacy mode
- the interpreters module (PEP 554) would *not* use legacy mode
This would take care of compatibility with the existing limited API.
FWIW, something like PyInterpreterState.mode has been part of the plan all along. Serendipitously, it happens to fit here. :) Also, restricted mode could include other things, like requiring PEP 489.
That sounds like a great idea!
It would solve the ABI problem, leaving us with only the API problem: PyLong_Type & friends need to become pointers. This will need the extensions' code to change, and IMO that means we need to introduce PyInt_GetType(), deprecate the static objects, and flag extensions that use the static objects as also not usable with subinterpreters.
From the earlier mail:
The big question is if there are extensions out there that both define Py_LIMITED_API *and* are used in subinterpreters. It's worth checking but I expect the answer is that there are none.
Python's big enough that I can tell you with near certainty that yes, there are :)