Hello list!
After the e-mails from the previous week, I set up a call with Eric to
sync on Limited API/Stable ABI issues.
The higher-bandwidth (and more emotion-friendly) medium worked great for
us, but unfortunately, everyone else was excluded.
Here are some rather unstructued notes from my point of view. They might
be good discussion starters :)
Please read PEP 652 “Maintaining the Stable ABI” for my thoughts, plans
and rationales for the short term.
For the long term, there are some more vague plans and ideas:
There's not a 1:1 mapping between “Limited API”, “Stable ABI” and
“C-API, the Good Parts™”, *but* unless you're deep in this space, it
makes sense to conflate them. I aim to make them converge.
I intend to focus my CPython time on Limited API/Stable ABI for the
next... few years, probably.
The work I'm doing (extension isolation & now stable ABI) is similar to
the subinterpreters effort in several ways:
- They are both incomplete, but useful *today* for some limited cases,
and have lots of potential
- They need long-term effort, but most of the many steps are useful in
their own right.
In addition to the use cases in PEP 652, the stable ABI *should* ideally
be useful for:
- bindings for non-Python software, where shoehorning Python into the
buildsystem is not straightforward and building for several Python
versions at once is not practical. Also, raw speed tends to not matter.
- GUI apps, whose scripting plugins could use any Python the
user/scripter has installed. (Think Blender, if it was a smaller project
that couldn't afford to bundle Python.)
And here's some more concrete stuff. (Remember that these are still
vague ideas):
Static `PyObject` and `PyObject*` are the main thing in the Limited API
that's blocking subinterpreters. We see no other blockers.
A possible way to solve this is to make isolated subinterpreters support
*opt-in* for extension modules:
- Introduce a macro similar to the old PY_SSIZE_T_CLEAN that removes the
problematic items from the headers.
- This macro will give you access to a flag you can use to tell the
runtime the extension is subinterp-safe.
- Python will support both the current, GIL-sharing subinterpreters, and
isolated subinterpreters. (All signs say implementing this will be easy,
relative to making subinterpreters always isolated and breaking existing
code.)
- The macro will affect both the full API and the limited subset.