Hi all,
given the latest blow against exposing implementation details of CPython in
their C-API (see https://github.com/cython/cython/pull/5767 for the endless
story), I seriously start wondering if we shouldn't just define
"Py_BUILD_CORE" (or have our own "CYTHON_USE_CPYTHON_CORE_DETAILS" macro
guard that triggers its #define) and include the internal "pycore_*.h"
CPython header files from here:
https://github.com/python/cpython/tree/main/Include/internal
This would give us greater freedom in accessing all the implementation
details, so that we could directly integrate with those. We'd obviously
still need one or more fallback implementations for "stable CPython",
Limited API, PyPy and friends.
There's a risk, clearly, that these internals change even during point
releases. Maybe not a big risk, but not impossible either. We'd have to
deal with that and so would our users.
OTOH, having a single macro switch would make it easy for users to adapt if
something breaks on their side, and also easy to benchmark if it makes a
difference for their code.
We could also leave it off by default and simply allow users with high
performance needs to enable it manually. Or start by leaving it off until a
new CPython X.Y release has stabilised and its (used-by-us) internals have
proven not to change, and then switch it on for that release series. In any
case, having a single switch for this feels like it could be easy to handle.
What do you think?
Stefan
