Python C API has no strict separation between the 3 levels of API:
* core: Py_BUILD_CORE define * stable: Py_LIMITED_API define (it has a value) * regular: no define
IMHO the current design of header files is done backward: by default, everything is public. To exclude an API from core or stable, "#ifdef Py_BUILD_CORE" and "#ifndef Py_LIMITED_API" are used. This design caused issues in the past: functions, variables or something else exposed whereas they were supposed to be "private".
I propose a practical solution for that: Include/*.h files would only be be public API. The "core" API would live in a new subdirectory: Include/pycore/*.h. Moreover, files of this subdirectory would have the prefix "pycore_". For example, Include/objimpl.h would have a twin: Include/pycore/pycore_objimpl.h which extend the public API with "core" APIs.
I also propose to automatically load the twin: Include/objimpl.h would load Include/pycore/pycore_objimpl.h if Py_BUILD_CORE is defined:
#ifdef Py_BUILD_CORE # include "pycore/pycore_objimpl.h" #endif
Only in some rare cases, you would have to explicitly use: #include "pycore/pycore_pygetopt.h". This header is fully private, there is no public header in Include/pygetopt.h. Or maybe we should modify Include/Python.h to also include "pycore/pycore_pygetopt.h" if Py_BUILD_CORE is defined? Well, that's just a detail.
* Create Include/pycore/ * Move Py_BUILD_CORE specific code into Include/pycore/pycore_*.h * Automatically include pycore files from Include/*.h files (#ifdef Py_BUILD_CORE)
* Find a solution for Py_LIMITED_API :-)
Backward compatibility? The overall change is fully backward compatible. The default API doesn't change. C code (using header fles) doesn't have to be changed.
Only a specific kinds of applications like debugger may have to be modified if they really have to access the "core" API, the "most private" API. Honestly, today it's unclear to me if this API can technically be used outside CPython.