On Thu, 13 Jul 2017 at 09:12 Ronald Oussoren <ronaldoussoren@mac.com> wrote:
On 12 Jul 2017, at 20:51, Brett Cannon <brett@python.org> wrote:

On Wed, 12 Jul 2017 at 01:25 Ronald Oussoren <ronaldoussoren@mac.com> wrote:

> On 11 Jul 2017, at 12:19, Victor Stinner <victor.stinner@gmail.com> wrote:
> Step 3: first pass of implementation detail removal
> ---------------------------------------------------
> Modify the ``python`` API:
> * Add a new ``API`` subdirectory in the Python source code which will
>  "implement" the Python C API
> * Replace macros with functions. The implementation of new functions
>  will be written in the ``API/`` directory. For example, Py_INCREF()
>  becomes the function ``void Py_INCREF(PyObject *op)`` and its
>  implementation will be written in the ``API`` directory.

In this particular case (Py_INCREF/DECREF) making them functions isn’t really useful and is likely to be harmful for performance. It is not useful because these macros manipulate state in a struct that must be public because that struct is included into the structs for custom objects (PyObject_HEAD). Having them as macro’s also doesn’t preclude moving to indirect reference counts. Moving to anything that isn’t reference counts likely needs changes to the API (but not necessarily, see PyPy’s cpext).

I think Victor has long-term plans to try and hide the struct details at a higher-level and so that would make macros a bad thing. But ignoring the specific Py_INCREF/DECREF example, switching to functions does buy us the ability to actually change the function implementations between Python versions compared to having to worry about what a macro used to do (which is a possibility with the stable ABI).

I don’t understand. Moving too functions instead of macros for some thing doesn’t really help with keeping the public API stable (for the non-stable ABI).

Sorry, I didn't specify which ABI/API I was talking about; my point was from the stable ABI.

I think this is quickly showing how naming is going to play into this since e.g. we say "stable ABI" but call it "Py_LIMITED_API" in the code which is rather confusing.

Just to make sure I'm not missing anything, it seems we have a few levels here:

1. The stable A**B**I which is compatible across versions
2. A stable A**P**I which hides enough details that if we change a struct your code won't require an update, just a recompile
3. An API that exposes CPython-specific details such as structs and other details that might not be entirely portable to e.g. PyPy easily but that we try not to break
4. An internal API that we use for implementing the interpreter but don't expect anyone else to use, so we can break it between feature releases (although if e.g. Cython chooses to use it they can)

(There's also an API local to a single file, but since that is never exported to the linker it doesn't come into play here.)

So, a portable API/ABI, a stable API, a CPython API, and then an internal/core/interpreter API. Correct?