[Python-ideas] PEP: Hide implementation details in the C API

[Brett Cannon]

On Thu, 13 Jul 2017 at 09:12 Ronald Oussoren <ronaldoussoren at mac.com> wrote:

> On 12 Jul 2017, at 20:51, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Wed, 12 Jul 2017 at 01:25 Ronald Oussoren <ronaldoussoren at mac.com>
> wrote:
>
>>
>> > On 11 Jul 2017, at 12:19, Victor Stinner <victor.stinner at gmail.com>
>> wrote:
>>
> [SNIP]
>
> >
>> > Step 3: first pass of implementation detail removal
>> > ---------------------------------------------------
>> >
>> > Modify the ``python`` API:
>> >
>> > * Add a new ``API`` subdirectory in the Python source code which will
>> >  "implement" the Python C API
>> > * Replace macros with functions. The implementation of new functions
>> >  will be written in the ``API/`` directory. For example, Py_INCREF()
>> >  becomes the function ``void Py_INCREF(PyObject *op)`` and its
>> >  implementation will be written in the ``API`` directory.
>>
>> In this particular case (Py_INCREF/DECREF) making them functions isn’t
>> really useful and is likely to be harmful for performance. It is not useful
>> because these macros manipulate state in a struct that must be public
>> because that struct is included into the structs for custom objects
>> (PyObject_HEAD). Having them as macro’s also doesn’t preclude moving to
>> indirect reference counts. Moving to anything that isn’t reference counts
>> likely needs changes to the API (but not necessarily, see PyPy’s cpext).
>>
>
> I think Victor has long-term plans to try and hide the struct details at a
> higher-level and so that would make macros a bad thing. But ignoring the
> specific Py_INCREF/DECREF example, switching to functions does buy us the
> ability to actually change the function implementations between Python
> versions compared to having to worry about what a macro used to do (which
> is a possibility with the stable ABI).
>
>
> I don’t understand. Moving too functions instead of macros for some thing
> doesn’t really help with keeping the public API stable (for the non-stable
> ABI).
>

Sorry, I didn't specify which ABI/API I was talking about; my point was
from the stable ABI.

I think this is quickly showing how naming is going to play into this since
e.g. we say "stable ABI" but call it "Py_LIMITED_API" in the code which is
rather confusing.

Just to make sure I'm not missing anything, it seems we have a few levels
here:

1. The stable A**B**I which is compatible across versions
2. A stable A**P**I which hides enough details that if we change a struct
your code won't require an update, just a recompile
3. An API that exposes CPython-specific details such as structs and other
details that might not be entirely portable to e.g. PyPy easily but that we
try not to break
4. An internal API that we use for implementing the interpreter but don't
expect anyone else to use, so we can break it between feature releases
(although if e.g. Cython chooses to use it they can)

(There's also an API local to a single file, but since that is never
exported to the linker it doesn't come into play here.)

So, a portable API/ABI, a stable API, a CPython API, and then an
internal/core/interpreter API. Correct?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170713/53858a88/attachment-0001.html>