On Fri, Jun 4, 2021 at 12:29 AM Guido van Rossum <guido@python.org> wrote:
>> In the C API, there is the internal C API which fits with your
>> description. To access it, you have to declare the
>> Py_BUILD_CORE_MODULE macro. It's not usable directly on purpose. It's
>> an user agreement: I know what I am doing, and I know that this API is
>> not supported nor stable.
>
> Hm, but aren't for example all the fields of code objects (co_name, co_argcount, etc.) in the "non-internal" API? Those (and the functions that manipulate code objects) are a prime example of what I'd consider "unstable".
>
> On https://docs.python.org/3/c-api/code.html it already says about the fields "The fields of this type are subject to change at any time." But I think everything else on that page should be considered unstable as well. (And why do we even have PyCode_GetNumFree()?)
Hum, the C API is somehow off-topic, but let me reply anyway ;-)
I see the smiley but I still don't understand why you're saying this.
As I explained in my PEP 620, the Python C API never had any design.
That's not entirely true. *Parts* of it were definitely designed, e.g. the existing "abstract" API (PyObject_GetItem, _GetAttr, and many more; IIRC Zope's Jim Fulton pushed for these). Other parts are indeed not designed much, all the PyList, PyDict, PyTuple etc. APIs (which came before the "abstract" ones), and the many APIs that followed their example (to this day).
Things were only exposed because it was easy and technically possible,
and it was a convenient way to define a function in one file and uses
it from another file.
Historically, it was even worse -- people started embedding Python before we even had "Py" prefixes. Try to find out about the "great renaming" (I think it was the late '90s).
But 30 years later, we identified that exposing
some things are causing troubles and we are trying to make the C API
more "abstract".
This time the abstraction isn't just about supporting different object types, it's about allowing us to change other aspects of the interpreter, in particular object layout ("object model" as it's somehow known) and memory management.
Who knows what we'll want to change 20 years from now...
Exposing directly all structures is causing a lot of headaches at
every new Python 3.x release. Getter and setter functions can handle
structure changes, retrieve information from another structure, return
an error, etc. This abstractation is needed for users to not have to
update their code, and to CPython developers to be able to change
things.
If possible, I would prefer to make PyThreadState, PyCodeObject and
other structures opaque, and only go through getter and setter
functions ;-) PyCode_New() is another problem :-/ The PEP 570 first
changed it to add a new parameter. It broke Cython and other projects.
The change was reverted, and PyCode_NewWithPosOnlyArgs() was added.
The lesson is that it's possible to change PyCodeObject without
breaking PyCode_New() (which handles the "backward compatibility" for
you).
I'm afraid that won't always be possible though. At some point there just may not be a valid meaning for the original PyCode_New() call. It was easy in the case of positional arguments (by default don't have any) but it may not always be that simple, and we shouldn't make guarantees here.