Hi Victor,

On 03/06/2020 2:42 pm, Victor Stinner wrote:
> Hi,
>
> In Python 3.9, I *removed* dozens of functions from the *public* C
> API, or moved them to the "internal" C API:
> https://docs.python.org/dev/whatsnew/3.9.html#id3
>
> For a few internal C API, I replaced PyAPI_FUNC() with extern to
> ensure that they cannot be used outside CPython code base: Python 3.9
> is now built with -fvisibility=hidden on compilers supporting it (like
> GCC and clang).
>
> I also *added* a bunch of *new* "getter" or "setter" functions to the
> public C API for my project of hiding implementation details, like
> making structures opaque:
> https://docs.python.org/dev/whatsnew/3.9.html#id1

Adding "setters" is generally a bad idea.
"getters" can be computed if the underlying field disappears, but the
same may not be true for setters if the relation is not one-to-one.
I don't think there are any new setters in 3.9, so it's not an immediate
problem.

>
> For example, I added PyThreadState_GetInterpreter() which replaces
> "tstate->interp", to prepare C extensions for an opaque PyThreadState
> structure.

`PyThreadState_GetInterpreter()` can't replace `tstate->interp` for two
reasons.
1. There is no way to stop third party C code accessing the internals of
data structures. We can warn them not to, but that's all.
2. The internal layout of C structures has never been part of the API,
with arguably two exceptions; the PyTypeObject struct and the
`ob_refcnt` field of PyObject.

>
> The other 4 new Python 3.9 functions:
>
> * PyObject_CallNoArgs(): "most efficient way to call a callable Python
> object without any argument"
> * PyModule_AddType(): "adding a type to a module". I hate the
> PyObject_AddObject() function which steals a reference on success.
> * PyObject_GC_IsTracked() and PyObject_GC_IsFinalized(): "query if
> Python objects are being currently tracked or have been already
> finalized by the garbage collector respectively": functions requested
> in bpo-40241.
>
> Would you mind to elaborate why you consider that these functions must
> not be added to Python 3.9?

I'm not saying that no C functions should be added to the API. I am
saying that none should be added without a PEP or proper review.

Addressing the four function you list.

PyObject_CallNoArgs() seems harmless.
Rationalizing the call API has merit, but PyObject_CallNoArgs()
leads to PyObject_CallOneArg(), PyObject_CallTwoArgs(), etc. and an even
larger API.

PyModule_AddType(). This seems perfectly reasonable, although if it is a
straight replacement for another function, that other function should be
deprecated.

PyObject_GC_IsTracked(). I don't like this.
Shouldn't GC track *all* objects?
Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing
internal implementation details for no good reason. A cycle GC that
doesn't "track" individual objects, but treats all objects the same
could be more efficient. In which case, what would this mean?

What is the purpose of PyObject_GC_IsFinalized()?
Third party objects can easily tell if they have been finalized.
Why they would ever need this information is a mystery to me.

>
>
>> Every one of these functions represents a maintenance burden.
>> Removing them is painful and takes a lot of effort, but adding them is
>> done casually, without a PEP or, in many cases, even a review.
>
> For the new functions related to hiding implementation details, I have
> a draft PEP:
> https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst
>
> But it seems like this PEP is trying to solve too many problems in a
> single document, and that I have to split it into multiple PEPs.
>

It does need splitting up, I agree.

>
>> Why are these functions being added? Wasn't 1000 C functions enough?
>
> My PEP lists flaws of the existing C API functions. Sadly, fixing
> flaws requires adding new functions and deprecating old ones in a slow
> migration.

IMO, at least one function should be deprecated for each new function
added. That way the API won't get any bigger.

Cheers,
Mark.

>
> I'm open to ideas how to fix these flaws differently (without having
> new functions?). >
> As written in my PEP, another approach is to design a new C API on top
> of the existing one. That's exactly what the HPy project does. But my
> PEP also explains why I consider that it only fixes a subset of the
> issues that I listed. ;-)
> https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst#hpy-project
>
> Victor
>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5WLOHBCSFVMZJEFSJSKQQANZASU2WFV3/
Code of Conduct: http://python.org/psf/codeofconduct/