On 03/06/2020 8:49 pm, Pablo Galindo Salgado wrote:
Just some comments on the GC stuff as I added them myself.
Shouldn't GC track *all* objects? No, extension types need to opt-in to the garbage collector and if so, implement the interface.
When you say "GC" I think you mean the backup cycle breaker. But that's not what it means to me, or in general. GC includes reference counting and applies to all objects. Naming is hard, which is why proper review is important.
Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing internal implementation details for no good reason.
In python, there is gc.is_tracked() in Python 3.1 and the GC module already exposes a lot of GC functionality since many versions ago. This just allows the same calls that you can do in Python using the C-API.
Just because something is exposed in Python doesn't mean an extra function has to be added to the C API. More general functions like `PyObject_Call()` exist already to call any Python object.
What is the purpose of PyObject_GC_IsFinalized()?
Because some objects may have been resurrected and this allows you to know if a given object has already been finalized. This can help to gather advance GC stats, to control some tricky situations with finalizers and the gc in C extensions or just to know all objects that are being resurrected. Note that an equivalent gc.is_finalized() was added in 3.8 as well to query this information from Python in the GC module and this call just allows you to do the same from the C-API.
Cheers, Pablo
On Wed, 3 Jun 2020 at 18:26, Mark Shannon <mark@hotpy.org <mailto:mark@hotpy.org>> wrote:
Hi Victor,
On 03/06/2020 2:42 pm, Victor Stinner wrote: > Hi, > > In Python 3.9, I *removed* dozens of functions from the *public* C > API, or moved them to the "internal" C API: > https://docs.python.org/dev/whatsnew/3.9.html#id3 > > For a few internal C API, I replaced PyAPI_FUNC() with extern to > ensure that they cannot be used outside CPython code base: Python 3.9 > is now built with -fvisibility=hidden on compilers supporting it (like > GCC and clang). > > I also *added* a bunch of *new* "getter" or "setter" functions to the > public C API for my project of hiding implementation details, like > making structures opaque: > https://docs.python.org/dev/whatsnew/3.9.html#id1
Adding "setters" is generally a bad idea. "getters" can be computed if the underlying field disappears, but the same may not be true for setters if the relation is not one-to-one. I don't think there are any new setters in 3.9, so it's not an immediate problem.
> > For example, I added PyThreadState_GetInterpreter() which replaces > "tstate->interp", to prepare C extensions for an opaque PyThreadState > structure.
`PyThreadState_GetInterpreter()` can't replace `tstate->interp` for two reasons. 1. There is no way to stop third party C code accessing the internals of data structures. We can warn them not to, but that's all. 2. The internal layout of C structures has never been part of the API, with arguably two exceptions; the PyTypeObject struct and the `ob_refcnt` field of PyObject.
> > The other 4 new Python 3.9 functions: > > * PyObject_CallNoArgs(): "most efficient way to call a callable Python > object without any argument" > * PyModule_AddType(): "adding a type to a module". I hate the > PyObject_AddObject() function which steals a reference on success. > * PyObject_GC_IsTracked() and PyObject_GC_IsFinalized(): "query if > Python objects are being currently tracked or have been already > finalized by the garbage collector respectively": functions requested > in bpo-40241. > > Would you mind to elaborate why you consider that these functions must > not be added to Python 3.9?
I'm not saying that no C functions should be added to the API. I am saying that none should be added without a PEP or proper review.
Addressing the four function you list.
PyObject_CallNoArgs() seems harmless. Rationalizing the call API has merit, but PyObject_CallNoArgs() leads to PyObject_CallOneArg(), PyObject_CallTwoArgs(), etc. and an even larger API.
PyModule_AddType(). This seems perfectly reasonable, although if it is a straight replacement for another function, that other function should be deprecated.
PyObject_GC_IsTracked(). I don't like this. Shouldn't GC track *all* objects? Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing internal implementation details for no good reason. A cycle GC that doesn't "track" individual objects, but treats all objects the same could be more efficient. In which case, what would this mean?
What is the purpose of PyObject_GC_IsFinalized()? Third party objects can easily tell if they have been finalized. Why they would ever need this information is a mystery to me.
> > >> Every one of these functions represents a maintenance burden. >> Removing them is painful and takes a lot of effort, but adding them is >> done casually, without a PEP or, in many cases, even a review. > > For the new functions related to hiding implementation details, I have > a draft PEP: > https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst > > But it seems like this PEP is trying to solve too many problems in a > single document, and that I have to split it into multiple PEPs. >
It does need splitting up, I agree.
> >> Why are these functions being added? Wasn't 1000 C functions enough? > > My PEP lists flaws of the existing C API functions. Sadly, fixing > flaws requires adding new functions and deprecating old ones in a slow > migration.
IMO, at least one function should be deprecated for each new function added. That way the API won't get any bigger.
Cheers, Mark.
> > I'm open to ideas how to fix these flaws differently (without having > new functions?). > > As written in my PEP, another approach is to design a new C API on top > of the existing one. That's exactly what the HPy project does. But my > PEP also explains why I consider that it only fixes a subset of the > issues that I listed. ;-) > https://github.com/vstinner/misc/blob/master/cpython/pep-opaque-c-api.rst#hp... > > Victor > _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5WLOHBCS... Code of Conduct: http://python.org/psf/codeofconduct/