On Wed, Jun 3, 2020 at 2:13 PM Victor Stinner <vstinner@python.org> wrote:
Le mer. 3 juin 2020 à 19:17, Mark Shannon <mark@hotpy.org> a écrit :
> > I also *added* a bunch of *new* "getter" or "setter" functions to the
> > public C API for my project of hiding implementation details, like
> > making structures opaque:
> > https://docs.python.org/dev/whatsnew/3.9.html#id1
>
> Adding "setters" is generally a bad idea.
> "getters" can be computed if the underlying field disappears, but the
> same may not be true for setters if the relation is not one-to-one.
> I don't think there are any new setters in 3.9, so it's not an immediate
> problem.

You're making the assumption that the member can be set directly. But
my plan is to make the structure opaque. In that case, you need
getters and setters for all fields you would like to access. No member
would be accessible directly anymore.

> `PyThreadState_GetInterpreter()` can't replace `tstate->interp` for two
> reasons.
> 1. There is no way to stop third party C code accessing the internals of
> data structures. We can warn them not to, but that's all.
> 2. The internal layout of C structures has never been part of the API,
> with arguably two exceptions; the PyTypeObject struct and the
> `ob_refcnt` field of PyObject.

My long term plan is to make all structures opaque :-) So far,
PyInterpreterState structure was made opaque in Python 3.7. It helped
*a lot* the development of Python 3.8 and 3.9, especially for
subinterpreters. And I made PyGC_Head opaque in Python 3.9.

Examples of issues to make structures opaque:

PyGC_Head: https://bugs.python.org/issue40241 (done in Python 3.9)
PyObject: https://bugs.python.org/issue39573
PyTypeObject: https://bugs.python.org/issue40170
PyThreadState: https://bugs.python.org/issue39947
PyInterpreterState: https://bugs.python.org/issue35886 (done in Python 3.8)

For the short term, my plan is to make structure opaque in the limited
C API, before breaking more stuff in the public C API :-)

Indeed, your plan and the work you've been doing and discussing with other core devs about this (including at multiple sprints and summits) over the past 4+ years is the right one.  Our reliance on structs and related cpp macros unfortunately exposed as public is a burden that freezes reasonable CPython VM implementation evolution options.  This work moves us away from that into a better place one step at a time without mass disruption.

More prior references related to this work are critical reading and should not be overlooked:

[2017 "Keeping Python Competitive"] https://lwn.net/Articles/723949/
[2018 "Lets change the C API" thread] https://mail.python.org/archives/list/python-dev@python.org/thread/B67MYCAO4H4AJNMLSWVT3UVFTHSDGQRB/#B67MYCAO4H4AJNMLSWVT3UVFTHSDGQRB
[2019 "The C API"] https://pyfound.blogspot.com/2019/06/python-language-summit-lightning-talks-part-2.html
[2020-04 "PEP: Modify the C API to hide implementation details" thread - with a lot of links to much earlier 2017 and such references] https://mail.python.org/archives/list/python-dev@python.org/thread/HKM774XKU7DPJNLUTYHUB5U6VR6EQMJF/#HKM774XKU7DPJNLUTYHUB5U6VR6EQMJF
and Victors overall https://pythoncapi.readthedocs.io/roadmap.html as referenced a few places in those.

It is also worth paying attention to the https://mail.python.org/archives/list/capi-sig@python.org/latest mailing list for anyone with a CPython C API interest.

-gps



> PyObject_CallNoArgs() seems harmless.
> Rationalizing the call API has merit, but PyObject_CallNoArgs()
> leads to PyObject_CallOneArg(), PyObject_CallTwoArgs(), etc. and an even
> larger API.

PyObject_CallOneArg() also exists:
https://docs.python.org/dev/c-api/call.html#c.PyObject_CallOneArg

It was added as a private function https://bugs.python.org/issue37483
add made public in commit 3f563cea567fbfed9db539ecbbacfee2d86f7735
"bpo-39245: Make Vectorcall C API public (GH-17893)".

But it's missing in What's New in Python 3.9.

There is no plan for two or more arguments.


> PyObject_GC_IsTracked(). I don't like this.
> Shouldn't GC track *all* objects?
> Even if it were named PyObject_Cycle_GC_IsTracked() it would be exposing
> internal implementation details for no good reason. A cycle GC that
> doesn't "track" individual objects, but treats all objects the same
> could be more efficient. In which case, what would this mean?
>
> What is the purpose of PyObject_GC_IsFinalized()?
> Third party objects can easily tell if they have been finalized.
> Why they would ever need this information is a mystery to me.

Did you read the issues which added these functions to see the
rationale? https://bugs.python.org/issue40241

I like the "(Contributed by xxx in bpo-xxx.)" in What's New in Python
3.9: it became trivial to find such rationale.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QZ2Q7ELTDZUQLVS54T53CPEINWNQB6HF/
Code of Conduct: http://python.org/psf/codeofconduct/