On Fri, Apr 1, 2022 at 2:06 AM Victor Stinner <vstinner@python.org> wrote:
Hi,

Update on this issue: I merged my 2 PRs.
https://bugs.python.org/issue46850

The following APIs have been moved to the internal C API:

- _PyFrameEvalFunction type
- _PyInterpreterState_GetEvalFrameFunc()
- _PyInterpreterState_SetEvalFrameFunc()
- _PyEval_EvalFrameDefault()

If you use any of these API in your debugger/profiler project, you
have do add something like the code below to your project:
---
#ifndef Py_BUILD_CORE_MODULE
#  define Py_BUILD_CORE_MODULE
#endif
#include <Python.h>
#if PY_VERSION_HEX >= 0x030B00A7
#  include <internal/pycore_interp.h> // _PyInterpreterState_SetEvalFrameFunc()
#  include <internal/pycore_ceval.h>  // _PyEval_EvalFrameDefault()
#endif
---

Contact me if you need help to update your affected projects.

IMO PEP 523 doesn't have to be updated since it already says that the
APIs are private.

Thanks for bringing this up on python-dev, Victor. That was good. But the point of the discussion should've been to continue working with people based on the replies rather than proceeding to merge removals of the APIs after people said they used them.  (echoing Steve and Petr here...)

We discussed this on the steering council today. These APIs were in a weird state and despite past decisions at the time of PEP-523 in 2016 they should be treated as public-ish rather than entirely private. Because we published a document saying "here they are, use them!" and multiple projects have done so to good effect.

For 3.11 we'd like those PRs reverted.  We see the following as the better way forward for these APIs:

Add a new #define that can be set before the #include <Python.h> that exposes non-limited but stable within a bugfix/patch releases APIs (ie: Petr's earlier suggestion).
These would be the first to fall within. To do so we should give these, behind that #define, non _-prefixed "public style" names as these are quasi-public and cannot be changed as readily as other true internals.  We still, per the PEP, reserve the right to turn these into no-op potentially warning setting APIs in a future release (likely 3.12?) as they are at least documented as being unstable/private in the PEP. 

So in 3.11 these should continue to exist as in 3.6-3.10:
- _PyFrameEvalFunction type
- _PyInterpreterState_GetEvalFrameFunc()
- _PyInterpreterState_SetEvalFrameFunc()
- _PyEval_EvalFrameDefault()

AND in 3.11:
 - #define protected versions of those without the leading _ become available.
 - (i'm intentionally not suggesting a #define name, y'all can pick something)

In 3.12:
 - the _ prefixed versions can go away.  People using the APIs should've updated their code to use the new #define and new names when building against >=3.11 by then.
 - Whether the APIs continue to be as useful and act as originally claimed in PEP 523 is up to the 3.12 implementors (out of scope for this thread). They occupy a newly defined middle ground between the "forever" style limited API and the "can break even on bugfix/patch release" internal API that wasn't a concept for us in 2016 when PEP 523 was written.

Why? Being conservative with things in active use that weren't really private, and similar to what Mark Shannon and Petr said, we *do not* want people to #define Py_BUILD_CORE_MODULE and start poking at internals in arbitrary ways. That exposes a whole pile of other things for (ab)use that are even more unstable. Avoiding that helps avoid temptation to go wild and helps us identify users.

-gps (steering council hat on)


Since these APIs were added by PEP 523, I documented these changes in
What's New in Python 3.11 > C API > Porting to Python 3.11,even if
these APIs are private.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DNJC6U36CDA7S7ATEGAMUPABBSEIYHC4/
Code of Conduct: http://python.org/psf/codeofconduct/