Rejected Ideas ==============

It might be good to add a similar tier in the Python (not C) API, e.g. for ``types.CodeType``. However, the opt-in mechanism would need to be different (if any). This is outside the scope of the PEP.

For types.CodeType constructor, would it be possible to just a mention in the *documentation* that this API is "unstable"? It would come with a link to definition of the "unstable" C API: explain that it can change in 3.x.y bugfix releases, not not in 3.x.0 releases (major? minor? I never recall how they should be called). For now, I don't think that there is a need to actively remove this API from the "default" Python API and add an opt-in option to get access to these functions. But having a mention just in the documentation would be better than nothing. It seems to be popular complain and request. For example, most of the ast module would fall into this "unstable API". Previous discussions: * Proposal: declare "unstable APIs" https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM... * Making code object APIs unstable https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... On one side, it's important to communicate that the API *can* change in 3.x.0 releases, but also provide some warranties that the API *must not change* in 3.x.y bugfix releases. Victor

On Fri, Apr 29, 2022 at 10:15 AM Petr Viktorin wrote:

On 29. 04. 22 16:32, Victor Stinner wrote:

...

Ok, let me start with the serious business: API name.

I'm not comfortable with "semi-stable". Python already has a "limited API" and a "stable ABI". Just by its name, it's unclear what "semi-stable" means.

Honestly, I would be more comfortable with the name: "unstable API". It would be clear that the API *can* change often. People who want to know exactly the backward compatibility warranties can dig into the API documentation to learn more about it.

"Unstable API" is also the name the Guido proposed for PyCode_New() last year:

* Proposal: declare "unstable APIs"

https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM...

...

* Making code object APIs unstable

https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY...

...

Victor

Nick Coghlan argued against that term:

...

"unstable" is the wrong term. We already have an unstable API tier: the internal API, which can change even in maintenance releases. The value of the new tier is that it is "semi stable": stable in maintenance releases, unstable in feature releases.

—

https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV...

But I also like “unstable” better than “semi-stable”. Splitting the internals into “private”/“internal” and “unstable” seems reasonable.

I think picking "semi-stable" would be giving in to the OCD nerd in all of us. :-) While perhaps technically less precise, "unstable" is the catchy name with the right association. (And yes, we should keep it stable within bugfix releases, but the name doesn't need to reflect that detail.) The "internal API" isn't an API at all (except for CPython core developers and contributors). The "unstable API" would definitely be an *API* for users outside the core. So let's please go with "unstable". -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On 2022-04-29 18:02, Guido van Rossum wrote:

On Fri, Apr 29, 2022 at 10:15 AM Petr Viktorin mailto:encukou@gmail.com> wrote:

On 29. 04. 22 16:32, Victor Stinner wrote: > Ok, let me start with the serious business: API name. > > I'm not comfortable with "semi-stable". Python already has a "limited > API" and a "stable ABI". Just by its name, it's unclear what > "semi-stable" means. > > Honestly, I would be more comfortable with the name: "unstable API". > It would be clear that the API *can* change often. People who want to > know exactly the backward compatibility warranties can dig into the > API documentation to learn more about it. > > "Unstable API" is also the name the Guido proposed for PyCode_New() last year: > > * Proposal: declare "unstable APIs" > https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM... https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM... > * Making code object APIs unstable > https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... > > Victor

Nick Coghlan argued against that term:

> "unstable" is the wrong term. We already have an unstable API tier: the > internal API, which can change even in maintenance releases. The value of > the new tier is that it is "semi stable": stable in maintenance releases, > unstable in feature releases.

— https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV... https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV...

But I also like “unstable” better than “semi-stable”. Splitting the internals into “private”/“internal” and “unstable” seems reasonable.

I think picking "semi-stable" would be giving in to the OCD nerd in all of us. :-) While perhaps technically less precise, "unstable" is the catchy name with the right association. (And yes, we should keep it stable within bugfix releases, but the name doesn't need to reflect that detail.) The "internal API" isn't an API at all (except for CPython core developers and contributors). The "unstable API" would definitely be an *API* for users outside the core.

So let's please go with "unstable".

I was going to suggest "metastable". Too late? :-)

On 4/29/2022 11:42 AM, Stephen J. Turnbull wrote:

MRAB writes:

...

On 2022-04-29 18:02, Guido van Rossum wrote:

...

On Fri, Apr 29, 2022 at 10:15 AM Petr Viktorin mailto:encukou@gmail.com> wrote:

On 29. 04. 22 16:32, Victor Stinner wrote: > Ok, let me start with the serious business: API name. > > I'm not comfortable with "semi-stable". Python already has a "limited > API" and a "stable ABI". Just by its name, it's unclear what > "semi-stable" means. > > Honestly, I would be more comfortable with the name: "unstable API". > It would be clear that the API *can* change often. People who want to > know exactly the backward compatibility warranties can dig into the > API documentation to learn more about it. > > "Unstable API" is also the name the Guido proposed for PyCode_New() last year: > > * Proposal: declare "unstable APIs" > https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM... https://mail.python.org/archives/list/python-dev@python.org/thread/JM6SQ2YNM... > * Making code object APIs unstable > https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... https://mail.python.org/archives/list/python-dev@python.org/thread/ZWTBR5ESY... > > Victor

Nick Coghlan argued against that term:

> "unstable" is the wrong term. We already have an unstable API tier: the > internal API, which can change even in maintenance releases. The value of > the new tier is that it is "semi stable": stable in maintenance releases, > unstable in feature releases.

— https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV... https://mail.python.org/archives/list/python-dev@python.org/message/CTKKTHUV...

But I also like “unstable” better than “semi-stable”. Splitting the internals into “private”/“internal” and “unstable” seems reasonable.

I think picking "semi-stable" would be giving in to the OCD nerd in all of us. :-) While perhaps technically less precise, "unstable" is the catchy name with the right association. (And yes, we should keep it stable within bugfix releases, but the name doesn't need to reflect that detail.) The "internal API" isn't an API at all (except for CPython core developers and contributors). The "unstable API" would definitely be an *API* for users outside the core.

So let's please go with "unstable".

I was going to suggest "metastable". Too late? :-) A Bikeshedding to a new level! --+

Metabikeshedding...

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CGNNVU7C... Code of Conduct: http://python.org/psf/codeofconduct/

On 2022-04-30 03:17, Greg Ewing wrote:

On 30/04/22 5:25 am, MRAB wrote:

...

I was going to suggest "metastable". Too late? :-)

What, the API is balanced on a knife edge and likely to collapse into something else if you sneeze too hard?

There's a possibility that the universe might be metastable, so a metastable API might not be that big a deal.

On 29. 04. 22 19:02, Guido van Rossum wrote:

On Fri, Apr 29, 2022 at 10:15 AM Petr Viktorin mailto:encukou@gmail.com> wrote:

On 29. 04. 22 16:32, Victor Stinner wrote: > Ok, let me start with the serious business: API name. > > I'm not comfortable with "semi-stable". Python already has a "limited > API" and a "stable ABI". Just by its name, it's unclear what > "semi-stable" means. [...] Nick Coghlan argued against that term: [...] But I also like “unstable” better than “semi-stable”. Splitting the internals into “private”/“internal” and “unstable” seems reasonable.

I think picking "semi-stable" would be giving in to the OCD nerd in all of us. :-) While perhaps technically less precise, "unstable" is the catchy name with the right association. (And yes, we should keep it stable within bugfix releases, but the name doesn't need to reflect that detail.) The "internal API" isn't an API at all (except for CPython core developers and contributors). The "unstable API" would definitely be an *API* for users outside the core.

So let's please go with "unstable".

Thanks, you worded that perfectly! Alright, the PEP now uses “unstable” rather than “semi-stable”. And I don't see any issues with the technical details, so I'll see if it can still get into Python 3.11. Hopefully Pablo agrees as the Release Manager. Thanks for the discussion, everyone!

On Thu, May 5, 2022 at 5:35 AM Thomas Wouters wrote: [...]

I've already brought this up to Petr directly, but I would greatly prefer new unstable API functions have leading underscores, and that existing functions being moved to the unstable API are _not_ renamed.

Renaming existing functions means a lot of unnecessary code churn. It looks like we have more _-prefixed unstable functions than not, but I don't think the churn is worth renaming the currently public ones.

Leading underscores for unstable API functions (that aren't currently public) means we keep the widely assumed guarantee that Py*/PY* are part of the public API. The Py_USING_UNSTABLE_API define is per-file, not per symbol/use, so I would rather not open the door to unintended or unaware use of unstable APIs. By giving the functions the leading underscore, we're forcing people to think about -- or check the documentation -- whether the specific function is okay to use.

The unstable API is intended for specific use-cases, and I think it's preferable to put the burden of figuring out if a _Py/_PY* symbol is acceptable for them to use, rather than putting the burden of figuring out if a Py/PY* symbol is acceptable up use on _everyone else_.

I don't think that's necessary. I still see the unstable API as public: it needs more maintenance, but it's not particularly dangerous to use it. I think defining Py_USING_UNSTABLE_API once is quite enough.

On Fri, 6 May 2022 at 22:50, Petr Viktorin wrote:

IMO it's fine to delay formalizing this until 3.12, so we can have a proper discussion. However, if someone has the time, it would be nice to ensure the use cases from PEP 523 are possible with the 3.11 API. Specifically, AFAIK, struct _PyInterpreterFrame needs to be exposed (as an incomplete, opaque struct) to make _PyFrameEvalFunction usable, _PyFrame_GetFrameObject needs to be exposed, and a PyEval_EvalFrameDefault that takes PyFrameObject should be added. Also, the What's New currently says “See PEP 523 for more details of how to use [_PyFrameEvalFunction]”, which isn't very helpful.

Given the proximity of beta1, I think affected projects are going to need to define Py_BUILD_CORE for the Python 3.11 series - at least some of them have already been updated to declare that during the alpha cycle to handle the frame API changes. PEP 689 can then be informed by feedback from those projects regarding what breaks for them if they *don't* declare Py_BUILD_CORE. That said, one slightly awkward consequence of this approach is affected projects would end up needing to include CPython's "patchlevel.h" [1] directly, so they can query PY_VERSION_HEX *before* including "Python.h". Something like: #include "patchlevel.h" # CPython's version declaration header #if Py_VERSION_HEX+0 >= 0x030c0000 # CPython 3.12+ #define Py_USING_UNSTABLE_API #elif Py_VERSION_HEX+0 >= 0x030b0000 # CPython 3.11.x #define Py_BUILD_CORE #endif #include "Python.h" So maybe it would be worth asking the SC for an exception to the beta freeze, and allow the unstable API tier to land during the beta period? Cheers, Nick. [1] https://github.com/python/cpython/blob/main/Include/patchlevel.h -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

I would love to see header files used for this -- while I know there is a long tradition of feature-flags that must be #defined by the user before #including a header in order to affect what the header exports (or not!), 30 years later I still find that approach pretty unintuitive. But yes, it's going to be a complex transition. On Mon, May 30, 2022 at 12:30 PM Brett Cannon wrote:

We discussed having leading underscores for this API tier, and it was decided that a leading underscore was preferred.

This did start a discussion, though, about whether we should control API access/opt-in via `#include` by having `.h` files that convey what API the user is opting into, or use `#define` to control what gets exposed via `Python.h`. The general feeling was that the header file idea is ideal, but it is a little extra work to transition to if you want to be compatible with older versions of Python that wouldn't have the header files (Victor's compatibility project could help here). The question for the team is whether separate header files makes sense to others, or would people prefer using `#define` and `Python.h` to control API access/opt-in? _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q5JU5YKG... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Mon, May 30, 2022 at 12:54 PM Steve Dower wrote:

I prefer separate header files, provided people outside of core always have one (presumably "Python.h") that should be included first and includes enough info to check which headers will be available (i.e. the version defs).

The idea we were kicking around was e.g. `Python-unstable.h` would be all of the limited API plus the unstable parts, `Python-unlimited.h` would be **everything**, etc. I would expect `Python.h` would continue to be what it is today for compatibility purposes. There wouldn't necessarily be an "always have one" header since these header files would cascade into each other as you opted into more and more unstable APIs (think about this as layers of APIs 😉 and representing each encompassing layer with a header file). This would also let teams set policies of how much instability risk they were willing to take by having CI have an allowlist/blocklist of Python header files. -Brett

Modifying preprocessor definitions for different Python versions, or having to set them before knowing what version is being used, seems more complicated.

Cheers, Steve

On 5/30/2022 8:26 PM, Brett Cannon wrote:

...

We discussed having leading underscores for this API tier, and it was decided that a leading underscore was preferred.

This did start a discussion, though, about whether we should control API access/opt-in via `#include` by having `.h` files that convey what API the user is opting into, or use `#define` to control what gets exposed via `Python.h`. The general feeling was that the header file idea is ideal, but it is a little extra work to transition to if you want to be compatible with older versions of Python that wouldn't have the header files (Victor's compatibility project could help here). The question for the team is whether separate header files makes sense to others, or would people prefer using `#define` and `Python.h` to control API access/opt-in?

On Mon, 30 May 2022 12:53:57 -0700 Guido van Rossum wrote:

I would love to see header files used for this -- while I know there is a long tradition of feature-flags that must be #defined by the user before #including a header in order to affect what the header exports (or not!), 30 years later I still find that approach pretty unintuitive.

Agreed that #defining a flag before #including a header is a brittle approach. If something else included the header before you set your #define, then include guards can prevent you from seeing any effects. This is a common issue in Windows land with the godawful Windows.h header file. Regards Antoine.

But yes, it's going to be a complex transition.

On Mon, May 30, 2022 at 12:30 PM Brett Cannon wrote:

...

We discussed having leading underscores for this API tier, and it was decided that a leading underscore was preferred.

This did start a discussion, though, about whether we should control API access/opt-in via `#include` by having `.h` files that convey what API the user is opting into, or use `#define` to control what gets exposed via `Python.h`. The general feeling was that the header file idea is ideal, but it is a little extra work to transition to if you want to be compatible with older versions of Python that wouldn't have the header files (Victor's compatibility project could help here). The question for the team is whether separate header files makes sense to others, or would people prefer using `#define` and `Python.h` to control API access/opt-in? _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Q5JU5YKG... Code of Conduct: http://python.org/psf/codeofconduct/

I understand issues and welcome any discussions. For that matter I do not rush to conclusions. I am not expert in C and Python as the rest of the folks on this list but I am pretty good with Python itself. I just suggested naming to be as simple as possible for all relevant API's including full descriptions in the code base regarding stable, semi-stable, unstable and so forth. I do that in my projects with Python libraries I write ... Sorry for intruding and possibly clouding the email thread... Regards, On Wed, Jun 1, 2022, 4:39 AM Stephen J. Turnbull wrote:

Sasha Kacanski writes:

...

Why you don't simplify with api A,B,C and forth and then follows explanation ofr what is stable, unstable, semi... So forth....

This is exactly what they're hammering out. It's not easy for several reasons, chief of which is that in each case the boundary is a matter of opinion as to the balance among what is most convenient for the developers of Python itself, the developers of separately distributed C/C++ modules, and for existing modules that were developed before the divisions were set and would need to either be changed or to risk API incompatibility with future versions of Python. The nomenclature also matters, as individual programmers have various ideas about the meaning of terms like "stable", and we want as much agreement as possible that the "stable API" is "stable enough", and so on.

If you have specific ideas about which APIs belong where, feel free to bring them forward. But this is not a process that should be rushed nor would anyone benefit from pushing it forward more quickly.