On 06. 12. 21 21:50, Guido van Rossum wrote:
> On Mon, Dec 6, 2021 at 12:12 PM Petr Viktorin <encukou@gmail.com
> <mailto:encukou@gmail.com>> wrote:
>
> On 06. 12. 21 20:29, Guido van Rossum wrote:
> > Hi Petr,
> >
> > In PEP 384 it is written that no functions starting with an
> underscore
> > are part of the stable ABI:
> >
> > PEP 384 -- Defining a Stable ABI | Python.org
> > <https://www.python.org/dev/peps/pep-0384/#excluded-functions
> <https://www.python.org/dev/peps/pep-0384/#excluded-functions>>
> > > All functions starting with _Py are not available to applications
> >
> > OTOH there's a data file in the repo, Misc/stabe_abi.txt, which
> lists
> > many functions starting with _Py_, for example
> _PyObject_GC_Malloc. Then
> > again, that function is not listed in Doc/data/stable_abi.dat. (I
> didn't
> > check other functions, but maybe there are others.)
> >
> > So is Misc/stable_abi.txt just out of date? Or how can the
> discrepancy
> > be explained?
>
> These are not part of the limited API, so extension authors can't use
> them in the C source. But they typically are (or have been) called by
> macros from the limited API. So, they are part of the stable ABI; they
> need to be exported.
>
> Misc/stable_abi.txt says "abi_only" for all of these. They don't
> show up
> in the user-facing docs.
>
>
> Thanks, that helps. It's too bad that there's no comment at the top
> explaining the format (in fact it appears to discourage reading the file?).
You can read it, but I want to discourage people from relying on the
format: Tools/scripts/stable_abi.py should be the only consumer.
I will add a comment though.
I don't mind that the format might change. But I feel that I should be able to understand the current format so I can know how to maintain the document without using cargo-culting (and trying to understand what the tool does is not an option). So thanks for adding the comment.
> Also, it looks like Mark is proposing to *remove* _PyObject_GC_Malloc
> from stable_abi.txt in https://github.com/python/cpython/pull/29879
> <https://github.com/python/cpython/pull/29879> Is that allowed? If it's
> being used by a macro it means code using that macro will fail unless
> recompiled for 3.11.
Generally, that's not allowed. In this particular case, Victor's
analysis is right: if you trawl through the history from 3.2 on, you can
see that you can't call _PyObject_GC_Malloc via macros in the limited
API. So yes, this one can be removed.
Okay, that's very subtle, so thanks for confirming.
I'll also note that removing things that are "allowed" to go is not nice
to people who relied on PEP 384, which says that defining Py_LIMITED_API
"will hide all definitions that are not part of the ABI" -- even though
that's incompatible with the part where it says "All functions starting
with _Py are not available to applications".
I don't actually really follow what you are trying to say here. Probably because I've never paid much attention to PEP 384. I guess the API is confusing because the "right" way to do it (having to define some symbol to *expose* extra stuff rather than to *hide* stuff) was not possible for backwards compatibility reasons. But the extra negative will forever make this confusing. Also,
"All functions starting with _Py are not available" sounds like a clumsy way to say "No functions starting with _Py are available" (and you left out whether Py_LIMITED_API affects that availability, whether it was intended to affect it, whether it did in practice affect it in all cases, etc.
I assume it would be insensitive to ask whether we could just get rid of the stable ABI altogether and focus on the limited API? Just tell everyone they have to rebuild binary wheels for every Python feature release. Presumably the deprecation of the stable ABI itself would require a two-release waiting period. But maybe it would be worth it, given how subtle it is to do the historical research about even a single function.
PEP 384 is a historical document, but before 3.10 it was the best
available documentation. PEP 652 sort of changed the rules mid-course
(ref. https://www.python.org/dev/peps/pep-0652/#backwards-compatibility).
But for _PyObject_GC_Malloc specifically, IMO the speedup is worth it.
Go ahead and remove it.
It's gone. Mark landed his change earlier today.