[Python-Dev] Difference between Include/internal and Include/cpython ?

[Victor Stinner]

Hi Antoine,

The rules to decide what goes where have been discussed in the issues which
created Include/cpython/ and the issue moving more headers to
Include/internal/.

In short, internal/ should not be used outside CPython codebase. In Python
3.7, these headers were even not installed. I chose to install them because
I moved more headers into internal/ which is a backward incompatible
change. You should not use these headers outside CPython code base, but the
typical use case to use them are debug tools: debugger, tracer and
profiler. The internal/ subdir is not included in Python default search
path when you use python-config --cflags for example. It is a deliberate
choice that these headers are not easily accessible.

There file names are prefixed by pycore_ for practical reasons: if 2 header
files have the same name in internal/ and Include/, the C preprocessor can
pick the wrong one. See the internal/ issue which gives a concrete example
(but in Python 3.7).

cpython/ is just a practical separation to force developers to decide if a
new API is part of the stable API or not. Previously, too many APIs have
been added to the stable API by mistake (not on purpose).

About inconsistencies, I invite you to open issues. I worked by small
steps. I tried to not move too much code from "one API" (stable, cpython,
internal) to another. IMHO all _Init() and _Fini() APIs must be internal.
For historical reasons, they are even part of the public API (!) which is a
mistake. I don't see the point of calling them explicitly.

I tried to take notes at https://pythoncapi.readthedocs.io/ for the
rationale, examples and track progess, but I didn't update this site with
the work I did last 6 months.

I hope that it makes more sense to you now?

Victor


Le dimanche 3 février 2019, Antoine Pitrou <solipsis at pitrou.net> a écrit :
>
> Hello,
>
> Can someone explain why we have two separate directories
> Include/internal and Include/cpython?  What is the rule for declaring an
> API inside one or another?
>
> At first sight, it seems to me we're having gratuitous complication
> here.  For example, I notice that PyFloat_Fini() is declared in
> Include/cpython/pylifecycle.h but PyLong_Fini() is declared in
> Include/internal/pycore_pylifecycle.h?
>
> (and why the additional "pycore_XXX.h" naming convention for some of
> those files?)
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
>

-- 
Night gathers, and now my watch begins. It shall not end until my death.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190203/1a603779/attachment.html>