[Python-Dev] Private header files (Was: Renaming Include/object.h)

["Martin v. Löwis"]

Neal Norwitz schrieb:
> I'm a Python embedder and I want to know what's available to me.  I
> look in Include and see a ton of header files.  Do I need all these?
> What do I *need* and what can I *use*?  I only want to see the public
> stuff that is available to me.  Thus I want anything that has
> internal/implementation details/etc out of my sight to reduce my
> learning curve.  I don't ever want to learn about something I won't
> need nor include anything I won't need.

Ideally, you should look at the documentation. It should have
everything you can use, and only that.

I know that this currently doesn't work, for two reasons:
- the documentation is incomplete, i.e. it omits things that people
  do use on a regular basis
- people read header files, even if the documentation was complete

> Another part of my mental model is that I'm a Python developer and I'm
> modifying a header file that is implementation specific.  I need to
> share it among different subdiretories (say Python and Objects).  So I
> really need to stick the header file in a common place, Include/ is
> it.

There could be another standard directory for header files also, like
Python.

> I don't want to export anything, but I don't know if other third party
> developers will use the header or not.

I find that debatable. I see no reason not to export everything, unless
a stable ABI is also an objective (which it may be). If people really
want to get to the internals, they can access structures even if they
aren't defined in a header file, by replicating the structure definition
in their own code.

There should be a "mostly stable" API, certainly, and this is the one
that is documented. Other things in the header files aren't "internal",
they are just undocumented and thus may change without notice.

> I want clear rules on when identifiers need to be prefixed.

Always, unless they are static functions. Even an "internal" function
must be prefixed, as you may otherwise get conflicts when embedding
Python. At the moment, only the AST exports symbols that aren't properly
mangled.

[internal symbols]
> I can also change it in a point release.  If anyone uses anything from
> here, they are on their own.

No. There shouldn't be any possible ABI breakage in a point release.

> Finally, by putting everything in directories and always including a
> directory in the header file, like:
> 
>  #include "python/python.h"
>  #include "python/internal/foo.h"
> 
> There can never be an include file name collision as what started this
> thread.  It also provides a simple way of demonstrating what's public
> and what is not.  It addresses all my complaints.  There are only a
> few rules and they are simple.

As specified, above, it is incompatible with the current API. I think

#include <Python.h>

should be preserved. I personally see no problem with a single header
file, and would prefer that include to indicate all API that we are
committed to document and keep "mostly stable".

Regards,
Martin