On Sun, 11 Apr 2021 at 10:25, Stéfane Fermigier <sf@fermigier.com> wrote:

> 2) What's a "public API" ? A particular library can have an internal API (with regular public functions and methods, etc,) but only willingly expose a subset of this API to its clients. __all__ doesn't help much here, as the tools I know of don't have this distinction in mind.

I don't think that this distinction is so important in practice. 

What matters is the boundary between different codebases. Within a codebase
a team can agree whatever conventions they want because any time
something is changed it can be changed everywhere at once in a single
commit. What matters is communicating from project A to all downstream
developers and users what can be expected to remain compatible across
different versions of project A.

> 3) A library author can decide that the public API of its library is the part exposed by its top-level namespace, e.g:
>
> from flask import Flask, resquest, ...
>
> where Flask, request, etc. are defined in subpackages of flask.

This approach really doesn't scale.

For one thing imports in Python
have a run-time performance cost and if everything is imported at
top-level like this then it implies that the top level __init__.py has
imported every module in the codebase. Also organising around
sub-packages and modules is much nicer for organising documentation,
module level docstrings etc.

> I guess this issue also comes from the lack of a proper way to define (either as a language-level construct or as a convention among developers and tools authors) a proper notion of a "public API", and, once again, I don't believe __all__ helps much with this issue.
>
> => For these use cases, a simple solution could be devised, that doesn't involve language-level changes but  needs a wide consensus among both library authors and tools authors:
>
> 1) Mark public API namespaces package with a special marker (for instance: __public_api__ = True).
>
> 2) Statical and runtime tools could be easily devised that raise a warning when:
>
> a) Such a marker is present in one or more modules of the package.
>
> b) and: one imports another module of the same package from another package.
>
> This is just a rough idea. Additional use cases could easily be added by adding other marker types.
>
> An alternative could be to use decorators (eg. @public like mentioned in another message), as long as we don't confuse "public = part of the public API of the library" with "public = not private to this particular module".

I don't think __all__ helps either but for different reasons.

Python makes everything implicitly public. What is actually needed is a way
to clearly mark the internal code (which is most of the code in a
large project) as being obviously private.

That's why I prefer the Jax approach of putting the implementation in
a jax._src package.