[Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

[Terry Reedy]

On 7/15/2013 12:17 AM, Nick Coghlan wrote:

> You'd be surprised how many  non-core devs react with astonishment
> when I suggest that not documenting something isn't enough to avoid
> having users consider it a supported public API - they usually get it
> after I point out how far you can usually get just by using dir() and
> help() to play with a new library at the interactive prompt instead of
> looking at out-of-band docs. I figure including the second part will
> help prevent some "But why?" reactions in the future.

I agree with the goal of preventing such reactions, but I suggest 
slightly different means to do so.

I just looked and idlelib.__init__.py consists of
"""# Dummy file to make this a package.
"""
help(idlelib) prints the comment (which surprised me, but it seems to be 
smart enough to look for a 'doc comment' when it does not find a doc 
string). I now plan to change the comment to a docstring so 
'idlelib.__doc__' also works and change the text to something like

"""A package of private modules that implement the IDLE integrated shell 
and editor application. See the manual for the public entry points.""" 
(or maybe list them briefly here in the doc string).

This should properly inform people who introspect.

Also, the manual needs a small, indexed, section on idlelib that says 
much the same thing along with documenting what *is* public (which is 
not completely clear to me at the moment).

> I think Steven has a reasonable point about being clearer that an
> explicit leading underscore is the preferred solution for any new
> private modules, though, so here's an updated proposal:

The public/private distinction is not always as clean-cut as seems to be 
assumed in this discussion. Test and idlelib both have public entry 
points to an application based on private files. ('Running the Python 
test suite' is a batch application.)

So I do not think either should be _ prefixed. For both, and anything 
similar, a proper docstring and manual entry explaining the nuances 
should be completely acceptable as a substitute for a blunt, 
all-or-nothing flag.

I also think that marking something private, either way, or partially 
private with words, should extend to its contents. In particular, the 
100s of implementation files in test and idlelib need not be _ marked as 
as they would be covered once test and idlelib are properly documented. 
The special _ mark makes sense when it is the exception, but not in a 
context where it would be the overwhelming norm, because everything but 
one or two files is private.

> =================
> Private interfaces
>
> Unless explicitly documented otherwise, a leading underscore on any
> name indicates that it is an internal implementation detail and any
> backwards compatibility guarantees do not apply. It is strongly
> encouraged that private APIs (whether modules, classes, functions,
> attributes or other names) be clearly marked in this way, as Python
> users may rely on introspection to identify available functionality
> and may be misled into believing an API without a leading underscore
> is in fact a public API with the standard backwards compatibility
> guarantees.
>
> While the explicit use of a leading underscore in the names of private
> modules is preferred, all test modules and all modules that are not
> explicitly covered in the documentation are also considered private
> interfaces, even when their names do not start with a leading
> underscore and even if they include a module level documentation
> string. This includes submodules of packages that are documented as if
> they were a single module.

Since I am planning to explicitly cover idlelib in the doc, to list the 
public interface, just as is done for test, I would like it explicitly 
mentioned along with test.

I do not think absence of documentation is a good signal. I would rather 
say that any private or partially private package that does not have a _ 
name *must* have a docstring saying that it is all or mostly  private. 
(Or be a subpackage or module of something so marked.) And presuming 
that it is associated with something that is documented, then the doc 
should state that the implementation files are private.

In other words, I suggest that "modules that are not explicitly covered 
in the documentation ... [whose] names do not start with a leading 
underscore" should be empty. And I think "even if they include a module 
level documentation string." is backwards. Such files *should* have a 
docstring that serves as a substitute for a _ prefix.

> Similarly, the specific modules and external APIs imported by a module
> are always considered an implementation detail. Other modules should
> not rely on indirect access to such imported interfaces unless they
> are an explicitly documented part of the containing module's API (such
> as ``os.path`` or a package ``__init__`` module that exposes
> functionality from submodules).
> =================

-- 
Terry Jan Reedy