[Python-Dev] pydoc for named tuples is missing methods

[Tim Lesher]

On Mon, Mar 14, 2011 at 08:28, Tim Lesher <tlesher at gmail.com> wrote:
> On Mon, Mar 14, 2011 at 05:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> There are two relatively simple ways forward I can see:
>>
>> A. Add a __public__ attribute that pydoc (and import *) understand.
>> This would overrride the default "this is private" detection and add
>> affected names to the public list (without needing to respecify all
>> the "default public" names - this is important in order to support
>> subclassing correctly)
>
> I believe this was the direction the bug report was implying.
>
> I'll be sprinting for a few hours this morning; if there are no
> objections, I will try to turn this idea into a patch that makes
> pydoc.visiblename look for a __public__ function attribute as "step
> 0".
>
> Maybe there should also be a @public decorator to apply it, although
> that name may be an attractive nuisance, tempting C++ or Java
> programmers new to Python to ask for @protected and @private...

I implemented this on Monday, and it worked great... for instance
methods.  Unfortunately, it doesn't do a thing for classmethods or
data attributes, which are 2/4 of the original namedtuple use cases,
so it feels like a bad idea.

In the process of implementing it, though, I realized that the __all__
mechanism for modules could still be reused.  Two implementations came
to mind:

1. Allow an __all__ class attribute that affects pydoc for classes the
same way it does for modules.
Pro: it's familiar, easy to explain, and easy to implement (four lines of code)
Con: the original use case (adding a small number of
otherwise-excluded attributes to existing documentation) is
cumbersome. Because __all__ means "don't document anything but
__special_names__ and the contents of __all__, you'd have to manually
add names that pydoc would normally pick up.  Also, __all__ itself
will show up in the documentation, which seems to me like useless
clutter to a reader of help().

2. Add a new class attribute that uses the same mechanism, with a
different name (e.g., __api__).
Pro: fairly easy to explain; because it doesn't share a name with
__all__ its semantics can be tweaked without confusing people.
Con: slight additional cognitive burden of a new feature

I'm implementing both of these separately this week to see which works
better in practice.  So far I'm liking __api__ better, with these
semantics:

1. Classes without __api__ are treated just as they are today.
2. __api__ on classes works just like __all__ on modules (only special
names, plus its contents are documented)
3. Additionally, __api__ can take an Ellipsis.  That makes the __api__
list additive--pydoc documents everything it normally would, *plus*
the contents of __api__
4. __api__ is added to pydoc's "hidden names" list; since its only
purpose is to help generate docs, its value would be of little use in
the generated docs themselves.

Point 3 (Ellipsis) is unusual, but to me it makes the declaration read
well and solves the namedtuple case succinctly and compatibly, without
changing a published API.  It also could be used to address the issue
of stdlib classes that have non-underscored members that really
shouldn't be considered part of the "public" API, but can't be renamed
now for fear of breaking code.

Usage example:

class C1:
    __api__ = ['meth1', '_data2'] # read as "The API consists of meth1
and _data2."
    def __init__(self):
        """documented because it's a special name"""
    def meth1(self):
        """documented because it's in __api__"""
    def meth2(self):
        """Shouldn't really have been exposed, but it was released that way,
           and we don't want to break users' code.
           NOT documented because it's not in __api__
           """
    def _meth3(self):
        """NOT documented because it's not in __api__"""
    data1 = None # not documented--not in __api__
    _data2 = None # documented--it's in __api__

class C2:
    __api__ = ['_data2', ...] # read as "The API includes _data2."
    def __init__(self):
        """This is documented because it's a special name"""
    def meth1(self):
        """documented because it follows the normal rules"""
    def meth2(self):
        """documented because it follows the normal rules"""
    def _meth3(self):
        """NOT documented because it's not in __api__ and starts with
underscore"""
    data1 = # documented because it follows the normal rules
    _data2 = None # documented--it's in __api__

Comments and criticisms welcome.

-- 
Tim Lesher <tlesher at gmail.com>