[Python-Dev] inconsistent __abstractmethods__ behavior; lack of documentation

[Guido van Rossum]

On Sat, Aug 6, 2011 at 5:58 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 8/6/2011 8:29 AM, Guido van Rossum wrote:
>
>> Do you realize that __xxx__ names can have any semantics they darn
>> well please?
>
> That does not seem to be to be the issue Cristoff raised.

I apologize, I was too fast on this one. My only excuse is that
Christoph didn't indicate he was trying to figure out what another
Python implementation should do -- only that he was "playing with ABCs
and looking at their implementation". Looking around more I agree that
*for implementers of Python* there needs to be some documentation of
__abstractmethods__; alternatively, another Python implementation
might have to provide a different implementation of abc.py.

>> If a particular __xxx__ name (or some aspect of it) is
>> undocumented that's not a bug (not even a doc bug), it just means
>> "hands off".
>
> "__abstractmethods__" is used in the stdlib at least in abc.py:
>
> 95 class ABCMeta(type):
> ...
> 116 def __new__(mcls, name, bases, namespace):
> ...
> 123 for name in getattr(base, "__abstractmethods__", set()):
> 124 value = getattr(cls, name, None)
> 125 if getattr(value, "__isabstractmethod__", False):
> 126 abstracts.add(name)
> 127 cls.__abstractmethods__ = frozenset(abstracts)
>
> Since this module implements a PEP (3119) and is not marked as CPython
> specific, it should run correctly on all implementations.

I wouldn't draw that conclusion.

IMO its occurrence as a "pure-Python" module in the stdlib today says
nothing about how much of it is tied to CPython or not. That can only
be explained in comments, docstrings or offline documentation. (Though
there may be some emerging convention that CPython-specific code in
the stdlib must be marked in some way that can be detected by tools,
I'm not aware that much progress has been made in this area. But
admittedly I am not the expert here.)

> So implementors
> need to know what the above means. (

Here I agree.

> The doc to abc.py invites readers to read this code:
> **Source code:** :source:`Lib/abc.py`

<aybe that was an unwise shortcut.

> For both reasons, this attribute appears to be part of Python rather than
> being private to CPython. If so, the special name *should* be documented
> somewhere.

I'm happily to agree in this case, but I disagree that you could
conclude all this from the evidence you have so far shown.

> If it should *never* be used anywhere else (which I suspect after seeing
> that it is not used in numbers.py), that could be said.
>
> "__abstractmethods__: A special attribute used within ABCmeta.__new__ that
> should never be used anywhere else as is has a special-case effect for this
> one use."
>
> The problem with intentionally completely not documenting names publicly
> accessible in the stdlib code or from the interactive interpreter is that
> the non-documentation is not documented, and so the issue of documentation
> will repeatedly arise. The special names section of 'data model' could have
> a subsection for such. "The following special names are not documented as to
> their meaning as users should ignore them." or some such.

I disagree. If you see a __dunder__ name which has no documentation
you should simply refrain from using it, and no harm will come to you.
There is a clearly stated rule in the language reference saying this.
It also documents some specific __dunder__ names that are significant
for users and can be used in certain specific ways (__init__,
__name__, etc.). But I see no reason for a requirement to have an
exhaustive list of undocumented __dunder__ names, regardless if they
are supposed to be special for CPython only, for all Python versions,
for the stdlib only, or whatever.

-- 
--Guido van Rossum (python.org/~guido)