inconsistent __abstractmethods__ behavior; lack of documentation
Hi, while playing with abstract base classes and looking at their implementation, I've stumbled across the following issue. With Python 3.2, the script class Foo(object): __abstractmethods__ = ['boo'] class Bar(object): pass Bar.__abstractmethods__ = ['boo'] f = Foo() b = Bar() produces the following output Traceback (most recent call last): File "/home/cwg/test2.py", line 9, in <module> b = Bar() TypeError: Can't instantiate abstract class Bar with abstract methods buzz This seems to violate PEP 3119: it is not mentioned there that setting the __abstractmethods__ attribute already during class definition (as in "Foo") should have no effect. I think this happens because CPython uses the Py_TPFLAGS_IS_ABSTRACT flag to check whether a class is abstract. Apparently, this flag is not set when the dictionary of the class contains __abstractmethods__ already upon creation. As a second issue, the special __abstractmethods__ attribute (which is a feature of the interpreter) is not mentioned anywhere in the documentation. If these are confirmed to be bugs, I can enter them into the issue tracker. Christoph
Christoph, Do you realize that __xxx__ names can have any semantics they darn well please? If a particular __xxx__ name (or some aspect of it) is undocumented that's not a bug (not even a doc bug), it just means "hands off". That said, there may well be a bug, but it would be in the behavior of those things that *are* documented. --Guido On Sat, Aug 6, 2011 at 7:55 AM, Christoph Groth <cwg@falma.de> wrote:
Hi,
while playing with abstract base classes and looking at their implementation, I've stumbled across the following issue. With Python 3.2, the script
class Foo(object): __abstractmethods__ = ['boo'] class Bar(object): pass Bar.__abstractmethods__ = ['boo'] f = Foo() b = Bar()
produces the following output
Traceback (most recent call last): File "/home/cwg/test2.py", line 9, in <module> b = Bar() TypeError: Can't instantiate abstract class Bar with abstract methods buzz
This seems to violate PEP 3119: it is not mentioned there that setting the __abstractmethods__ attribute already during class definition (as in "Foo") should have no effect.
I think this happens because CPython uses the Py_TPFLAGS_IS_ABSTRACT flag to check whether a class is abstract. Apparently, this flag is not set when the dictionary of the class contains __abstractmethods__ already upon creation.
As a second issue, the special __abstractmethods__ attribute (which is a feature of the interpreter) is not mentioned anywhere in the documentation.
If these are confirmed to be bugs, I can enter them into the issue tracker.
Christoph
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
Guido, thanks for the quick reply! Of course I am aware that __xxx__ names are special. But I was assuming that the features of a python interpreter which are necessary to execute the pure python modules of the standard library are supposed to be documented. Christoph
On 8/6/2011 8:29 AM, Guido van Rossum wrote:
Do you realize that __xxx__ names can have any semantics they darn well please?
That does not seem to be to be the issue Cristoff raised.
If a particular __xxx__ name (or some aspect of it) is undocumented that's not a bug (not even a doc bug), it just means "hands off".
"__abstractmethods__" is used in the stdlib at least in abc.py: 95 class ABCMeta(type): ... 116 def __new__(mcls, name, bases, namespace): ... 123 for name in getattr(base, "__abstractmethods__", set()): 124 value = getattr(cls, name, None) 125 if getattr(value, "__isabstractmethod__", False): 126 abstracts.add(name) 127 cls.__abstractmethods__ = frozenset(abstracts) Since this module implements a PEP (3119) and is not marked as CPython specific, it should run correctly on all implementations. So implementors need to know what the above means. ( The doc to abc.py invites readers to read this code: **Source code:** :source:`Lib/abc.py` For both reasons, this attribute appears to be part of Python rather than being private to CPython. If so, the special name *should* be documented somewhere. If it should *never* be used anywhere else (which I suspect after seeing that it is not used in numbers.py), that could be said. "__abstractmethods__: A special attribute used within ABCmeta.__new__ that should never be used anywhere else as is has a special-case effect for this one use." The problem with intentionally completely not documenting names publicly accessible in the stdlib code or from the interactive interpreter is that the non-documentation is not documented, and so the issue of documentation will repeatedly arise. The special names section of 'data model' could have a subsection for such. "The following special names are not documented as to their meaning as users should ignore them." or some such. -- Terry Jan Reedy
On Sat, Aug 6, 2011 at 5:58 PM, Terry Reedy <tjreedy@udel.edu> wrote:
On 8/6/2011 8:29 AM, Guido van Rossum wrote:
Do you realize that __xxx__ names can have any semantics they darn well please?
That does not seem to be to be the issue Cristoff raised.
I apologize, I was too fast on this one. My only excuse is that Christoph didn't indicate he was trying to figure out what another Python implementation should do -- only that he was "playing with ABCs and looking at their implementation". Looking around more I agree that *for implementers of Python* there needs to be some documentation of __abstractmethods__; alternatively, another Python implementation might have to provide a different implementation of abc.py.
If a particular __xxx__ name (or some aspect of it) is undocumented that's not a bug (not even a doc bug), it just means "hands off".
"__abstractmethods__" is used in the stdlib at least in abc.py:
95 class ABCMeta(type): ... 116 def __new__(mcls, name, bases, namespace): ... 123 for name in getattr(base, "__abstractmethods__", set()): 124 value = getattr(cls, name, None) 125 if getattr(value, "__isabstractmethod__", False): 126 abstracts.add(name) 127 cls.__abstractmethods__ = frozenset(abstracts)
Since this module implements a PEP (3119) and is not marked as CPython specific, it should run correctly on all implementations.
I wouldn't draw that conclusion. IMO its occurrence as a "pure-Python" module in the stdlib today says nothing about how much of it is tied to CPython or not. That can only be explained in comments, docstrings or offline documentation. (Though there may be some emerging convention that CPython-specific code in the stdlib must be marked in some way that can be detected by tools, I'm not aware that much progress has been made in this area. But admittedly I am not the expert here.)
So implementors need to know what the above means. (
Here I agree.
The doc to abc.py invites readers to read this code: **Source code:** :source:`Lib/abc.py`
<aybe that was an unwise shortcut.
For both reasons, this attribute appears to be part of Python rather than being private to CPython. If so, the special name *should* be documented somewhere.
I'm happily to agree in this case, but I disagree that you could conclude all this from the evidence you have so far shown.
If it should *never* be used anywhere else (which I suspect after seeing that it is not used in numbers.py), that could be said.
"__abstractmethods__: A special attribute used within ABCmeta.__new__ that should never be used anywhere else as is has a special-case effect for this one use."
The problem with intentionally completely not documenting names publicly accessible in the stdlib code or from the interactive interpreter is that the non-documentation is not documented, and so the issue of documentation will repeatedly arise. The special names section of 'data model' could have a subsection for such. "The following special names are not documented as to their meaning as users should ignore them." or some such.
I disagree. If you see a __dunder__ name which has no documentation you should simply refrain from using it, and no harm will come to you. There is a clearly stated rule in the language reference saying this. It also documents some specific __dunder__ names that are significant for users and can be used in certain specific ways (__init__, __name__, etc.). But I see no reason for a requirement to have an exhaustive list of undocumented __dunder__ names, regardless if they are supposed to be special for CPython only, for all Python versions, for the stdlib only, or whatever. -- --Guido van Rossum (python.org/~guido)
On Sun, Aug 7, 2011 at 10:45 PM, Guido van Rossum <guido@python.org> wrote:
On Sat, Aug 6, 2011 at 5:58 PM, Terry Reedy <tjreedy@udel.edu> wrote:
The problem with intentionally completely not documenting names publicly accessible in the stdlib code or from the interactive interpreter is that the non-documentation is not documented, and so the issue of documentation will repeatedly arise. The special names section of 'data model' could have a subsection for such. "The following special names are not documented as to their meaning as users should ignore them." or some such.
I disagree. If you see a __dunder__ name which has no documentation you should simply refrain from using it, and no harm will come to you. There is a clearly stated rule in the language reference saying this. It also documents some specific __dunder__ names that are significant for users and can be used in certain specific ways (__init__, __name__, etc.). But I see no reason for a requirement to have an exhaustive list of undocumented __dunder__ names, regardless if they are supposed to be special for CPython only, for all Python versions, for the stdlib only, or whatever.
Indeed, the way it tends to work out in practice (especially for pure Python code) is that the other implementations will just copy the internal details from CPython, and only if that turns out to be problematic for some reason will they suggest a clarification in the language reference to separate out the details of Python-the-language from CPython-the-implementation in a particular case. That doesn't happen very often, but when it does it generally seems to be because they want to do something more sane than what we do, but the more sane behaviour is hard for us to implement for some reason. Even more rarely such questions may expose an outright bug in the reference implementation (e.g. the operand precedence bug for sequence objects implemented in C that's on my to-do list for 3.3). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (4)
-
Christoph Groth
-
Guido van Rossum
-
Nick Coghlan
-
Terry Reedy