[Python-ideas] abc.optionalabstractmethod

[Nick Coghlan]

On Tue, Aug 7, 2012 at 12:18 PM, alex23 <wuwei23 at gmail.com> wrote:
> On Aug 7, 10:35 am, Steven D'Aprano <st... at pearwood.info> wrote:
>> But even putting that aside, the interface that it (implicitly?) documents is
>> surely *required* interface. If you can neglect to override an abstract method
>> with impunity, then it shouldn't have been an abstract method in the first place.
>
> This I completely agree with. I don't understand the point of
> declaring an abstract method that you cannot guarantee will be on an
> implementation.
>
> Wouldn't it make more sense to define the optional aspect as a
> secondary interface?

No. That leads to a combinatorial explosion of interface classes that
only makes sense in languages which don't readily support attribute
level introspection (*cough*such-as-Java*cough*).

Look at the IO stack for an example - think how much more complicated
it would need to be if implementations weren't allowed to raise
NotImplemented error for unsupported methods and each of those methods
thus had an associated ABC.

As I wrote in my earlier message, there are several ways to define a
runtime protocol to check for optional methods in Python today. The
problem is that *NONE* of them are particularly open to programmatic
introspection, thus it is difficult for automatic documentation tools
to flag them correctly the way they can flag abstract methods.

To avoid people having to dig up the alternatives (all of which are
used in the core or standard library in various situations):
- the method/attribute may be missing entirely
- the method/attribute is always present, but may be None
- the method is always present, but returns NotImplemented by default
- the method is always present, but raises NotImplementedError by default

The first case you can't indicate in an ABC at all (except in a comment)
The second case makes it difficult to add a docstring (although you
can use a property to get around that)
The latter two cases mean you have to actually *call* the method to
find out if if is implemented or not

So, here's a concrete suggestion that would be suitable for many cases
where this is desired (although obviously not all such cases, due to
backwards compatibility concerns, as well as cases where it is
desirable that the base implementation be suitable for termination of
a chain of cooperative super calls)

1. Add a __call__ definition to type(NotImplemented) that accepts
arbitrary arguments and always raises NotImplementedError
2. Add an @optional decorator that functions roughly as follows

    class optional:
        def __init__(self, f):
            functools.update_wrapper(self, f)
        def __get__(self, *args):
            return NotImplemented

A demo (using a custom "not implemented" marker and property rather
than a custom descriptor):

>>> def _not_implemented(*args, **kwds):
...     raise NotImplementedError
...
>>> def optional(f):
...     return property((lambda self: _not_implemented), doc=f.__doc__)
...
>>> class C:
...     @optional
...     def may_not_be_implemented(signature_details):
...         """Documentation of this method"""
...
>>> C.may_not_be_implemented.__doc__
'Documentation of this method'
>>> C.may_not_be_implemented
<property object at 0x7fa5cd06afc8>
>>> C().may_not_be_implemented
<function _not_implemented at 0x7fa5cd070e60>
>>> C().may_not_be_implemented is _not_implemented
True
>>> C().may_not_be_implemented()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in _not_implemented
NotImplementedError

That specific approach does have the problem that you lose the
signature details, so you'd probably want a custom descriptor that is
recognised by inspect.getsignature() rather than reusing the property
descriptor as I have done here. A custom descriptor would also be
easier to pick out on the class object without needing an instance.

Such an approach would improve the expressiveness of the ABC dialect,
while remaining broadly consistent with current practices for marking
optional methods. Most importantly, it would provide a fairly obvious
way to flag optional methods in APIs in a way that is open to static
introspection.

However, it's not a universal solution. As noted above, it's not
usable in any cases where you want the optional method to be usable as
a terminal for multiple inheritance.

It may be with some API tweaks, you could convert optional to a
decorator factory that also handles the "suitable foundation for
multiple inheritance" case. Perhaps the answer is even simpler than
what I have above: perhaps the decorator could just set
"f.__implemented__ = False", allowing introspection via
"getattr(method, '__implemented__', True)".

This is something that really needs to be explored before making a
concrete proposal. Looking at the various ways optional methods are
used in the standard library and interpreter core would be a good
place to start.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia