[Python-3000] Abilities / Interfaces

[Guido van Rossum]

On 11/20/06, Guido van Rossum <guido at python.org> wrote:
> I'd like to have a discussion of abilities and interfaces but right
> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
> mostly isomorphic to Zope's notion of interfaces, but the new
> terminology may make it possible to clarify certain distinctions
> better, e.g. the distinction between a class that provides an ability
> to its instances, and the abilities that an instance has.
>
> Feel free to riff on this theme here. I'll check in within 24 hours or so.

OK, I have some more time for this now. I'm hoping Andrew, Bill and
Phillip will join the discussion here (instead of various other
threads).


A. Preliminaries.

There are some open issues that we should probably address before
diving too deep into the specification of abilities or interfaces.

1. Abstract Base Classes (ABCs)?

Are we sure that we have passed the station of ABCs, or should we
revisit why those aren't sufficient for our purposes?

Bill Janssen seems to be in favor of just using ABCs. Pro: less new
infrastructure. Con: hard to add a new ABC on the fly to an existing
3rd party class (mucking with __bases__ is too much of a hack to
seriously propose).

I'd like to explore this some more before committing to anything.

2. Do we need interfaces if we have generic functions?

Phillip Eby argues that we don't need interfaces, because the right
API for generic functions makes it easy to do what we do with
interfaces. Let me paraphrase his argument.

Suppose we have a generic function F with implementations F1, F2, ...
(generally: Fi for 1 <= i <= n) suitable for different kinds of
arguments. (Concrete example: flatten() implemented one way for
strings, another way for lists.) Suppose we have a class C and we want
to make it acceptable to F, using some implementation Fi. Using
interfaces, each Fi is declared to take an argument that has a
particular interface, and we bind C to a certain Fi by claiming that C
provides the interface that Fi takes. This also makes C instances
behave in a certain way for all other generic functions that mention
the same interface. Using Phillip's proposal, we select the right Fi
and bind it to the class C using the assignment F[C] = Fi. This
doesn't do anything for other generic functions that might also
conceivably work with C. Phillip addresses that by proposing help
functions that do a bunch of these bindings at once, and by noticing
that if you just use the standard library and generic functions, the
default implementation might just work.

Phillip then argues that he doesn't want to encourage introspectable
interfaces. I think others have use cases for those though. It seems
that Phillip's approach only hanges well together if everybody totally
adopts the generic functions religion; I think that's an unlikely
(since too radical) change of course for Python 3.0. It also doesn't
seem to work for abilities that are tied to an instance instead of to
a class as Zope allows (see below).

Assuming we go with abilities (of some sort), we have more concrete
things to argue over:


B. Developing a spec.

I see this as a number of steps, although there's feedback possible
between them. We can borrow heavily from Zope, and perhaps somewhat
from Java.

1. Naming and general ideas. Should we call these Abilities or
Interfaces? Abilities, the term proposed by Andrew Koenig, have less
baggage, but possibly they're just an isomorphic relabeling of
interfaces. From Zope comes the important question: Do we want to
discuss the interfaces/abilities of an instance separately from those
provided by its class? Zope's answer is Yes, reminds Jean-Paul
Calderone, and I tend to like this (having been exposed to it in the
past).

2. More precise concepts. I'd like to distinguish between checking for
abilities (which should be quick as it may be used many times during
execution, e.g. whenever a generic function is called) and verifying
an ability, which would entail checking the class to see whether it
actually implements the API (and possibly maintains the invariants, if
we go that route) stipulated by a claimed ability. It would be a
performance nightmare if during he normal course of an application we
would be *verifying* abilities over and over -- this is equivalent to
a structural type check and can be very expensive. (Even more so if we
have optional argument type declarations.)

3. API design -- how do we spell the various concepts? E.g.
has_ability(x, A) asks whether object x has the ability A;
provides_ability(C, A) asks whether class C provides the ability A to
its instances. We could state that provides_ability(C, A) and
isinstance(x, C) implies has_ability(x, A). (A corollary would be that
a subclass can't take away an ability provided by a base class!) I
think it would be handy to have APIs that ask a class or instance for
the set of abilities it provides or has, e.g. get_abilities(x) or
get_provisions(C). Should And probably also for APIs to add/remove
abilities dynamically (if not constrained by base classes). The
verification API is separate also.

Some other questions:

- We need to come up with syntax for the common case, declaring a
class that claims to provide some abilities. Some thoughts: putting
the interfaces in the list of base classes might work -- the metaclass
can easily tell whether a particular base is a real class or an
ability. Another idea might be to add keyword parameters to the class
declaration, so we could write e.g. class C(B1, B2, abilities=(A1,
A2)): ...

- Should has_ability(x, A) pass the responsibility of computing the
outcome to the object A, which could implement an arbitrarily complex
algorithm to decide? Antoine Pitrou proposes this. The downside is
that we won't able to implement get_abilities(x) properly, or
efficiently. I'm also not sure of the use case.

- If has_ability(x, A) is true, should isinstance(x, C) be true?

- How does this interact with generic functions? I think the generic
function mechanism (if we introduce one) will have to be able to use
either classes or abilities to trigger behavior; while abilities are
nice and general, one shouldn't be forced to declare a new ability
when all one cares about is one particular concrete class (this
happens often enough in applications I develop). Obviously this is an
area where Bill's ABC's score better.


C. A library of standard abilities and mapping of abilities to
standard objects (Sequence to list, etc.).

It's easy to come up with a strawman set of interfaces, e.g. Iterable,
Iterator, Collection, Set, Sequence, Mapping, String, MutableSet,
MutableSequence, MutableMapping, MutableString, and inheritance
relations between them, e.g. Iterable - Collection - Sequence -
MutableSequence - list.

But it's not so easy to define which APIs must be present to for
verification of a particular interface to pass. A Mapping must
implement __getitem__() in a certain way. But does it need to
implement __len__? get()? keys()? values() and items()? Is it okay for
a particular mapping implementation to allow only one iterator to
exist  (per instance) at any time, like dbm files do? Does a String
need to implement lower()? Must a Sequence implement __add__() as
concatenation?

Answering these and similar questions probably requires a
standardization committed. (Any volunteers?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)