[Python-3000] Builtin iterator type

[Guido van Rossum]

On 11/18/06, Andrew Koenig <ark-mlist at att.net> wrote:
> I've just come up with an idea that I think may help clarify this discussion
> in a new way.  I'm afraid the description is going to be a little long; but
> I'd appreciate it if you would bear with me and read the whole thing.
>
> I'd like to start by coining a usage, namely the "abilities" of a type.  By
> an ability, I mean a claim such as "Objects of type Foo are iterators" or
> "Objects of type Foo are copyable."  I would like to use the word "ability"
> to refer to "iterator" and "copyable" in this context.
>
> When we say that a an object of a given class (or the class itself) has an
> ability, we are really saying that objects of that class behave in a
> particular way when used in a particular way.  So, for example, when we say
> that x is an iterator (or x has the iterator ability), we mean that we can
> call x.next() to obtain successive values of a sequence, and that doing so
> for long enough will eventually raise StopIteration.
>
> So far, this discussion has been about the idea that a class should be able
> to acquire an ability by inheriting it.  For quite a while, I thought this
> was an excellent idea; but now I'm not so sure.  Here's why.
>
> I'll begin by explaining why I think it's an excellent idea.  Suppose you
> have an object x of unknown type, and you want to know whether x is an
> iterator.  Some people in this discussion have suggested that the Pythonic
> way to answer this question is to try to use it as an iterator and see if it
> raises an exception.  The trouble with this technique is that it is
> destructive if it succeeds, because it consumes an element of the sequence.
> That means you can't use the technique until you have something to do with
> that first element, which rules out some plausible usages.
>
> I've raised this issue in discussion before, and gotten the response that
> the right way to deal with this problem is to use getattr(x,"next").  Doing
> so is certainly a way of determining whether x.next() is meaningful, but it
> doesn't offer a general way of telling whether an object has a given
> ability.  For example, there is no analogous technique for determining
> whether len(x) will work without evaluating it.  Nor is there a way to know
> whether someone just happened to define x.next() without intending to
> implement the iterator protocol.
>
> A good bit of the current discussion is about a way to solve this kind of
> problem, namely to define types that represent abilities.  So, for example,
> if "iterator" were such a type, we could evaluate isinstance(x,iterator) to
> determine whether x is of a type that is intended to behave as an iterator.
> I've proposed this idea myself a while ago, and had been quite fond of it.
> Yes, there is an objection to this scheme, namely that existing user-defined
> iterator types would not automatically inherit from the "iterator" type; but
> maybe there is a way around that problem.
>
> However, I want to argue against this idea for a completely different
> reason: Inheritance should establish an "is-a" relationship, and that's not
> the right kind of relationship here.  To put it bluntly:
>
>         I can see no reason why the iterator type should
>         have the iterator ability!
>
> More generally, I see no reason why a type that signals the presence of a
> given ability should itself possess that ability.  Because if it does, then
> it becomes difficult to distinguish types that signal abilities from types
> that do not.
>
> In other words:  Suppose that we have a family of types with names such as
> "iterator", "copyable", "equality_comparable", "comparable", and so on.
> Each of these types signals the corresponding ability.  Some of these types
> may be related by inheritance.  For example, I would expect "comparable" to
> inherit from "equality_comparable", because any type that is comparable had
> better be comparable for equality.
>
> It seems to me to make sense for the notion of "signaling an ability" to be
> an ability.  In other words, the types in the last paragraph can all be used
> in a particular way, which is the hallmark of an ability.
>
> Now, suppose that every type that signals the presence of an ability
> inherits from "ability".  Then we've just broken the whole ability system.
> Because if "iterator" inherits from "ability" and every iterator type
> inherits from "iterator", then every iterator is also able to signal the
> presence of an ability--and we most definitely do not want that!
>
> In other words, whether or not we choose to define a family of types that
> stand for particular abilities, I think we should not use inheritance as the
> mechanism for specifying that a particular class has a particular ability.
> I don't know what the mechanism should be, but it shouldn't be inheritance.
>
> Am I missing something?
>
>                                                         --Andrew Koenig

This looks like you're rediscovering or reinventing Java- or
Zope-style interfaces: those are not passed on via inheritance but via
a separate "implements" clause. I don't remember how it's done in Java
(and don't have time to look it up) but it makes sense to me to use
something akin to inheritance for constructing interfaces out of other
interfaces, like your example of comparable inheriting from
equality-comparable. But "being an interface" is not transmitted
through this same mechanism -- that's a meta-property. I think that it
would be a mistake to us isinstance to test for an interface. I
believe Zope already solves most of the deeper problems; the main
issue seems to be that the surface syntax for creating and using
interfaces isn't very elegant (due to lack of language support) and
that the standard types and common de-facto interfaces (e.g. sequence,
mapping, iterator, file) don't play along and to some extent defy
capturing their semantics in interfaces.

I realize that Java- and Zope-style interfaces *seem* to be all about
syntax (they define names and signatures of methods but not semantics)
but IMO that's only a theoretical objection; in *practice* these
interfaces have strong connotations of semantics (albeit written in
Enlish rather than using assertions or pre- and post-conditions) and
nobody would think of claiming to implement an interface without
implementing the proper semantics (or some interpretation thereof :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)