[Python-3000] Iterators for dict keys, values, and items == annoying :)

[Brett Cannon]

On 3/23/06, Guido van Rossum <guido at python.org> wrote:
> On 3/23/06, Brett Cannon <brett at python.org> wrote:
> > I think there is a fundamental difference between your views of
> > iterators.  It sounds like Ian is viewing them as a separate object;
> > something that happens to have derived its values from a dict in the
> > situation begin discussed.  While it seems Guido views the iterator
> > for the dict as a view (ala Java in a way) of the data and providing
> > an object with an API for viewing that data.
>
> Actually I'd say it's the opposite -- I *don't* see an iterator as a
> view, I see it as a throw-away way to access the elements of some
> underlying container in a sequential fashion. Sometimes the nature of
> the containing makes it impractical to inquire the container directly
> about its size (e.g. if the "container" represents the lines of a
> file, or a graph represented by the pages of a web server).
>

OK, I think view was the wrong word to choose then.  But your view of
iterators as providing a way to access the data of the underlying
container does match up with what I was thinking.

> Ian (or perhaps SQLobject) seems to collapse the notion of the
> iterator and the underlying container, and that causes the problems
> because the properties of the iterator are quite limited (all you can
> really do is ask for the next item, fingers crossed) while the
> container may have other metadata even if it can't tell you how many
> values the iteration will return.
>
> [SNIP]
>
> > I understand Ian's view since I know I like to pass around iterators
> > for use and that disconnects the iterator from the object that
> > generated it and thus makes it impossible to find out possible info on
> > the data contained without exhausting the iterator compared to just
> > performing data upon the object containing the original the data.
>
> You shouldn't do that unless you are consciously designing an API that
> must be able to work with in(de)finite sequences or other strange
> things. The itertools library is an example of such an API because (by
> intent) it must work for all iterators.
>
> Most APIs aren't as constrained and it's fine to require an iterable
> instead of an iterator.
>
> > But I think if objects returned iterators instead of lists the
> > iterator-as-view will begin to be used more than viewing them as
> > iterator-has-own-data.  But this also means that making a more
> > view-like interface would be handy.  In terms of what would need to be
> > supported (len, deletion, etc.) I don't know.  I personally have not
> > had that much of a need since I then just pass the originating object
> > and get the iterator as needed instead of passing around the iterator.
>
> I'm dead set against giving iterators more view-like properties; it
> would rule out generators and other potentially infinite sequences.
>

Good point.

> But I'm all for a different concept, views in the sense of Java's
> collection framework. Please study it; it's worth it:
>
> http://java.sun.com/j2se/1.4.2/docs/api/java/util/Collection.html
>

This looks like a container/set protocol to me.  Beyond having
stronger guarantees in terms of the deletion methods (would have them
always raise NotImplemented if they don't mutate the object they
represent) and dicthing Java-specific stuff (like the toArray() cruft)
this would be a nice complement to iterators.

If we formalize the mapping protocol we would have a specified API for
most major uses of data structures.  We would have a protocol for
detecting if an object contains something (container), a way to
linearly go over the the values of an object (iterator), or provide
random-access to values in an object (mapping).  And the container
protocol could have an optional location or key method that returns
the value associated with the wanted object if such a things exist (so
lists would return the index, dicts the key, and sets would not
implement the method).

-Brett