[Python-ideas] __iter__ implies __contains__?

[Nick Coghlan]

On Mon, Oct 3, 2011 at 5:07 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> Actually I think there is at least *some* trend in the opposite
>> direction -- we now have a much more refined library (and even
>> vocabulary) for "iterator algebra" than before iter() was introduced,
>> and a subgroup of the community who can easily whip out clever ways to
>> do things by combining iterators in new ways.
>
> Hmmm, not sure what to do about that.
>
> Maybe we should be thinking about a "reiterator algebra"
> to sit on top of the iterator algebra.
>
> For example, given two reiterables x and y, zip(x, y)
> would return a reiterable that, when iterated over, would
> extract iterators from x and y and return a corresponding
> iterator.

Can't be done in general, since one of the main points of iterator
algebra is that it works with *infinite* iterators.

Basically, my understanding is that iterators started life as a way of
generalising various operations on containers. This is reflected in
the "for x in y: assert x in y" symmetry currently ensured by the
respective definitions of the two variants of 'in'.

Once the iterator protocol existed, though, people realised it made
possible certain things that containers can't do (such as operating on
theoretically infinite data sets or data sets that won't fit in RAM
all at once).

The two domains are essentially disjoint (one assumes reiterability
and the ability to load the whole data set into memory, while the
latter denies both of those assumptions as invalid), but they share a
protocol and syntax. Often, list(itr) is used to ensure the first set
of assumptions holds true, while the latter is handled by carefully
ensure to iterate only once over supplied iterators and using tools
like itertools.tee() to preserve any state needed.

I'm not sure it's actually feasible to separate the two domains
cleanly at this late stage of the game, although the
collections.Container ABC may get us started down that path if we
explicitly register the various builtin containers with it (a
duck-typed check for __contains__ would break for classes that
explicitly raise TypeError, as is proposed for _BaseIO).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia