[Python-ideas] Introduce collections.Reiterable

[Steven D'Aprano]

On Fri, Sep 20, 2013 at 06:18:47AM -0400, Neil Girdhar wrote:
> On Fri, Sep 20, 2013 at 5:48 AM, Steven D'Aprano <steve at pearwood.info>wrote:
> 
> > On Thu, Sep 19, 2013 at 11:02:57PM +1000, Nick Coghlan wrote:
> > > On 19 September 2013 22:18, Steven D'Aprano <steve at pearwood.info> wrote:
> > [...]
> > > > At the moment, dict views aren't directly iterable (you can't call
> > > > next() on them). But in principle they could have been designed as
> > > > re-iterable iterators.
> > >
> > > That's not what iterable means. The iterable/iterator distinction is
> > > well defined and reflected in the collections ABCs:
> >
> > Actually, I think the collections ABC gets it wrong, according to both
> > common practice and the definition given in the glossary:
> >
> > http://docs.python.org/3.4/glossary.html
> 
> 
> Where does the glossary disagree with collections.abc?

I show below a class that is iterable, yet is not an instance of 
collections.Iterable. By the glossary definition it is iterable (it has 
a __getitem__ method that raises IndexError when there are no more items 
to be returned).


[...]
> What you're calling "indirectly iterable" is what the docs call "Iterable"
> and what collections.abc call Iterable, right?

I've explained this further in my reply to Paul Moore. What I should 
have said was *manually* iterable, in the sense of directly calling 
__next__ or __getitem__ on the view.


Here's an example of an iterable class that collections.Iterable claims 
is not an iterable:

> > py> class Seq:
> > ...     def __getitem__(self, index):
> > ...             if 0 <= index < 5: return index+1000
> > ...             raise IndexError
> > ...
> > py> s = Seq()
> > py> isinstance(s, Iterable)
> > False
> > py> list(s)  # definitely iterable
> > [1000, 1001, 1002, 1003, 1004]
> >
> 
> PEP 3119 makes it clear that isinstance( collections.Sequence) is the de
> facto way of checking whether something is a sequence.

I'm not testing whether it is a sequence. I explicitly stated it isn't a 
sequence, since it doesn't implement __len__. The Sequence ABC gets this 
right.


> Casting to list is not the de facto way.

No, but casting to list demonstrates that it can be iterated over. In my 
reply to Paul, I explicitly used it in a for-loop.


> Therefore, Seq is neither Iterable nor a Sequence
> according to collections.abc.

I'm not concerned by Sequence. It's not a Sequence. No dispute there. 
But it is an iterable, since it obeys the sequence protocol and can be 
iterated over. (Which is not the same as being a sequence.)


> > (Note that although Seq obeys the sequence protocol, and is can be
> > iterated over, it is not a fully-fledged Sequence since it has no
> > __len__.)
> >
> 
> I guess we disagree that Seq obeys the sequence protocol.

I'm not sure why you think it doesn't obey the sequence protocol. It is 
demonstrably true that it does. If it wasn't obvious from the source 
code, it should be obvious from a few seconds' experimentation at the 
interactive interpreter:

py> s = Seq()
py> s[0]
1000
py> s[1]
1001
[...cut s[2], s[3], s[4] for brevity...]
py> s[5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __getitem__
IndexError


That's all there is to the sequence protocol, and it's enough to make 
Seq objects iterable.


-- 
Steven