[Python-ideas] Introduce collections.Reiterable

[Andrew Barnert]

On Sep 19, 2013, at 5:18, Steven D'Aprano <steve at pearwood.info> wrote:

> On Thu, Sep 19, 2013 at 07:12:26PM +1000, Nick Coghlan wrote:
> 
>> is there any obvious case where "iterable but
>> not an iterator" gives the wrong answer?
> 
> I'm not sure if it counts as "obvious", but one can write an iterator 
> that is re-iterable. A trivial example:
> 
> class Reiter:
>    def __init__(self):
>        self.i = 0
>    def __next__(self):
>        i = self.i
>        if i < 10:
>            self.i += 1
>            return i
>        self.i = 0
>        raise StopIteration
>    def __iter__(self):
>        return self
> 
> 
> I know that according to the iterator protocol, such a re-iterator 
> counts as "broken":

It also wouldn't break the OP's code, or any other reasonable code that cares about the distinction; at worst, it would cause it to unnecessarily make an extra list copy.

The only thing that would break the code is something that isn't an iterator, is an iterable, and can only be iterated once.

Of course you could build that as well, but it would be even more pathological than your example. For example:

class OneShot:
    def __init__(self, it):
        self.it = iter(it)
    def __iter__(self):
        return self.it

Besides being something no one should ever write, it's also something no code could ever guard against. If we had the Reiterable ABC, I could just register OneShot as Reiterable.

> Another example might be iterators with a reset or restart method, or 
> similar. E.g. file objects and seek(0). File objects are officially 
> "broken" iterators, since you can seek back to the beginning of the 
> file. I don't think that's a bad thing.

They can't be reiterated if you just treat them as iterators. You have to treat them as files--e.g., call seek(0)--if you want to reiterate them. So there's no way this could be a problem in any real code.

> But nor am I sure that it requires a special Reiterable class so we can 
> test for it.

Unless you added a "__reiter__" method, or some other way to get a new, reset-to-the-start, iterator from the iterable, such a class wouldn't help anyway. And even if we had that method, for loops and yield from and so on would all have to try __reiter__ first and fall back to __iter__. Otherwise, you still wouldn't be able to pass a file to the OP's code, or any other code that distinguishes on Reiterable to decide whether to copy or tee or use a one-pass algorithm instead of multi-pass or whatever.