when an iterable object is exhausted or not

[Steven D'Aprano]

On Sat, 04 Aug 2012 21:20:36 +0200, Franck Ditter wrote:

> Two similar iterable objects but with a different behavior :
[...]
> IMHO, this should not happen in Py3k. What is the rationale of this (bad
> ?) design, which forces the programmer to memorize which one is
> exhaustable and which one is not ?...

What makes you say that they are "similar" iterable objects? Except that 
they are both iterable, they are very different. You might as well say 
that lists and dicts are "similar iterable objects".

filter objects are iterators, and so obey the intentionally simple 
iterator protocol. range objects are iterable but not iterators, and do 
not obey the iterator protocol.

py> it = filter(lambda x: x, set('abc'))
py> iter(it) is it
True
py> x = range(1, 15, 2)
py> iter(x) is x
False


filter relies on its input, which it consumes as it does. Since the input 
may not be restartable, filter cannot be restartable either. For 
simplicity, filter does not try to be "clever" and guess when the input 
argument is restartable. Instead, it simply and consistently behaves the 
same for any iterable input.

range is a specialist iterable data structure that does not consume 
anything. It computes its output lazily, but that is the only similarity 
with iterators. There's no need to limit range objects to the simple 
iterator protocol, since they don't consume their input -- a single 
object can easily compute its output as often as you want. range objects 
are indexable and sliceable:

py> x = range(1, 15, 2)
py> x[4]
9
py> x[2:4]
range(5, 9, 2)


Why artificially make range objects unrestartable just to satisfy 
compatibility with iterators?

The caller already has to remember that range and filter take different 
arguments, do different things, and return different objects. Why is it 
hard to remember that range is restartable and filter is not?


-- 
Steven