[Tutor] iterators

[Oscar Benjamin]

On 19 January 2014 12:55, spir <denis.spir at gmail.com> wrote:
>
> 'range' ('xrange' in python2) is certainly (at least in my view) a kind of
> iterator in the latter, more general sense used in programming (some thing
> providing items one at a time); however, it does not implement python's
> iterator protocal. Thus, it cannot be used directly in a 'for' traversal
> loop: if i'm right, python builds a python iterator for ranges in the
> background. Like all other kinds of 'sequences' (in the python sense, again)
> ranges are traversable ("iteratable") because they can in principle provide
> items one at a time, and there exist builtin iterators for them.

It's not really that complicated. Basically range on 3.x (or xrange on
2.x) returns a range object:

$ python3
Python 3.3.2+ (default, Oct  9 2013, 14:56:03)
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = range(2, 4)
>>> a
range(2, 4)

A range object is not an "iterator":

>>> next(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'range' object is not an iterator

However it is an "iterable" which means that I can call iter on it to
get an "iterator":

>>> b = iter(a)
>>> b
<range_iterator object at 0xb70ef770>

Once we have the iterator object we can call next() on it:

>>> next(b)
2
>>> next(b)
3

The distinction between the iterator and the iterable is important
since it affects what happens if we call iter() multiple times:

>>> c = iter(a)
>>> next(c)
2
>>> next(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> next(c)
3
>>> next(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Iterators (as opposed to iterables) must be their own iterator. So
when you call iter you get back the same object:

>>> iter(c) is c
True
>>> iter(c) is b
False
>>> iter(c) is a
False

This means that if we call iter several times we don't start from the
beginning of the sequence of values that come from the iterator.

>>> a = range(9)
>>> a
range(0, 9)

When we iterate over the range object we always get the same values:

>>> for x in a:
...   if x > 3:
...     break
...   print(x)
...
0
1
2
3
>>> for x in a:
...   if x > 5:
...     break
...   print(x)
...
0
1
2
3
4
5

When we iterate over the range_iterator it remembers where it left off:

>>> b = iter(a)
>>> for x in b:
...   if x > 3:
...     break
...   print(x)
...
0
1
2
3
>>> for x in b:
...   if x > 5:
...     break
...   print(x)
...
5

It's important to know whether you have an iterable or an iterator. If
you write a function that should work with either then it's sometimes
necessary to explicitly call iter at the start:

def myfunc(iterable):
    iterator = iter(iterable)
    # Do stuff with iterator

If you just loop over all the values in the iterable once then this is
unneeded. If your iteration pattern is more complicated then you may
need this. This is important if e.g. you have a function that should
work equally well with a file object or with a list of strings or a
generator that yields strings etc. The list object is an iterable but
not an iterator. File objects and generators are iterators.

> For iterators, in python there is additional confusion with generators (at
> term which AFAIK in programming means either about the same as iterator, or
> a subclass of iterators implemented using poor man's coroutines), precisely
> generator objects; and with generator expressions and other comprehensions.
>
> A bit exaggerately complicated, in my view, esp when considering the
> narrowness of the application field. Maybe a case of over-abstraction or
> over-engineering?

What's the narrowness of the application field? Iteration is
fundamental in pretty much every program I write. Iterators and
generators are great at writing scalable programs without needing to
complicate your code. Try doing a similar thing in C where you have to
write call-back functions or functions to allocate and hold on to
iteration state and so on.


Oscar