[docs] [issue18558] Iterable glossary entry needs clarification

Terry J. Reedy report at bugs.python.org
Mon Jul 17 15:43:02 EDT 2017


Terry J. Reedy added the comment:

The problem with the Iterable ABC is that 'iterable' and 'iterator' are *dynamically* defined, with a possibly infinite time required to possibly destructively check either definition.  In general, an algorithmic *static* check can only guess whether an object is iterable, though humans analyzing enough code can potentially get it right.  Therefore, using isinstance(ob, Iterable) is not 100% reliable, and in my opinion *should not be used* as the definition of lower-case 'iterable'.

Definition: Object ob is iterable if 'iter(ob)' returns an iterator. For the reasons given above, iter may return a non-iterator, but it will if ob implements either the old or new iterator protocol.  If ob has .__iter__, iter returns ob.__iter__().  If ob has .__getitem__, iter returns iterator(ob), where iterator is a hidden internal class that embodies the old iterator protocol by defining a .__next__ method that calls .__getitem__.  In both cases, iter does the best it can by assuming that the methods are correctly written as per one of the two protocols.

Loose definition: Object 'it' is iterable if it can be looped over.
Python definition: Object 'it' is iterable if repeated 'next(it)' calls either return an object or raise StopIteration.  This means that

try:
    while True:
        next(it)
except StopIteration:
   pass

runs, possibly forever, without raising.

As Raymond noted, an iterator can be created multiple ways: IteratorClass(), iter(ob), iter(func, sentinal), generator_func().
---

Iterable versus iter with respect to classes with __getitem__:

Iter was added in 2.2.  Built-in iterables were only gradually converted from old to new protocol, by adding a new .__iter__.  So even ignoring user classes, iter *had* to respect .__getitem__.  Even today, though only a small fraction of classes with .__getitem__ are iterable, people do not generally call iter() on random objects.  

Iterable (added 2.6) is documented as the "ABC for classes that provide the __iter__() method."  In other words, isinstance(ob, Iterable) replaces hasattr(ob, '__iter__').  Except that the former is more than that.  The magic word 'register' does not appear in the collections.ABC doc, and I think that this is the omission to be remedied.

"ABC for classes that provide the __iter__() method, or that provide a __getitem__ method that implements the old iterator protocol and register themselves as Iterable."

An example could be given using a patched version of IsIterable.

If one adds two lines of code

from collections.abc import Iterable
...
Iterable.register(IsIterable)

then isinstance(IsIterable(3), Iterable) is True, except that this is a lie in the other direction.

Traceback (most recent call last):
  File "F:\Python\mypy\tem.py", line 17, in <module>
    for i in it2:
  File "F:\Python\mypy\tem.py", line 7, in __getitem__
    return self.data[key]
TypeError: 'int' object is not subscriptable

Either IsIterable.__init__ must check that data itself has .__getitem__ or IsIterable.__next__ must capture exceptions and raise IndexError instead.

        def __getitem__(self, key):
            try:
                return self.data[key]
            except Exception:
                raise IndexError

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18558>
_______________________________________


More information about the docs mailing list