[issue20663] Introduce exception argument to iter

[Terry J. Reedy]

Terry J. Reedy added the comment:

Ram, your opening post here is essentially a copy of your opening post on python-ideas, as if the long discussion there, involving about 6 serious discussants other than you, never happened. Instead of restarting the discussion from scratch, you need to summarize the previous discussion, including a proposed python equivalent for the revised iter() (to exactly pin down the api) and how much support the proposal got.

A couple of notes that might be additions to what I said before: If a collection is fixed during an iteration, then destructive iteration might as well be done, when possible, by normal iteration followed by deletion of the collection.

That leaves as use cases iterations where the collection is mutated during the iteration, as in breadth-first search. For many collections, like deques and hashables, mutation means that direct (normal) iteration with for is prohibited.  The current solution is to interleave exception-raising access and mutation within a try and while-True loop.

The following example is similar to a 'breadth-first search'. It uses a deque rather than a list to limit the maximum length of the collection to the maximum number of live candidates rather than the total number of candidates.

from collections import deque
d = deque((0,))
try:
  while True:
        n = d.popleft()
        print(n, len(d))
        if n < 5:
            d.extend((n+1, n+2))
except IndexError:
    pass

This prints 25 items, with a max len before the pop of 11.

Under one variation of the proposal, the try-while block would be replaced by 

for n in iter(d.popleft, None, IndexError):
        print(n, len(d))
        if n < 5:
            d.extend((n+1, n+2))

Is the difference enough to add a parameter to iter?  Perhaps so. It reduces boilerplate and is a little easier to get right.  It eliminates there question of whether the try should be inside or outside the loop. It also matches

d = [0]
for n in d:
    print(n, len(d))
    if n < 5:
        d.extend((n+1, n+2))

which processes the same items in the same order, but extends the list to 25 rather than a max of 11 items. It makes deques and sets look more like direct replacements for lists.

Ram: If you program in Python, you should be able to write a test. To start, replace the prints above with out = [] ... out.append((n, len(d))) and assert that the out lists of the current and proposed deque loops are the same.

Raymond: I started this post with a recommendation to close but changed my mind after actually writing out the two deque examples. The fact that people rarely relegate the try - while True loop to a separate function (which would often be used just once) does not mean that the pattern itself is rare. Just yesterday or so, someone asked on python-list about how to search a graph when the set of candidate nodes got additions and 'for item in set' would not work. He was given the try - while True pattern as the answer.

I think iter(func, ... exception) would be more useful than iter(func, sentinel) is now. The problem with the latter is that for general collections, the sentinel needs to be a hidden instance of object() so that it cannot be placed in the collection and become a non-special legal return value. It is then inaccessible to pass to iter.  To signal 'no return', Python often raises an exception instead of returning a special object.

----------
stage:  -> test needed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20663>
_______________________________________