Idioms combining 'next(items)' and 'for item in items:'

[Terry Reedy]

Python's iterator protocol for an iterator 'items' allows combinations 
of explicit "next(items)" calls with the implicit calls of a "for item 
in items:" loop. There are at least three situations in which this can 
be useful. (While the code posted here is not testable, being incomplete 
or having pseudocode lines, I have tested full examples of each idiom 
with 3.2.)


1. Process first item of an iterable separately.

A traditional solution is a flag variable that is tested for each item.

first = True
<other setup>
for item in iterable:
   if first:
     <process first>
     first = False
   else:
     <process non-first>

(I have seen code like this posted on this list several times, including 
today.)

Better, to me, is to remove the first item *before* the loop.

items = iter(iterable)
<set up with next(items)
for item in items:
   <process non-first>

Sometimes <other setup> and <process first> can be combined in <setup 
with next(items)>. For instance, "result = []" followed by 
"result.append(process_first(item))" becomes "result = 
[process_first(item)]".

The statement containing the explicit next(items) call can optionally be 
wrapped to explicitly handle the case of an empty iterable in whatever 
manner is desired.

try:
     <set up with next(items)>
except StopIteration:
     raise ValueError("iterable cannot be empty")


For an iterable known to have a small number of items, the first item 
can be removed, with only a small copying penalty, with a simple assignment.

first, *rest = iterable

The older and harder to write version of this requires the iterable to 
be a sequence?

first, rest = seq[0], seq[1:]


2. Process the last item of an iterable differently. As far as I know, 
this cannot be done for a generic iterable with a flag. It requires a 
look ahead.

items = iter(iterable)
current = next(items)
for item in items:
   <process non-last current>
   current = item
<process last current>

To treat both first and last differently, pull off the first before 
binding 'current'. Wrap next() calls as desired.


3. Process the items of an iterable in pairs.

items = iter(iterable)
for first in items:
     second = next(items)
     <process first and second>

This time, StopIteration is raised for an odd number of items. Catch and 
process as desired. One possibility is to raise ValueError("Iterable 
must have an even number of items").

A useful example of just sometimes pairing is iterating a (unicode) 
string by code points ('characters') rather than by code units. This 
requires combining the surrogate pairs used for supplementary characters 
when the units are 16 bits.

chars = iter(string)
for char in chars:
     if is_first_surrogate(char):
         char += next(chars)
     yield char

-- 
Terry Jan Reedy