[Python-ideas] Add lookahead iterator (peeker) to itertools

[Terry Reedy]

On 2/25/2013 10:29 AM, Ron Adam wrote:
>
>
> On 02/24/2013 09:41 PM, Terry Reedy wrote:

>> class lookahead():
>>      "Wrap iterator with lookahead to both peek and test exhausted"
>>
>>      _NONE = object()
>>      def __init__(self, iterable):
>>          self._it = iter(iterable)
>>          self._set_peek()
>>      def __iter__(self):
>>          return self
>>      def __next__(self):
>>          ret = self.peek
>>          self._set_peek()
>>          return ret
>>      def _set_peek(self):
>>          try:
>>              self.peek = next(self._it)
>>          except StopIteration:
>>              self.peek = self._NONE
>>      def __bool__(self):
>>          return self.peek is not self._NONE
>
>>
>> def test_lookahead():
>>     it = lookahead('abc')
>>     while it:
>>         a = it.peek
>>         b = next(it)
>>         print('next:', b, '; is peek:', a is b )
>>
>> test_lookahead()

Since revised.

> I think with a few small changes I would find it useful.
>
> The key feature here is that the result is pre calculated and held until
> it's needed, rather than calculated when it's asked for.
>
> You should catch any exception and hold that as well.  On the next
> .next() call, it should raise the exception if there was one, or emit
> the value.

My revised version has

     def _set_peek(self):
         try:
             self.peek = next(self._it)
         except StopIteration:
             self.peek = self._NONE

That could easily be changed to e

         except Exception as e:
             self.peek = self._NONE
             self._error = e

__next would then raise self._error instead of explicitly StopIteration. 
I do not especially like the redundancy of two 'exhausted' indicators 
and thought of storing e as self.peek, but an iterator can legitimately 
yield exception classes and instances.

I think some argument can be made that if the iterator is broken, the 
exception should be raised immediately even if it means not returning 
the last item. No user should be expecting anything other than 
StopIteration.

> I'm not sure if using the __bool__ attribute is the best choice.

I am ;-). It hides the test for exhaustion, which could change, without 
complication.

> I would prefer a .error flag, along with a .next_value attribute.  It
> would make the code using it easier to follow.

Not clear to me, but a minor detail.

>       it.error        <--    True if next(it) will raise an exception.
>       it.next_value   <--    The next value, or the exception to raise.


> About it.error.  If it was a concurrent version, then it.error could
> have three values.
>
>       it.error == True   # Will raise an exception
>       it.error == False  # Will not raise an exception
>       it.error == None   # Still calculating
>
> I wonder how this type of generator will behave with "yield from".

lookaheads are iterators, but not generators and 'yield from' requires a 
generator. A generator function could recompute items to yield, but it 
cannot add methods or attributes to the generator instances it will produce.

-- 
Terry Jan Reedy