[Python-ideas] Propagating StopIteration value

[Serhiy Storchaka]

On 09.10.12 01:44, Guido van Rossum wrote:
> I don't understand that code at all, and it seems to be undocumented
> (no docstrings, no mention in the external docs). Why is it using
> StopIteration at all? There isn't an iterator or generator in sight.
> AFAICT it should just use a different exception.

I agree with you. StopIteration is not needed here (or I don't 
understand that code), ValueError can be raised instead it. Perhaps the 
author was going to use it for the iterative parsing.

This is a bad example, but it is only real example which I have. I have 
also the artificial (but also imperfect) example:

def file_reader(f):
     while not f.eof:
         yield f.read(0x2000)

def zlib_decompressor(input):
     d = zlib.decompressobj()
     while not d.eof:
         yield d.decompress(d.unconsumed_tail or next(input))
     return d.unused_data

def bzip2_decompressor(input):
     decomp = bz2.BZ2Decompressor()
     while not decomp.eof:
         yield decomp.decompress(next(input))
     return decomp.unused_data

def detect_decompressor(input):
     data = b''
     while len(data) < 5:
         data += next(input)
     if data.startswith('deflt'):
         decompressor = zlib_decompressor
         data = data[5:]
     elif data.startswith('bzip2'):
         decompressor = bzip2_decompressor
         data = data[5:]
     else:
         decompressor = None
     input = itertools.chain([data], input)
     return decompressor, input

def multi_stream_decompressor(input):
     while True:
         decompressor, input = detect_decompressor(input)
         if decompressor is None:
             return input
         unused_data = yield from decompressor(input)
         if not unused_data:
             return input
         input = itertools.chain([unused_data], input)

Of cause this can be implemented without generators, using a class to 
hold a state.

> I think you're going at this from the wrong direction. You shouldn't
> be using this feature in circumstances where you're at all likely to
> run into this "problem".

I think that the new language features (as returning value from 
generators/iterators) will generated new methods of solving problems. 
And for these new methods will be useful to expand the existing tools. 
But now I see that it is still too early to talk about it.

> Itertools is for iterators, and all the extra generator
> features make no sense for it.

As said Greg, the question is whether returning a value with 
StopIteration is part of the iterator protocol or the generator protocol.