[Python-ideas] Propagating StopIteration value

[Jan Kaliszewski]

Hello .*

On 09.10.2012 17:28, Serhiy Storchaka wrote:
> On 09.10.12 16:07, Oscar Benjamin wrote:
>> I really should have checked this before posting but I didn't have
>> Python 3.3 available:
>
> Generator expression also eats the StopIteration value:
>
>>>> next(x for x in f())
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> StopIteration
[snip]

1.
Why shouldn't it "eat" that value? The full-generator equivalent, even 
with `yield from`, will "eat" it also:

     >>> def _make_this_gen_expr_equivalent():
     >>>     yield from f()  # or:  for x in f(): yield x
     ...
     >>> g = _make_this_gen_expr_equivalent()
     >>> next(g)
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     StopIteration

After all, any generator will "eat" the StopIteration argument unless:
* explicitly propagates it (with `return arg` or `raise 
StopIteration(arg)`), or
* iterates over the subiterator "by hand" using the next() builtin or 
the __next__() method and does not catch StopIteration.

2.
I believe that the new `yield from...` feature changes nothing in the 
*iterator* protocol.

What it adds are only two things that can be placed in the code of a 
*generator*:
1) a return statement with a value -- finishing execution of the 
generator + raising StopIteration instantiated with that value passed as 
the only argument,
2) a `yield from subiterator` expression which propagates the items 
generated by the subiterator (not necessarily a generator) + does all 
that dance with __next__/send/throw/close (see PEP 380...) + caches 
StopIteration and returns the value passed with this exception (if any 
value has been passed).

Not less, not more. Especially, the `yield from subiterator` expression 
itself* does not propagate* its value outside the generator.

3.
The goal described by the OP could be reached with a wrapper generator 
-- something like this:

     def preservefrom(iter_factory, *args, which):
         final_value = None
         subiter = iter(which)
         def catching_gen():
             nonlocal final_value
             try:
                 while True:
                     yield next(subiter)
             except StopIteration as exc:
                 if exc.args:
                     final_value = exc.args[0]
                 raise
         args = [arg if arg is not which else catching_gen()
                 for arg in args]
         yield from iter_factory(*args)
         return final_value


Example usage:

     >>> import itertools
     >>> def f():
     ...     yield 'g'
     ...     return 1000000
     ...
     >>> my_gen = f()
     >>> my_chain = preservefrom(itertools.chain, 'abc', 'def', my_gen, 
which=my_gen)
     >>> while True:
     ...     print(next(my_chain))
     ...
     a
     b
     c
     d
     e
     f
     g
     Traceback (most recent call last):
       File "<stdin>", line 2, in <module>
     StopIteration: 1000000
     >>> my_gen = f()
     >>> my_filter = preservefrom(filter, lambda x: True, my_gen, 
which=my_gen)
     >>> next(my_filter)
     'g'
     >>> next(my_filter)
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     StopIteration: 1000000


Best regards.
*j