[Python-ideas] should there be a difference between generators and iterators?

[Bruce Frederiksen]

Greg Ewing wrote:
> Bruce Frederiksen wrote:
>
>> what is happening is that the 'finally' clause in a generator isn't 
>> being honored by 'for' statements.
>
> It's not all that clear whether the for-loop should be
> doing anything special to force a generator to finalize
> if the loop is exited prematurely. The for-loop isn't
> necessarily the only thing using the iterator -- other
> code may want to carry on getting items from it, in
> which case you *don't* want it forcibly terminated.
>
> More generally, I'm a bit worried by all the extra
> complications that generators seem to be accruing. If
> it becomes a general expectation that an iterator based
> on other iterators is supposed to pass on all these
> special conditions, it's going to put a big burden on
> implementors of iterators, and turn what ought to be
> very simple and straightforward code into something
> convoluted.
>
It doesn't seem that difficult to pass 'close' on to your child iterator.

Passing 'throw' on might require interpreting non-error results from the 
child iterator, which would depend on what your iterator is doing.  If 
there are places where this is too complicated, then don't implement 
'throw' for that iterator.

And the same thing for 'send' as for 'throw'.

But the most important one is 'close', because this runs the 'finally' 
clause in the generator.

Looking through the itertools, a user of any of these except 'chain' can 
capture the input iterable(s) in a with closing clause prior to calling 
the itertools function.  So for everything except the 'chain' function 
the 'close' is more of a "nice to have" rather than a "must have".

But for the 'chain' function, the caller of 'chain' does not have access 
to the iterables generated by the iterable passed to 'chain' 
(specifically with 'from_iterable').  Thus, adding 'close' to 'chain' is 
more of a "must have".  (Well, I guess you could argue that the user 
should write his own chain function with a 'close' capability rather 
than using the itertools function).

So 'chain' would become equivalent to:

def chain(*iterables):
    # chain('ABC', 'DEF') --> A B C D E F
    it = None
    try:
        for it in iterables:  # <---- 'it' is the inaccessible iterator!
            for element in it:
                yield element
    finally:
        if hasattr(it, 'close'): it.close()
        if hasattr(iterables, 'close'): iterables.close() 



-bruce