[Python-ideas] should there be a difference between generators and iterators?

[Bruce Frederiksen]

There are several points here that might or might not be adopted 
individually.

The specific prompt for this message is that I've run into a snag using 
generators with 'itertools.chain.from_iterable'.  I need to be able to 
control when the generators called by 'chain' get closed (so that their 
'finally' clauses are run).

Unfortunately, chain does not propagate 'close' (or 'throw' or 'send') 
back to the generators that it's using.

Nor, for that matter do any of the other tools in itertools, or the 
builtin map function (to my knowledge).

I propose the following.  Each of these is independent of the others, 
but all related to cleaning up how this works:

1.  All of the itertools and map (and I've no doubt left some others out 
here) be extended to propagate the extra generators methods: close, 
throw and send.

2.  That the 'for' loop be extended so that if an exception occurs 
within its body, it calls 'throw' on its iterable (if it has a throw 
method).

3.  That the 'for' loop be extended to call the 'close' method on its 
iterable (if it has a close method) when the loop terminates (either 
normally, with break, or with an exception).

4.  That a 'not_closing' builtin function be added that takes an 
iterable and shields it from a 'close' call.

5.  That 'close' and 'throw' be added to all iterables.


Motivation:

The motivation for each proposal is (by their number):

1.  This is the one that I'm specifically stuck on.  I was relying on 
garbage collection to do this, but this doesn't work in jython and 
ironpython...  Since the chain function is dealing with two iterables 
(an inner and outer iterable), I think that it makes sense for it to 
check whether each of these have the extra methods or not.  For example, 
the inner iterable may have a 'close', but not the outer iterable (or 
vise versa).  This shouldn't cause an error if 'close' is called on the 
chain.

This one, specifically, would be helpful for me to move to Python 3K; so 
the sooner the better!  (Please!)

2.  There has been some discussion here about extending 'for' loops that 
has touched on non-local continue/break capability.  If step 2 is 
provided, this capability could be provided as follows:

    class funky:
        ...
        def continue_(self): raise ContinueError(self)
        def break_(self): raise BreakError(self)
        def throw(self, type, value, tb):
            if issubclass(type, ContinueError) and value.who is self:
                return next(self)
            if issubclass(type, BreakError) and value.who is self:
                raise StopIteration

    top = funky(iterable1)
    for x in top:
        middle = funky(iterable2)
        for y in middle:
            bottom = funky(iterable3)
            for z in bottom:
                ...
                middle.continue_()

3.  But this is a problem with:

     for line in filex:
         if test1(line): break
     for line in filex:
         ...

     which brings us to:

4.  A solution to the above:

     for line in not_closing(filex):
         if test1(line): break
     for line in filex:
         ...

5.  I thought that I may as well throw this in for discussion...  This 
might cause some consternation to those who has written their own 
iterables...