[Python-Dev] yield back-and-forth?

[Alex Martelli]

On Jan 20, 2006, at 1:39 PM, Phillip J. Eby wrote:

> At 11:19 AM 01/20/2006 -0800, Guido van Rossum wrote:
>> (There *are*other uses besides the trampoline,
>> right? :-)
>
> It's easy to come up with use cases where you feed data *into* a  
> generator
> (parsers and pipelines, for example).  I just don't know of any
> "simultaneous bidirectional" uses other than in association with a
> coroutine scheduler, or a dedicated pipelining tool.  In both the  
> coroutine

Hmm, what about an iterator of iterators, where the sub-iterators  
need to be advanced until some condition is satisfied, then processed  
from that point onwards (only those subiterators for which some item  
did satisfy the condition).  E.g.:

goodfiles = []
for afile in manyfiles:
   for aline in afile:
     if aline.startswith('*starthere*'):
       goodfiles.append(afile)
       break
for afile in goodfiles: ...

I'm sure we've all met this kind of issue (indeed, standard library  
module fileinput does support this case directly, by wrapping the  
double loop into one yielding lines, and still providing ways to get  
the current file and skipping to the next file).

It might be nice to wrap the general logic (double nested looping &c)  
in a generator leaving the specifics (exactly when to consider a  
subiterator 'good' and what to do in that case) out of it, sort of  
like fileinput does but with more generality than just files and  
lines thereof.  In Python 2.5 it seems that a .send call with a non- 
None argument might be a natural convention for the loop body to tell  
the generator "abandon this subiterator and yield it, then move to  
the next one".  E.g.:

goodfiles = []
allinput = inputfrom(manyfiles)
for aline in allinput:
   if aline.startswith('*starthere*'):
     goodfiles.append(allinput.send(True))
for afile in goodfiles: ...

Perhaps not the greatest of simplifications, but it might be  
generalized to (e.g.) recursive walks with the same client-logic as  
above, while the original (nested for-loops) is pretty rigid.  And  
the inputfrom generator doesn't look too complicated either:

def inputfrom(iterators):
   for iterator in iterators:
     for item in iterator:
       if (yield item) is not None:
         yield iterator
         break

Wouldn't this be considered a reasonable use for the new generator  
features?


Alex