Please comment on Draft PEP for Enhanced Generators

[Raymond Hettinger]

"Alex Martelli" <aleax at aleax.it> wrote in message
news:a3976j$qkt$1 at serv1.iunet.it...
> "Raymond Hettinger" <othello at javanet.com> wrote in message
> news:a38c1u$945$1 at bob.news.rcn.net...
> > I have written a draft PEP summarizing proposed enhancements to
> generators:
> > http://users.javanet.com/~othello/download/genpep.htm
> >
> > The draft reflects all the comments I've received to date on the threads
> > where
> > the ideas where first proposed.
> >
> > Please post your comments (and maybe a little encouragment) here on
> > comp.lang.py or email them directly to me.
>
> Neat!  One thing, I believe, needs fixing:
>
>
> def xmap( fun, *sequences ):
>     ...
>  for args in xzip( *sequences ):
>      yield fun(*args)
>
> has the wrong behavior when the sequences have different lengths.
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66448 has
> an implementation which I believe satisfies the specs (complicated,
> alas, but those specs ARE tricky;-).
>

Thanks for the pointer.  I'll include that alternative in the next draft of
the PEP.
The complexity of the code is a good argument for having a built-in as
opposed
to having everyone continually re-invent the wheel.

I have one misgiving -- padding shorter sequences with None makes sense
when all of the sequences are expected to be finite, but may not be the
desired
behavior when one of the sequences is potentially infinite. I would like to
xmap( None, xrange(sys.maxint), iter(f.readline,'') ) to return a finite
sequence
and respond to list(mygen) without crashing in an infinite loop.

>
> Why introduce a new method g.submit(x) rather than allow the
> existing g.next to take an optional arg?  I've missed the
> rationale for that.
>
Originally, I proposed using .next(value) but found two problems.

The critical and decisive problem was that generators behave differently
when x = yield None is used.  The initial invocation RUNs code in the
generator and stops when it hits a yield statement.  Then it waits for
a .submit().   The is different behavior from the non-assigned version
which does not RUN any code inside the generator until .next() is called.
I spent a week-end trying to get exactly parallel definitions but it was
futile.

The second problem arose, when I was creating sample code using the
format and found that I never had occassion to both send and receive data
to the generator.  Writing x=g.next(y) turned out to be confusing and more
than a little mind-blowing.  Let's see, execution returns to the generator,
binds y to a local, hmm, when does the argument to yield get passed back
to be bound to x, is it before y is assigned or after a full loop when the
next yield is encountered.

In other words, overloading the .next() method is a bug factory.

In my sample code, I found that .submit(y) had the added virtue of being
very clear in its intention to push data into a generator rather than
retrieve it.

>
> Wouldn't it be nice (without _forcing_ generators to be able
> to clone or rewind themselves) to standardize the ways in which
> a generator supplies such functionality IF it does?  E.g.,
> __copy__ (so one uses clonedgen = copy.copy(originalgen)) for
> cloning generator state if supported...
>
Now, YOU'RE blowing my mind.  I'm always amazed at the depth
of your insights.

>
> Alex
>


Raymond Hettinger