[Python-ideas] x=(yield from) confusion [was:Yet another alternative name for yield-from]

[Nick Coghlan]

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> Note that my example actually *does* modify the class as it goes along
>> (for exactly the reason you give - next() has to be defined on the class
>> or the interpreter will ignore it).
> 
> Sorry, I didn't notice you were doing that.
> 
> But this doesn't seem useful to me. What if you
> have two instances of Tricky in use at the same
> time? They're going to interfere with each other.

> I suppose you could make it work if you created
> a new class each time.

I provided a more complete example in another message that showed how to
put the "start" value idiom in a helper function so it could be used
with any iterator:

 def start(iterable, start):
     itr = iter(iterable)
     class TrapFirstNext(object):
       def __iter__(self):
         return self
       def next(self):
         TrapFirstNext.next = itr.next
         return start
       # Should also set send and throw to
       # itr.send and itr.throw if they exist
     return TrapFirstNext()


>>> for val in start(range(2), "Hello World!"):
...   print val
...
Hello World!
0
1

> But my earlier comment
> stands -- I don't want to preclude optimizing the
> common case for the sake of an uncommon case.

My main concern with allowing next() to be cached is that none of the
existing looping constructs (for loop, comprehensions, generator
expressions) have ever cached the bound method, so doing it in
yield-from would make the new expression an odd special case (e.g. the
for loop above would work, but using start() in a yield-from that cached
the bound method would result in an infinite loop).

That said, the existing language reference doesn't actually *say* that
an implementation isn't allowed to cache the bound next() method in a
for loop - it's just a long-standing convention that CPython has done
the lookup every time around the loop.

If this turns out to be "too slow" for coroutine setups that use
send(val) rather than next(), then a better optimisation may be to turn
send() and throw() into proper optional elements of the iterator
protocol and give them their own slots on the type object. Deviating
from existing looping behaviour in order to allow caching seems like a
worse option, even if the details of the current behaviour were
originally just an implementation accident.

Also, for the simple case where send() and throw() aren't used we want:

  yield from subiter

to do the same thing as:

  for x in subiter:
    yield x

For CPython, that currently means no caching of the next() method in
yield-from, since caching it would break the equivalence with the latter
expansion.

I think allowing next() to be cached in looping constructs should be
pulled out of the PEP and turned into a separate PEP seeking explicit
clarification from Guido as to whether the lack of caching of next() is
part of the definition of looping in the language, or whether it is just
an implementation detail of CPython that shouldn't be relied on.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------