Greg Ewing wrote:

Nick Coghlan wrote:

...

Note that my example actually *does* modify the class as it goes along (for exactly the reason you give - next() has to be defined on the class or the interpreter will ignore it).

Sorry, I didn't notice you were doing that.

But this doesn't seem useful to me. What if you have two instances of Tricky in use at the same time? They're going to interfere with each other.

I suppose you could make it work if you created a new class each time.

I provided a more complete example in another message that showed how to put the "start" value idiom in a helper function so it could be used with any iterator: def start(iterable, start): itr = iter(iterable) class TrapFirstNext(object): def __iter__(self): return self def next(self): TrapFirstNext.next = itr.next return start # Should also set send and throw to # itr.send and itr.throw if they exist return TrapFirstNext()

...

...

for val in start(range(2), "Hello World!"): ... print val ... Hello World! 0 1

But my earlier comment stands -- I don't want to preclude optimizing the common case for the sake of an uncommon case.

My main concern with allowing next() to be cached is that none of the existing looping constructs (for loop, comprehensions, generator expressions) have ever cached the bound method, so doing it in yield-from would make the new expression an odd special case (e.g. the for loop above would work, but using start() in a yield-from that cached the bound method would result in an infinite loop). That said, the existing language reference doesn't actually *say* that an implementation isn't allowed to cache the bound next() method in a for loop - it's just a long-standing convention that CPython has done the lookup every time around the loop. If this turns out to be "too slow" for coroutine setups that use send(val) rather than next(), then a better optimisation may be to turn send() and throw() into proper optional elements of the iterator protocol and give them their own slots on the type object. Deviating from existing looping behaviour in order to allow caching seems like a worse option, even if the details of the current behaviour were originally just an implementation accident. Also, for the simple case where send() and throw() aren't used we want: yield from subiter to do the same thing as: for x in subiter: yield x For CPython, that currently means no caching of the next() method in yield-from, since caching it would break the equivalence with the latter expansion. I think allowing next() to be cached in looping constructs should be pulled out of the PEP and turned into a separate PEP seeking explicit clarification from Guido as to whether the lack of caching of next() is part of the definition of looping in the language, or whether it is just an implementation detail of CPython that shouldn't be relied on. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Greg Ewing wrote:

Nick Coghlan wrote:

...

Note that my example actually *does* modify the class as it goes along (for exactly the reason you give - next() has to be defined on the class or the interpreter will ignore it).

Sorry, I didn't notice you were doing that.

But this doesn't seem useful to me. What if you have two instances of Tricky in use at the same time? They're going to interfere with each other.

I suppose you could make it work if you created a new class each time.

I provided a more complete example in another message that showed how to put the "start" value idiom in a helper function so it could be used with any iterator: def start(iterable, start): itr = iter(iterable) class TrapFirstNext(object): def __iter__(self): return self def next(self): TrapFirstNext.next = itr.next return start # Should also set send and throw to # itr.send and itr.throw if they exist return TrapFirstNext()

...

...

for val in start(range(2), "Hello World!"): ... print val ... Hello World! 0 1

But my earlier comment stands -- I don't want to preclude optimizing the common case for the sake of an uncommon case.

My main concern with allowing next() to be cached is that none of the existing looping constructs (for loop, comprehensions, generator expressions) have ever cached the bound method, so doing it in yield-from would make the new expression an odd special case (e.g. the for loop above would work, but using start() in a yield-from that cached the bound method would result in an infinite loop). That said, the existing language reference doesn't actually *say* that an implementation isn't allowed to cache the bound next() method in a for loop - it's just a long-standing convention that CPython has done the lookup every time around the loop. If this turns out to be "too slow" for coroutine setups that use send(val) rather than next(), then a better optimisation may be to turn send() and throw() into proper optional elements of the iterator protocol and give them their own slots on the type object. Deviating from existing looping behaviour in order to allow caching seems like a worse option, even if the details of the current behaviour were originally just an implementation accident. Also, for the simple case where send() and throw() aren't used we want: yield from subiter to do the same thing as: for x in subiter: yield x For CPython, that currently means no caching of the next() method in yield-from, since caching it would break the equivalence with the latter expansion. I think allowing next() to be cached in looping constructs should be pulled out of the PEP and turned into a separate PEP seeking explicit clarification from Guido as to whether the lack of caching of next() is part of the definition of looping in the language, or whether it is just an implementation detail of CPython that shouldn't be relied on. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------