Raise StopIteration with same value on subsequent `next`s
Consider the following code, which creates a generator that immediately returns 1, and then catches the StopIteration twice. def generatorfunction(): if False: yield return 1 def get_return(gen): try: next(gen) except StopIteration as e: return e.value else: raise ValueError("Generator was not ready to stop.") gen = generatorfunction() get_return(gen) #=> 1 get_return(gen) #=> None The first time StopIteration is raised, it contains the returned value. If StopIteration is forced again, the value is missing. What about keeping the return value for subsequent raises? Perhaps as an attribute on the generator object? The main disadvantage is that it'd add another reference, keeping the return value alive as long as the generator is alive. However, I don't think you'll want to keep a dead generator around anyway. Note: Javascript, which added generators, agrees with the Python status quo: the return value is only available on the first stop. C# does not have the concept of returning from an iterator. ---- Background: I made a trampoline for a toy problem, using generators as coroutines to recurse. When it came time to memoize it, I had to couple the memoization with the trampoline, because I could not cache the answer before it was computed, and I could not cache the generator object because it would not remember its return value later. I would have attached the returned value to the generator object, but for some reason, generator and coroutine objects can't take attributes. Maybe I should ask for that feature instead. Either feature would allow the concept of a coroutine that is also a thunk.
IIRC we considered this when we designed this (PEP 380) and decided that hanging on to the exception object longer than necessary was not in our best interest. On Mon, Dec 11, 2017 at 1:10 AM, Franklin? Lee < leewangzhong+python@gmail.com> wrote:
Consider the following code, which creates a generator that immediately returns 1, and then catches the StopIteration twice.
def generatorfunction(): if False: yield return 1
def get_return(gen): try: next(gen) except StopIteration as e: return e.value else: raise ValueError("Generator was not ready to stop.")
gen = generatorfunction() get_return(gen) #=> 1 get_return(gen) #=> None
The first time StopIteration is raised, it contains the returned value. If StopIteration is forced again, the value is missing.
What about keeping the return value for subsequent raises? Perhaps as an attribute on the generator object? The main disadvantage is that it'd add another reference, keeping the return value alive as long as the generator is alive. However, I don't think you'll want to keep a dead generator around anyway.
Note: Javascript, which added generators, agrees with the Python status quo: the return value is only available on the first stop. C# does not have the concept of returning from an iterator.
----
Background: I made a trampoline for a toy problem, using generators as coroutines to recurse.
When it came time to memoize it, I had to couple the memoization with the trampoline, because I could not cache the answer before it was computed, and I could not cache the generator object because it would not remember its return value later.
I would have attached the returned value to the generator object, but for some reason, generator and coroutine objects can't take attributes. Maybe I should ask for that feature instead.
Either feature would allow the concept of a coroutine that is also a thunk. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
What about hanging onto just the value, and creating new StopIteration instances instead of raising the same one again? On Mon, Dec 11, 2017 at 3:06 PM, Guido van Rossum <guido@python.org> wrote:
IIRC we considered this when we designed this (PEP 380) and decided that hanging on to the exception object longer than necessary was not in our best interest.
On Mon, Dec 11, 2017 at 1:10 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
Consider the following code, which creates a generator that immediately returns 1, and then catches the StopIteration twice.
def generatorfunction(): if False: yield return 1
def get_return(gen): try: next(gen) except StopIteration as e: return e.value else: raise ValueError("Generator was not ready to stop.")
gen = generatorfunction() get_return(gen) #=> 1 get_return(gen) #=> None
The first time StopIteration is raised, it contains the returned value. If StopIteration is forced again, the value is missing.
What about keeping the return value for subsequent raises? Perhaps as an attribute on the generator object? The main disadvantage is that it'd add another reference, keeping the return value alive as long as the generator is alive. However, I don't think you'll want to keep a dead generator around anyway.
Note: Javascript, which added generators, agrees with the Python status quo: the return value is only available on the first stop. C# does not have the concept of returning from an iterator.
----
Background: I made a trampoline for a toy problem, using generators as coroutines to recurse.
When it came time to memoize it, I had to couple the memoization with the trampoline, because I could not cache the answer before it was computed, and I could not cache the generator object because it would not remember its return value later.
I would have attached the returned value to the generator object, but for some reason, generator and coroutine objects can't take attributes. Maybe I should ask for that feature instead.
Either feature would allow the concept of a coroutine that is also a thunk. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
On Mon, Dec 11, 2017 at 6:21 PM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
What about hanging onto just the value, and creating new StopIteration instances instead of raising the same one again?
Doesn't really matter, as we're still prolonging the lifespan of the returned object. To me the current behaviour of generators seems fine. For regular generator users this is a non-problem. For trampolines and async frameworks--so many of them have been implemented and all of them worked around this issue in one way or another.
I would have attached the returned value to the generator object, but for some reason, generator and coroutine objects can't take attributes. Maybe I should ask for that feature instead.
You can use a WeakKeyDictionary to associate any state with a generator; that should solve your problem. We wouldn't want to add __dict__ to generators to have another way of doing that. Yury
On Mon, Dec 11, 2017 at 6:44 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
On Mon, Dec 11, 2017 at 6:21 PM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
What about hanging onto just the value, and creating new StopIteration instances instead of raising the same one again?
Doesn't really matter, as we're still prolonging the lifespan of the returned object.
As I said, I don't see a problem with this. How often does one accidentally hold on to an exhausted generator that they no longer have a need for? How often do those held generators have return values? How often are the generators held past the life of their returned values? And if one holds an exhausted generator on purpose, but doesn't need the returned value, what _is_ needed from it? I suspect it's a rarity of rarities.
To me the current behaviour of generators seems fine. For regular generator users this is a non-problem. For trampolines and async frameworks--so many of them have been implemented and all of them worked around this issue in one way or another.
For regular generator users, there is also little harm, as most simple generators won't return values. (Yes, I know the burden of proof is on the one looking to change the status quo.) I haven't figured out if there's a nice way to compose trampolines and memoization. I don't know if it's possible. Are there async frameworks that implement the concept of a coroutine which is also a thunk? I admit again, my use for it is not common. Even for myself. Maybe someone else has a better use?
I would have attached the returned value to the generator object, but for some reason, generator and coroutine objects can't take attributes. Maybe I should ask for that feature instead.
You can use a WeakKeyDictionary to associate any state with a generator; that should solve your problem. We wouldn't want to add __dict__ to generators to have another way of doing that.
I used a regular dict, which was discarded when the total computation was finished, and extracted the function args to use as the key. However, I had to memoize within the trampoline code, and could not use a separate memoization decorator.
participants (3)
-
Franklin? Lee -
Guido van Rossum -
Yury Selivanov