On May 9, 2020, at 03:46, Chris Angelico <rosuav@gmail.com> wrote:
But ultimately, a generator function is very similar to a class with a __next__ method. When you call it, you get back a state object that you can ping for the next value. That's really all that matters.
Well, it’s very similar to a class with __next__, send, throw, and close methods. But that doesn’t really change your point. For a different angle on this: What If Python 3.10 changed things so that every generator function actually did define a new class (maybe even accessible as a member of the function)? What would break? You could make inspect.isgenerator() continue to work, and provide the same internal attributes documented in the inspect module. So only code that depends on type(gen()) is types.GeneratorType would break (and there probably is very little of that—not even throwaway REPL code). Also: a generator isn’t actually a way of defining a class, but it’s a way of defining a factory for objects that meet a certain API, and Python goes out of its way to hide that distinction wherever possible (not just for generators, but in general). The only meaningful thing that’s different between a generator function and a generator class is that the author of the function doesn’t directly write the __next__ (and send, close, etc.) code, but instead writes code that defines their behavior implicitly. And that’s obviously just an implementation detail, and it isn’t that much different from the fact that the author of a @dataclsss doesn’t directly write the __init__, __repr__, etc. So you’re right, from outside, it really doesn’t matter.
I think the C implementations tend to be classes but the Python ones tend to be generators - possibly because a generator function is way easier to write in Python, but maybe the advantage isn't as strong in C.
It’s not just not as strong, it runs in the opposite direction. In fact, it’s impossible to write generator functions in C. There’s no way to yield control in a C function. (Even if you build a coro library around setjmp, or use C++20 coros, it wouldn’t help you yield back into CPython’s ceval loop.) A generator object is basically just a wrapper around an interpreter frame and its bytecode; there’s no way to exploit that from C. There are a few shortcuts to writing an iterator (e.g., when you have a raw array, implement the old-style sequence protocol, want to delegate to a member, or can steal another type’s implementation as frozenset does with set), but a generator function isn’t one of them. (If you’re curious how Cython compiles generators, it’s worth looking at what it produces—but doing the same thing in raw C would not be a shortcut to writing a generator class.)