[Python-ideas] Generators are iterators

[Andrew Barnert]

On Dec 14, 2014, at 0:58, Steven D'Aprano <steve at pearwood.info> wrote:

> In an effort to try to keep this already too-long thread a bit 
> shorter, I'm going to trim drastically.
> 
> On Sat, Dec 13, 2014 at 09:43:02PM -0800, Andrew Barnert wrote:
>> On Dec 13, 2014, at 17:17, Steven D'Aprano <steve at pearwood.info> wrote:
> 
>>> I am still unclear as to what this mystery unnamed thing is. It's not a 
>>> generator function, because that is called a generator function. It's 
>>> not the thing you get when you call a generator function, because that 
>>> is called a generator or an iterator, depending on context.
>> 
>> Again, it's the thing used by the generator instance's __next__ 
>> method: the frame and code objects.
> 
> Aha, now the penny drops. Given a generator object, is has attributes 
> holding a frame object and a code object (which it shares with the 
> generator function that created it):
> 
> py> def gen(): yield 1
> ...
> py> it = gen()
> py> it.gi_frame
> <frame object at 0xb7b8f95c>
> py> it.gi_code
> <code object gen at 0xb7b6e660, file "<stdin>", line 1>
> py> it.gi_code is gen.__code__
> True
> 
> Both have standard names: "frame object" and "code object" respectively.

Sure, but the problem is that people have been writing things that come "from the generator's internals", and then trying to explain how those things "make generators different from iterators" (or "... from explicit __next__ methods"). Because of the wording that was being used, people were looking for some way in which generators, or the generator type, or generator functions, are different from iterators, from other iterator types, or whatever, when none of that is at issue; it's the suspended generator frame, which doesn't have any parallel in other iterator implementations. I was thinking that if we had some term like "generator state" to refer to the frame-and-code (or just the frame, because the code isn't really necessary), the confusion wouldn't have arisen.

Now I notice that Nick Coghlan has been referring to "the generator frame", which is a nice way to avoid that confusion. What I was basically suggesting is that someone should rewrite the explanations exactly the way Nick already had been rewriting them. :)

>> I don't know how else to explain it. Look at the members stored by the 
>> generator type: a frame and a code object.
> 
> That's exactly the concrete example I was looking for.

That was in the message you replied to. I think the problem isn't that I didn't give the example you were looking for, it's that (at least in that message) I didn't explain why you (or Chris or anyone else) should care about it, so you passed right over it. Sorry for not making that part clear.