[Python-ideas] Generators are iterators

[Andrew Barnert]

On Dec 13, 2014, at 17:17, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sat, Dec 13, 2014 at 11:37:23AM -0800, Andrew Barnert wrote:
>> On Dec 13, 2014, at 5:36, Steven D'Aprano <steve at pearwood.info> wrote:
>> 
>>> On Fri, Dec 12, 2014 at 03:22:32PM -0800, Andrew Barnert wrote:
>>> 
>>>> In Python, the term "Iterator" is just as consistent and meaningful as 
>>>> in all these other languages. The fact that some people confuse 
>>>> iterables and iterators isn't a reason to abandon this simplicity.
>>> [...]
>>> 
>>> +1 to this.
>>> 
>>> 
>>> But:
>>> 
>>>> The things we need to be clear about here are the things that _dont't_ 
>>>> have an official name. In particular, the thing that's built from 
>>>> calling a generator function or evaluating a generator expression and 
>>>> used by the generator.__next__ method is not a generator, a generator 
>>>> function, a generator function body, an iterator, or anything else 
>>>> with a name in the language. And that's what's confusing people.
>>> 
>>> I don't know what thing you are referring to. If I write this:
>>> 
>>> py> def gen():
>>> ...     yield 1
>>> ...
>>> py> it = gen()
>>> 
>>> then `gen` is a function.
>> 
>> Chris Barker just asked exactly the same question yesterday, except 
>> that he trimmed the quote more and used "g" instead of "it" for his 
>> example. So I'll just paste the same reply, and you can manually 
>> s/g/it/.
> 
> I read your reply to Chris last night, and it didn't make sense to me 
> then and it still doesn't make sense to me now :-( Hence my request for 
> a concrete example.

You didn't ask for a concrete example, you provided the exact same answer as Chris but with different names. 

> I am still unclear as to what this mystery unnamed thing is. It's not a 
> generator function, because that is called a generator function. It's 
> not the thing you get when you call a generator function, because that 
> is called a generator or an iterator, depending on context.

Again, it's the thing used by the generator instance's __next__ method: the frame and code objects.

>> But you're not defining the generator type here--that already exists 
>> as a builtin type. What you're defining is a thing made up of a code 
>> object and a frame, which is used by that generator instance to do its 
>> generating. That thing is not a generator, or a generator type (or an 
>> iterator instance or type), or a __next__ method, or anything else 
>> with a name.
> 
> Sure the thing you are defining when you write "def gen() ..." is a 
> function? I thought we were in agreement about that. It's a function 
> with a flag set, a.k.a. a generator function.
> 
> If you're referring to something else, a concrete example will help. 

I don't know how else to explain it. Look at the members stored by the generator  type: a frame and a code object.

So, going back to the original point: Generator instances are not different from iterators (except that it's a subtype, of course). The builtin generator type is likewise no different than a custom iterator class. The generator body is a normal function body. The only thing that isn't like an iterator is the frame. Which is not an important distinction.

I think Chris was making a different distinction, claiming that generators are not an Iterator type (which they aren't, but only because they're _instances_ of an Iterator type, the builtin type generator), and I was pointing out that this can't be the distinction Guido was trying to make. But, again, since Guido has already said that sentence was mistaken, this really isn't important.

> Because as we stand now, I don't know whether we're talking about 
> something important or not, or if we are in agreement, disagreement, or 
> merely confusion :-)

I already said from the start that this is almost certainly not important.