[Python-ideas] Generator unpacking

[Steven D'Aprano]

On Mon, Feb 15, 2016 at 10:02:26PM -0800, Andrew Barnert via Python-ideas wrote:

> Because "reiterable" is really not immediately obvious. It has no 
> intuitive meaning; it's not even a real word. You have to learn what a 
> reiterable is, by definition.

What's a "real word"? Is "closure" a real word? How about "iterator"?

"Closure" is in the Oxford dictionary, but it doesn't mention anything 
about functions and variables from surrounding scopes.

"Iterator" isn't in the Oxford dictionary at all; neither is "iterable".

As far as intuitive meanings go, "re-" is a standard English prefix 
meaning (among other things) "again", as in reborn, renew, retake, 
rewind, retry -- and "reiterate".


> That's no easier than learning a simple 
> fact about collections, and you end up knowing less after that 
> learning.

I'm not sure I understand how learning the definition of "reiterable" 
means I know *less* than when I started. Presumably that means I have 
forgotten something that I previously knew. I hope it wasn't something 
important.


> Think of my random example.

I would, but I've forgotten what it is.


> Is that a reiterable? Well, you can call 
> iter on its multiple times and get different iterators. But they don't 
> iterate the same value. So, yes or no? There's no intuition to guide 
> you there; all you have is a definitional rule that you had to 
> memorize.

I'm honestly not sure what actual problem you think this word is going 
to solve. I can see that there is some difficulty with people confusing 
the concepts of iterables and iterators, and some edge cases where 
people mistake "lazy sequences" like (x)range for an iterator. I 
acknowledge those difficulties. But:

(1) I don't think those difficulties are worth the thousands of words 
written in this thread so far.

(2) I don't understand how redefining (there's that re- prefix again) 
"collection" or adding "reiterable" will help.

I think the meanings of iterable and iterator are already clearly 
documented, and people still get them confused, so how will adding 
"reiterable" help?

(3) I don't think that the confusion between lazy sequences and 
iterators is especially harmful. 9 times out of 10, it's a difference 
that makes no difference.

I like to be pedantic and tell people that (x)range is not an iterator, 
but if I'm honest to myself, that fact is rarely going to help them 
write better code. (It won't make them write worse code either.)

(4) You've already acknowledged that "collection" currently has a 
meaning in Python. It's normally used for lists, tuples and dicts, as 
well as the things in the `collections` module. I don't understand why 
"collection" is your preferred term for an iterable that can be iterated 
over multiple times (a *re-iterable* in plain English).


I think that this problem is similtaneously too big to solve, and too 
trivial to care about solving. As I see it, we have a great many related 
categories of things that can be iterated over. Some of them are 
represented by concrete or abstract classes in the standard library, 
some of them aren't. Many of them overlap:

* eager sequences, mappings and sets;

* lazy sequences, mappings or sets;

* things which support the `in` operator (collections);

* things that obey the iterator protocol ("iterators");

* things which obey the old sequence protocol;

* things which obey the iterator protocol, except for the part about 
"once they become empty and raise StopIteration, they should always 
raise StopIteration" -- these are officially called "broken iterators";

* things which are broken iterators with an official API for 
restarting them (perhaps called "reiterators"?);

* reiterators which can be restarted, but won't necessarily give the 
same results each time;

* iterators that provide a look-ahead or look-behind method;

* sequences that use the function-call API, like random.random();

* things that you can pass to iter();

* things which you can pass to iter() multiple times, and get iterators 
which yield the same values each time;

* things which you can pass to iter() multiple times, and get iterators 
which DON'T yield the same values each time;

and probably more. I don't think it is practical, or necessary, to try 
naming them all. We have a good set of names already:

- collection for things that support `in`;
- iterable for things that can be iterated over;
- iterator for a particular subset of iterables;
- sequence for things like lists;
- lazy sequence for things that are sequences where the items are 
  calculated on demand rather than in advance.

I think adding to that set of names is a case of YAGNI.

I do see some sense in having a name for iterators with a "restart" 
method, and the obvious name for that would be reiterator. But I don't 
think the glossary needs to have that entry until such a time there 
actually is a restartable iterator in the std lib.

I *don't* think there's any need for a name for "iterables apart from 
iterators". While it is occasionally useful to be able to talk about 
such, that's not frequent enough that we need a name for it. We can just 
say "iterables apart from iterators", or we can give concrete examples: 
"sequences, mappings and sets". 


> Similarly, you can't meaningfully attach adjectives to "reiterable" 
> that restrict or expand its scope. "Lazy reiterable" doesn't mean 
> anything; "lazy collection" does.

I don't see why you think lazy reiterable doesn't mean anything. It is 
clearly a reiterable where the items are calculated on demand rather 
than in advance. What else could it mean?


> But I've already made all these points and you just skipped over them. 

It's a huge thread, with (no offense) a lot of rambling, pedantic 
arguments over the number of angels that can dance on the head of a pin 
(what sort of angels? seraphim, cherubim, ophanim, malakhim, dominions, 
powers, virtues, or something else? what size pin?). It's easy to get 
lost in the discussion, partly (I believe) because the problem being 
solved is so abstract. This seems to me to be a debate over definitions 
for their own sake, not in order to solve an actual problem.

I could be wrong, of course. But if there is anything actually concrete 
and important being solved by this proposal, it is well and truly hidden 
in the thousands of words of abstract argument.



-- 
Steve