[Python-ideas] Generator unpacking

[Sven R. Kunze]

On 16.02.2016 06:56, Andrew Barnert via Python-ideas wrote:
> On Feb 15, 2016, at 19:42, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>> Are there really places where we *want* "foo(range(n))" to raise but
>> "foo(iter(range(n)))" to do something useful?  In other words, is this
>> really a terminology problem, rather than a problem with the design of
>> APIs that take iterators rather than iterables?
> Sure. Any multi-pass algorithm should raise when given an iterator. (Maybe some could instead copy the iterator to a list or something, but as a general rule, silently and unexpectedly jumping from 0 to O(N) space depending on the input type isn't very friendly.)
>
> In the other direction, of course, the next function requires an iterator, and should and does raise when given something else. And the slightly higher-level functions discussed in this thread are the same. A function intended to consume and return the first few values in an iterator while leaving the iterator holding the rest is basically just a "multinext", and it would be very confusing if it took a collection and left it holding all the values, including the two you already used. And how do you explain that to a human? You could say "While it's an iterable, it's not an iterator." And that's exactly what Python has said for the last however-many years. And that's exactly what leads to the confusion Paul was talking about (and the additional confusion he inadvertently revealed). And the problem isn't "Paul is an idiot", because I've seen multiple core devs, and other highly experienced Python programmers, make the same mistake. Even the official docs made a similar mistake, and nob
>   ody caught it for 4 years. If the current terminology were sufficiently clear and intuitive that nothing needed to be done, these problems wouldn't exist.

I think both is true. The current API, being one of the best available 
out there, exposes some warts (as you described) which in turn lead to a 
plethora of abstract wording expressing almost the same:

- lists, iterables, iterators, sets, generators, views, range, etc.

These datastructures evolved over time and proved useful. In my point of 
view, what's missing a structure to put them into.


If one looks closely, one can see how they differ in usage:

*
*1) Is it iterable?*

*- yes to all (as far as I know)*
**
*2) Is it subscriptable?*
*
- yes: list, range
- no: iterator, generator, set, view
*
*3) Has it a length?*
*
- yes: list, range, set, view
- no: iterator, generator

*4) Has it a contains test?*

- yes: list, range, set, view
- no: iterator, generator
*
5) Are items materialized?*

- yes: list, set
- no: iterator, generator, range, view

*6) Can it have mutable underlying data?*

- yes: generator, iterator, view
- no: list, set, range

*7) Does iteration change it?*

- yes: iterator, generator
- no: list, set, range, view


As expected, they almost always have nothing in common except 1). I am 
perfectly fine with calling these things collections; if I have a 
container of n things, I almost certainly can go through them one by one.
2) to 4) are usually the most common operations and the safe when 
implemented. range is like list; view is like set.
5) to 7) are usually things you worry about when you are concerned with 
performance. As usual, those things are hard to do right.

*My suggestions:*
Let's call all of them *collections*.
Let's call collections *list-like* if they support 2) to 4) and 
*set-like* if they support 4).
Let's call collections *lazy* when they don't fit 5).
Let's call collections *view-like* when they feature 6).
Let's call collections *iteration-stable* when they don't fit 7).

As you can see, for me it's not a matter of inventing new nouns, but a 
matter of finding the right adjective to further specify the type of 
collection.

Does it help? I don't know but it helped me to structure the concepts.


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160216/9ac80536/attachment-0001.html>