[Python-Dev] Allowing slicing of iterators

[Nick Coghlan]

Guido van Rossum wrote:
> Since we already have the islice iterator, what's the point?

I'd like to see iterators become as easy to work with as lists are. At the 
moment, anything that returns an iterator forces you to use the relatively 
cumbersome itertools.islice mechanism, rather than Python's native slice syntax.

In the example below (printing the first 3 items of a sequence), the fact that 
sorted() produces a new iterable list, while reversed() produces an iterator 
over the original list *should* be an irrelevant implementation detail from the 
programmer's point of view.

However, the fact that iterators aren't natively sliceable throws this detail in 
the programmer's face, and forces them to alter their code to deal with it. The 
conversion from list comprehensions to generator expressions results in similar 
irritation - the things just aren't as convenient, because the syntactic support 
isn't there.

Py> lst = "1 5 23 1234  57 89 2 1 54 7".split()
Py> lst
['1', '5', '23', '1234', '57', '89', '2', '1', '54', '7']

Py> for i in sorted(lst)[:3]:
...   print i
...
1
1
1234

Py> for i in reversed(lst)[:3]:
...   print i
...
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: unsubscriptable object

Py> from itertools import islice
Py> for i in islice(reversed(lst), 3):
...   print i
...
7
54
1

>It seems
> to me that introducing this notation would mostly lead to confuse
> users, since in most other places a slice produces an independent
> *copy* of the data.

Well, certainly everything I can think of in the core that currently supports 
slicing produces a copy. Slicing on a numarray, however, gives you a view. The 
exact behaviour (view or copy) really depends on what is being sliced.

For iterators, I think Raymond's islice exemplifies the most natural slicing 
behaviour. Invoking itertools.tee() behind the scenes (to get copying semantics) 
would eliminate the iterator nature of the approach in many cases.

I don't think native slicing support will introduce any worse problems with the 
iterator/iterable distinction than already exist (e.g. a for loop consumes an 
iterator, but leaves an iterable unchanged. Similarly, slicing does not alter an 
iterable, but consumes an iterator).

Continuing the sorted() vs reversed() example from above:

Py> sortlst = sorted(lst)
Py> max(sortlst)
'89'
Py> len(sortlst)
10

Py> revlst = reversed(lst)
Py> max(revlst)
'89'
Py> len(revlst)
0

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net