[Python-ideas] Slicing and Chainging iterables.

[Andrew Barnert]

On Jan 3, 2015, at 16:53, yotam vaknin <tomirendo at gmail.com> wrote:

> Sorry, I probably wasn't clear enough.
> My idea was to add these method (__add__, __getitem__) to the already available iterators (map,zip,dict.items) and generators by default. Not to make these methods part of the iterator protocol.

Reading this again, I think what you _really_ want is probably a generalized view protocol, where views of sequences act like (in fact, _are_) sequences, but with the main advantages of iterators (no storage or upfront computation). Like dictionary views (which is why I chose the name "view", of course).

And there's no reason map _couldn't_ return a view instead of an iterator. When called on (all) sequences it would be a sequence view; when called on (any) iterators it would be an iterator view; otherwise it would be a non-sequence non-iterator (that is, reusable, but not indexable) iterable view. But in any case, it would never call the mapped function on a given mapped value until the result is requested. In other words:

    >>> def spam(x):
    ...      print(x)
    ...      return x+1
    >>> m = map(spam, range(5))
    >>> m
    <MapSequenceView at 0x12345678>
    >>> m[2]
    2
    3
    >>> ms = m[2:4]
    >>> ms[1]
    3
    4

You can build view-based zip similarly, and filter, and most of the functions in itertools, and so on (although note that in some cases--like filter--a view of a sequence wouldn't be a sequence).

You could also make slices into lazy views instead of copying (as NumPy already does with its arrays).

In fact, Python could have designed all of its collections and higher-order functions around views instead of iterators. That's what Swift did. (Swift goes farther, with a hierarchy of kinds of indexing, based on C++, instead of just iterables and sequences.) I wrote a blog post last year (based on the first beta of Swift, which had some oddities in its implementation) examining how this could work in Python terms (http://stupidpythonideas.blogspot.com/2014/07/swift-style-map-and-filter-views.html). I think I've also got a partial implementation somewhere on Github of map, filter, and maybe a few more functions implemented as views.

But it would be a pretty major change to
Python to move from iterators to views. And iterators are much simpler to create than views, so the tradeoff probably wouldn't be worth it, even if it weren't for the historical/compatibility issue. (It's much the same with Haskell-style lazy lists; Python iterables can only substitute for lazy lists 90% of the time, but that doesn't mean lazy lists are a better language choice.)