On Sat, Jan 03, 2015 at 03:37:03PM +0200, yotam vaknin wrote:

I would like to purpose that in python 3 it will be easy to chain and slice Iterators just like you can add and slice lists and tuples easily.

I think there could be 2 methods to do this: 1. Implementing '+' and slicing ([1:2:3]) for iterators and generators by default, Resulting in Itertool's chain(a,b) and islice(a,b,c) respectively. 2. Having itertool's chain and islice imported by default.

The iterator protocol is intentionally simple. To create an iterator, you need to only define two methods: (1) __iter__, which returns the instance itself; (2) __next__, which returns the next item and raises StopIterator when done. With your proposal, you would have to define two or three more methods: (3) __add__ and possibly __radd__, to chain iterators; (4) __getitem__, for slicing. There is nothing stopping you from adding these methods to your own iterator classes, but with your proposal that would be compulsory for all iterators.

I think since python 3 return zip,map, dict.items and so on as iterators, it makes working with those kind of objects more difficult without having these methods around. And since those methods are important enough to have for lists, it seems important enough for iterators too.

It only takes a single line of code to get iterator chaining and slicing: from itertools import chain, islice And now you can chain and slice any iterator, regardless of where it came from. I don't think that is difficult. *If* iterators where instances of a concrete base class, then it would make sense to add chain and slice methods to that base class so all other iterators could inherit from it. But they aren't, iterators in Python are values which obey a protocol, not inheritence. That means that a functional approach, like itertools, is more appropriate. I can see the appeal of using + and [a:b:c] syntax instead of function syntax chain() and islice(), but I don't think the advantage is enough to out-weigh the costs. -- Steve

On Jan 3, 2015, at 16:53, yotam vaknin wrote:

Sorry, I probably wasn't clear enough. My idea was to add these method (__add__, __getitem__) to the already available iterators (map,zip,dict.items) and generators by default. Not to make these methods part of the iterator protocol.

Well, dict.items doesn't return an iterator, it returns an iterable view. More importantly, if you add this just to generators and map and zip, it won't apply to (among other things) filter; the built-in iterators for list, tuple, set, dict, and the dict views; the special type used to iterate types that implement __getitem__ but not __iter__; any of the iterators from itertools--including the result of slicing or chaining two map iterators; files; csv readers; XML iterparse; … I think that would make the language a lot worse. In general, code doesn't care whether it's gotten an iterator from mapping a function over a file, chaining two iterators together, or iterating a sequence. If you wrote some code that expected to get a generator, used the + operator, and then changed the calling code to filter that generator, it would break, for no good reason. If you really wanted to, you could make this general pretty easily: change the meaning of the + operator so that, after checking __add__ and __radd__, before raising a TypeError, it checks whether both operand have __iter__ methods and, if so, it returns a chain of the two. Slicing has another problem, however. First, it's a bit odd to be able to do i[5:7] but not i[5]. But, more seriously, I wouldn't expect i[5:7] to give me an iterable that, when accessed, discards the first 5 elements of i. And imagine how confusing this code would be: >>> i = iter(range(10)) >>> a = i[2:4] >>> b = i[4:6] >>> list(a) [2, 3] >>> list(b) [8, 9] Hiding the fact that i is an iterable rather than a sequence will just confuse your readers.

2015-01-03 17:28 GMT+02:00 Steven D'Aprano :

...

On Sat, Jan 03, 2015 at 03:37:03PM +0200, yotam vaknin wrote:

...

I would like to purpose that in python 3 it will be easy to chain and slice Iterators just like you can add and slice lists and tuples easily.

I think there could be 2 methods to do this: 1. Implementing '+' and slicing ([1:2:3]) for iterators and generators by default, Resulting in Itertool's chain(a,b) and islice(a,b,c) respectively. 2. Having itertool's chain and islice imported by default.

The iterator protocol is intentionally simple. To create an iterator, you need to only define two methods:

(1) __iter__, which returns the instance itself;

(2) __next__, which returns the next item and raises StopIterator when done.

With your proposal, you would have to define two or three more methods:

(3) __add__ and possibly __radd__, to chain iterators;

(4) __getitem__, for slicing.

There is nothing stopping you from adding these methods to your own iterator classes, but with your proposal that would be compulsory for all iterators.

...

I think since python 3 return zip,map, dict.items and so on as iterators, it makes working with those kind of objects more difficult without having these methods around. And since those methods are important enough to have for lists, it seems important enough for iterators too.

It only takes a single line of code to get iterator chaining and slicing:

from itertools import chain, islice

And now you can chain and slice any iterator, regardless of where it came from. I don't think that is difficult.

*If* iterators where instances of a concrete base class, then it would make sense to add chain and slice methods to that base class so all other iterators could inherit from it. But they aren't, iterators in Python are values which obey a protocol, not inheritence. That means that a functional approach, like itertools, is more appropriate.

I can see the appeal of using + and [a:b:c] syntax instead of function syntax chain() and islice(), but I don't think the advantage is enough to out-weigh the costs.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

yotam vaknin wrote:

1. Implementing '+' and slicing ([1:2:3]) for iterators and generators by default, Resulting in Itertool's chain(a,b) and islice(a,b,c) respectively.

This will break NumPy. Sturla

On Sat, Jan 3, 2015 at 11:23 PM, Nick Coghlan wrote:

On 4 January 2015 at 07:14, Andrew Barnert wrote:

...

But it would be a pretty major change to Python to move from iterators to views. And iterators are much simpler to create than views, so the tradeoff probably wouldn't be worth it, even if it weren't for the historical/compatibility issue. (It's much the same with Haskell-style lazy lists; Python iterables can only substitute for lazy lists 90% of the time, but that doesn't mean lazy lists are a better language choice.)

It's worth noting that many types implement Mapping.(keys,values,items) as iterators in Python 3 rather than as views, and generally don't receive any complaints from users.

They probably should implement keys/etc. properly. collections.abc.[Mutable]Mapping make it easy to support views if you otherwise implement a dictionary-like object. -- Devin

In addition to itertools.chain and itertools.islice, this is possible with a number of third-party packages: * http://toolz.readthedocs.org/en/latest/api.html#itertoolz (concat, pluck, first, last, take) * https://github.com/kachayev/fn.py#streams-and-infinite-sequences-declaration (<< 'Stream' operator) * http://funcy.readthedocs.org/en/latest/seqs.html Stdlib docs for this: * https://docs.python.org/2/howto/functional.html#iterators * https://docs.python.org/2/tutorial/classes.html#iterators * https://docs.python.org/2/library/stdtypes.html#iterator-types * https://docs.python.org/2/reference/datamodel.html#object.__iter__ * On Sat, Jan 3, 2015 at 7:37 AM, yotam vaknin wrote:

I would like to purpose that in python 3 it will be easy to chain and slice Iterators just like you can add and slice lists and tuples easily.

I think there could be 2 methods to do this: 1. Implementing '+' and slicing ([1:2:3]) for iterators and generators by default, Resulting in Itertool's chain(a,b) and islice(a,b,c) respectively. 2. Having itertool's chain and islice imported by default.

I think since python 3 return zip,map, dict.items and so on as iterators, it makes working with those kind of objects more difficult without having these methods around. And since those methods are important enough to have for lists, it seems important enough for iterators too.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/