[Python-ideas] Integrate some itertools into the Python syntax

[Andrew Barnert]

On Mar 22, 2016, at 13:11, Michel Desmoulin <desmoulinmichel at gmail.com> wrote:
> 
> Le 22/03/2016 21:03, Andrew Barnert a écrit :
>> On Mar 22, 2016, at 10:51, Michel Desmoulin <desmoulinmichel at gmail.com> wrote:
>>> 
>>> For every slice. We don't, because slicing is part of our standard data
>>> processing toolkit. And I think we can make it even better.
>>> 
>>> We already do in some places. E.G: range(10)[3:5] works while range()
>>> generate values on the fly. Why ? Because it's convenient, expressive
>>> and Pythonic.
>> 
>> Range is a sequence, just like list and tuple, not an iterator, like generator and list_iterator. So it supports slicing because sequences generally support slicing. Not because it's so convenient and expressive that it's worth making a special case, just because it follows the normal rules.
>> 
>> It's amazing to me that so many people still describe xrange/3.x range as an iterator or generator, and then draw further incorrect lessons from that. Yes, the docs were a bit confusing up to about 2.4, but how long ago is that now? Lazy sequences are still sequences. Strict non-sequences (like dict) are still not sequences. Lazy vs. strict has almost[1] nothing to do with sequence vs. iterator (or sequence vs. non-sequence collection, or collection vs. iterator, or anything else.)
>> 
>> ---
> 
> Yes, I worded my title with "all iterables", it was a mistake, I didn't
> think people would raise the debate on dict, set, etc because it was
> obvious to me that nobody would imagine I would try to make slicing work
> on those.
> 
> I was wrong, but please don't make drown the debate into a battle of
> semantics. Ok, let's recenter to lazy sequences, and how convenient it
> would be to be able to limit them to a size or a condition without
> needing itertools.

This is not just a battle of (meaningless) semantics. You seem to be completely misinterpreting the actual language semantics of Python, and therefore making proposals that don't make any sense.

Case in point: you can _already_ limit lazy sequences to a size without needing itertools. That's exactly what you're doing in your range example. So we don't need to fix the language to make something possible when that something is already possible.

> And in that context, the fact is that range(1000000) doesn't create
> 1000000 items in memory, and it's very convenient to be able to slice
> it. And it would would be as convenient to be able to do so on
> generators (which are sequences) instead of using islice.

No, generators are _not_ sequences. And that's exactly the point you're missing. 

You have to understand the difference between iterators and other iterables to use Python effectively. That's just fundamental to so much of Python that there's no way around that. Maybe that was a language design mistake, but trying to muddy the waters further and confuse even more people is hardly a solution.

Meanwhile, the range type points to a ready solution: when you want slicing (or other sequence behavior) with laziness, you can always write a lazy sequence type, usually in only a few lines of code. (I've got a couple blog posts on it, if you can't figure it out yourself.) And that solves the problem without introducing any confusion about what it should mean to slice an iterator, or any problems of the "if header(spam[:6]): process_body(spam[6:])" type.