[Python-ideas] Make len() usable on a generator

[Andrew Barnert]

On Oct 4, 2014, at 13:48, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 4 October 2014 02:45, Ian Cordasco <graffatcolmingov at gmail.com> wrote:
>> On Fri, Oct 3, 2014 at 11:32 AM, Alexander Belopolsky
>> <alexander.belopolsky at gmail.com> wrote:
>>> You are correct about python-ideas, but range is not a generator in python 3
>>> either:
>>> 
>>> Python 3.4.0a0 (default:e9cecb612ff7+, Oct  3 2014, 12:07:42)
>>>>>> type(range(10))  # not a generator
>>> <class 'range'>
>>>>>> type(x for x in range(10))  # a generator
>>> <class 'generator'>
>> 
>> Ah yeah. All too often I conflate lazy iterators with generators. Nevermind
> 
> Python 3 range isn't a lazy iterator either - it's a full sequence
> type (specifically, a calculated tuple:
> https://docs.python.org/3/library/stdtypes.html#ranges). The only
> immutable sequence operations it doesn't support are concatenation and
> repetition (since concatenating or repeating a range can't be
> represented using the simple "start + n*step" calculation that range
> uses internally).
> 
> We'd have gotten a *lot* more complaints if we just dropped xrange in
> as a substitute for the list objects returned by Python 2's range
> builtin :)

For some reason, most of the 3.x early adopters understood this, but later switchers seem to often believe that this is exactly what Python 3 did. For example, at least half a dozen times, I've written an answer on StackOverflow showing someone that `n in range(start, stop)` is what they were trying to write, and explaining that it means the same thing as `start <= n < stop` (with various caveats about Python 2, and `step`, etc.) only for someone to edit or comment on my answer claiming that range.__contents__ has to be linear and should never be used. I explain that it's a sequence, and can be indexed, and show them a quick timing test, and they still insist that's just an accidental optimization that shouldn't be relied on, because it's really just meant to be Python 2's xrange. Once someone insisted that if you want this functionality you should use a multi-interval class off PyPI, even though the docs for that class explicitly say that a single interval is just an empty subclass of range in Py 3. People who stick with 2.x just really want to believe you got this wrong, I guess.

I've seen similar things with, e.g., people not believing that dict.keys() returns a set that's a view into the dict's storage rather than some kind of magical iterator that knows how to be used multiple times. (I'm not even sure how that would work; how could __iter__ know when you were trying to get self and when you were trying to start over?)