[Python-ideas] Add irange with large integer step support to itertools

[Nick Coghlan]

On Tue, Jan 11, 2011 at 12:55 AM, Zac Burns <zac256 at gmail.com> wrote:
> -1 for any proposal that adds anything differentiating int/long.

It isn't about adding anything - the signature of the length slots at
the C level already uses a Py_ssize_t, so any time you get the length
of a container, you're limited to values that will fit in that size.
This is fine for real containers, as you will run out of memory long
before the container length overflows and throws an exception. It *is*
an issue for a virtual container like range() though - because it
doesn't actually *create* the whole range, it can be created with a
length that exceeds what Py_ssize_t can handle. That's fine, until you
run into one of the operations that directly or indirectly invokes
len() on the object.

Currently, indexing a range is such an operation (which is why it
fails). While it's a fairly straightforward (albeit somewhat tedious)
change to fix range_subscript and range_item to correctly handle cases
where the index and/or the length exceed sys.maxsize, it still
requires someone to actually create the issue on the tracker and then
propose a patch to fix it.

It's even theoretically possible to upgrade the __len__ protocol to
support lengths that exceed Py_ssize_t, but that's a much more
ambitious (PEP scale) project.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia