list.sort, was Re: [Python-Dev] decorate-sort-undecorate

[Gustavo Niemeyer]

Hi Raymond!

If that has been discussed long ago as you mention, please, just tell me
so and I won't try to recreate the same discussion. :-)

[...]
> islice(it, None, None, -1) is a disaster when presented with an infinite
> iterator and a near disaster with a long running iterator.

I don't agree with that approach. I think islice() would be more useful
if it was based on best effort to try to reduce memory usage. With
a negative index, it will be necessary to iterate trough all steps,
but it can do better than list(iter)[-1], for example. Knowing that you
have a negative index of -1 means that you may cache just a single
entry, instead of the whole list.

> Handling negative steps entails saving data in memory.

Indeed. But if I *want* to use a negative index over an iterator, it
would be nice if some smart guy did the work for "me" instead of having
to do that by hand, or even worse, having to use a list() which will
store *everything* in memory.

As a real world example, have a look at rrule's __getitem__() method
(more info on https://moin.conectiva.com.br/DateUtil):

    def __getitem__(self, item):
        if self._cache_complete:
            return self._cache[item]
        elif isinstance(item, slice):
            if item.step and item.step < 0:
                return list(iter(self))[item]
            else:
                return list(itertools.islice(self,
                                             item.start or 0,
                                             item.stop or sys.maxint,
                                             item.step or 1))
        elif item >= 0:
            gen = iter(self)
            try:
                for i in range(item+1):
                    res = gen.next()
            except StopIteration:
                raise IndexError
            return res
        else:
            return list(iter(self))[item]

Having negative indexes is *very* useful in that context, and I'd like
so much to turn it into simply

   return list(itertools.islice(self,
			        item.start, item.stop, item.step))

Now, have a look at the count() method, which is useful as
well (it is the same as a __len__() method, but introducing
__len__() kills the iterator performance).

    def count(self):
        if self._len is None:
            for x in self: pass
        return self._len

It is very useful as well, and having something like ilen() would
be nice, even though it must iterate over the whole sequence.

This would never end up in an infinite loop in that context, and
even if it did, I wouldn't care about it. Not introducing it for
being afraid of an infinite loop would be the same as removing
the 'while' construct to avoid "while 1: pass".

-- 
Gustavo Niemeyer
http://niemeyer.net