list.sort, was Re: [Python-Dev] decorate-sort-undecorate
Gustavo Niemeyer
niemeyer at conectiva.com
Tue Oct 28 08:59:38 EST 2003
Hi Raymond!
If that has been discussed long ago as you mention, please, just tell me
so and I won't try to recreate the same discussion. :-)
[...]
> islice(it, None, None, -1) is a disaster when presented with an infinite
> iterator and a near disaster with a long running iterator.
I don't agree with that approach. I think islice() would be more useful
if it was based on best effort to try to reduce memory usage. With
a negative index, it will be necessary to iterate trough all steps,
but it can do better than list(iter)[-1], for example. Knowing that you
have a negative index of -1 means that you may cache just a single
entry, instead of the whole list.
> Handling negative steps entails saving data in memory.
Indeed. But if I *want* to use a negative index over an iterator, it
would be nice if some smart guy did the work for "me" instead of having
to do that by hand, or even worse, having to use a list() which will
store *everything* in memory.
As a real world example, have a look at rrule's __getitem__() method
(more info on https://moin.conectiva.com.br/DateUtil):
def __getitem__(self, item):
if self._cache_complete:
return self._cache[item]
elif isinstance(item, slice):
if item.step and item.step < 0:
return list(iter(self))[item]
else:
return list(itertools.islice(self,
item.start or 0,
item.stop or sys.maxint,
item.step or 1))
elif item >= 0:
gen = iter(self)
try:
for i in range(item+1):
res = gen.next()
except StopIteration:
raise IndexError
return res
else:
return list(iter(self))[item]
Having negative indexes is *very* useful in that context, and I'd like
so much to turn it into simply
return list(itertools.islice(self,
item.start, item.stop, item.step))
Now, have a look at the count() method, which is useful as
well (it is the same as a __len__() method, but introducing
__len__() kills the iterator performance).
def count(self):
if self._len is None:
for x in self: pass
return self._len
It is very useful as well, and having something like ilen() would
be nice, even though it must iterate over the whole sequence.
This would never end up in an infinite loop in that context, and
even if it did, I wouldn't care about it. Not introducing it for
being afraid of an infinite loop would be the same as removing
the 'while' construct to avoid "while 1: pass".
--
Gustavo Niemeyer
http://niemeyer.net
More information about the Python-Dev
mailing list