[Python-3000] Builtin iterator type

[Guido van Rossum]

[Adding back python-3000 -- I'm not going to have private conversations]

On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> On 11/13/06, Guido van Rossum <guido at python.org> wrote:
>
> > Hm. Without knowing much of the background, this appears to be a
> > worrysome trend away from duck typing. Why would I have to inherit
> > from a standard class just so that I can implement next()? What's the
> > advantage of the proposed change?
>
> Duck typing will still apply of course; an object that just implements
> next() without inheriting from iter will continue to be iterable. The
> advantage is the iterator methods and syntax sugar you get for free,
> instead of importing them explicitly from itertools (I've noticed that
> most of my modules lately start with a variation of "import sys,
> itertools as it").

Well, if you move the itertools functions to the base class, without
inheriting from that base class you'd lose the itertools
functionality. (Although many itertools functions take multiple
generator arguments so I'm not sure how e.g. imap or izip can become
methods without losing functionality.)

> The way I see it, the trend is not away from duck typing but towards
> iterability (generators, itertools, gen. expressions). A builtin iter
> type replacing for the most part the need for itertools is a natural
> next step to this direction.

You need to have a better (more technical) argument than that to convince me.

Let me go back to your original arguments:

> 1. Formalize the currently informal concept of iterator by changing
> iter() from a builtin function to a type.

That surely sounds like a step away from duck typing. Or what else
does "formalize" mean to you?

> 2. Augment this type with operators to implement operations currently
> done explicitly by itertools.*.

Please go over all of the itertools functions and show how they can be
reimplemented as methods. I think you'll find most of them don't work
this way.

> 3. Make this type a base for all other (existing and future) iterator
> types, e.g. xrange and generator.

Here again you're contradicting your "duck typing isn't dead" claim --
you do say "all other (existing and future) iterator types".

> > Are you going to propose similar
> > changes for all standard de-facto interfaces, like sequences,
> > mappings, files etc.?
>
> No, I won't (at least not for now ;-)). Notice however that most
> user-defined sequences, mappings, etc. usually don't start from
> scratch; they either inherit from an existing type or from an
> appropriate mixin. For a baseline iterator that just implements next()
> there's no need for a base type or mixin, but if you consider that
> itertools.* functionality should/would-be-nice to be provided as
> methods and/or syntax instead of imported functions, the parallel
> becomes apparent.

Despite invoking c.l.py I don't think you have broad support for this idea.

It's not too late to drop it.

> George
>
> > On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > Following up on a recent c.l.py thread
> > > (http://groups.google.com/group/comp.lang.python/browse_frm/thread/42818717b400bcd4/#),
> > > I'd like to get an idea from python-dev folks on how much of a chance
> > > is there for a builtin iterator type. Although there might be a slight
> > > possibility to make this proposal backwards compatible, I am mainly
> > > targetting Py3K, so compatibility is not a strict requirement.
> > >
> > > The key points are:
> > > 1. Formalize the currently informal concept of iterator by changing
> > > iter() from a builtin function to a type.
> > > 2. Augment this type with operators to implement operations currently
> > > done explicitly by itertools.*.
> > > 3. Make this type a base for all other (existing and future) iterator
> > > types, e.g. xrange and generator.
> > >
> > > As a proof of concept, I provide below a sample implementation of how
> > > I imagine this type to work (also posted in the original c.l.py.
> > > thread):
> > >
> > >
> > > from itertools import chain, tee, islice
> > >
> > > import __builtin__
> > > _builtin_iter = __builtin__.iter
> > >
> > > class iter(object):
> > >
> > >     def __init__(self, iterable):
> > >         self._it = _builtin_iter(iterable)
> > >
> > >     def __iter__(self):
> > >         return self
> > >     def next(self):
> > >         return self._it.next()
> > >
> > >     def __getitem__(self, index):
> > >         if isinstance(index, int):
> > >             try: return islice(self._it, index, index+1).next()
> > >             except StopIteration:
> > >                 raise IndexError('Index %d out of range' % index)
> > >         else:
> > >             start,stop,step = index.start, index.stop, index.step
> > >             if start is None: start = 0
> > >             if step is None: step = 1
> > >             return islice(self._it, start, stop, step)
> > >
> > >     def __add__(self, other):
> > >         return chain(self._it, other)
> > >     def __radd__(self,other):
> > >         return chain(other, self._it)
> > >
> > >     def __mul__(self, num):
> > >         return chain(*tee(self._it,num))
> > >
> > >     __rmul__ = __mul__
> > >
> > > __builtin__.iter = iter
> > >
> > > if __name__ == '__main__':
> > >     def irange(*args):
> > >         return iter(xrange(*args))
> > >
> > >     assert list(irange(5)[:3]) == range(5)[:3]
> > >     assert list(irange(5)[3:])  == range(5)[3:]
> > >     assert list(irange(5)[1:3]) == range(5)[1:3]
> > >     assert list(irange(5)[3:1]) == range(5)[3:1]
> > >     assert list(irange(5)[:])   == range(5)[:]
> > >     assert irange(5)[3]         == range(5)[3]
> > >
> > >     s = range(5) + range(7,9)
> > >     assert list(irange(5) + irange(7,9)) == s
> > >     assert list(irange(5) +  range(7,9)) == s
> > >     assert list(range(5)  + irange(7,9)) == s
> > >
> > >     s = range(5) * 3
> > >     assert list(irange(5) * 3) == s
> > >     assert list(3 * irange(5)) == s
> > >
> > >
> > > Thoughts, opinions, suggestions, objections, all welcome.
> > >
> > > Regards,
> > > George
> > > _______________________________________________
> > > Python-3000 mailing list
> > > Python-3000 at python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
> > >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)