[Python-Dev] Re: Reiterability

[Guido van Rossum]

> On Monday 20 October 2003 07:43 pm, Guido van Rossum wrote:
>    ...
> > I'm not sure why you say it's separate from cloning; it seems to me
> > that copy.copy(iter(range(10))) should return *exactly* what we'd want
> > the proposed clone operation to return.

[Alex]
> I'd be tickled pink if it did, but I would have expected a shallow
> copy to return an iterator that's not necessarily independent from
> the starting one.  Maybe I have a bad mental model of the "depth"
> (indirectness) of iterators?

Hm.  Let's consider a Python implementation of a sequence iterator (I
think you've given a similar class before):

  class SeqIter:
      def __init__(self, seq, i=0):
          self.seq = seq
          self.i = i
      def __iter__(self):
          return self # Obligatory self-returning __iter__
      def next(self):
          try:
              x = self.seq[self.i]
          except IndexError:
              raise StopIteration
          else:
              self.i += 1
              return x

All we care about really is that this is an instance with two instance
variables, seq and i.  A shallow copy creates a new instance (with a
new __dict__!) with the same two instance variable names, referencing
the same two objects.  Since i is immutable, the copy/clone is
independent from the original iterator; but both reference the same
underlying sequence object.

Clearly this is the copy semantics that would be expected from a
sequence iterator object implemented in C.  Ditto for the dict
iterator.

Now if someone wrote a tree class with a matching iterator class
(which might keep a stack of nodes visited in a list), the default
copy.copy() semantics might not be right, but such a class could
easily provide a __copy__ method that did the right thing (making a
shallow copy of the stack).

> > I see this as a plea to add __copy__ and __deepcopy__ methods to all
> > standard iterators for which it makes sense.  (Or maybe only __copy__
> > -- I'm not sure what value __deepcopy__ would add.)
> 
> Hmmm, copy the underlying sequence as well?  Don't have any use
> case for it, but that's what would feel unsurprising to me (as I have
> already mentioned I may not have the right mental model of an
> iterator...).

Right.  I have no use case for this either, although it's close to
pickling, and who knows if someday it might be useful to be able to
pickle iterators along with their containers.

> > I find this a reasonable request for the iterators belonging to
> > stndard containers (list, tuple, dict).  I guess that some of the
> > iterators in itertools might also support this easily.  Perhaps this
> > would be the road to supporting iterator cloning?
> 
> It would surely do a lot to let me clone things, yes, and in fact
> doing it with the existing __copy__ protocol sounds much better
> than sprouting a new one.

Right, so that's settled.  We don't need an iterator cloning protocol,
we can just let iterators support __copy__.  (There's no C-level slot
for this.)

> (Thanks for the confirmations and clarifications on file internals.
> btw, any news of the new experimental file things you were
> playing with back at PythonUK...?)

No; I donated it to pypy and I think it's in their subversion depot.
I haven't had time to play with it further.  It would be great to do a
rewrite from the ground up of the file object without using stdio, but
it would be a lot of work to get it right on all platforms; I guess an
engineering team of volunteers should be formed to tackle this issue.

--Guido van Rossum (home page: http://www.python.org/~guido/)