[Python-Dev] RE: cloning iterators again
Guido van Rossum
guido at python.org
Tue Oct 28 13:00:14 EST 2003
> Armin Rigo posted an URL to a C extension module he has recently
> developed, and used. That module is able to copy running instances
> of a generator. Currently, it does so by just "knowing", and type by
> type dealing with, several types whose instances could be values
> referred to by the generator's frame. I was suggesting extending this
> so that, when values of other types are met, instead of automatically
> failing (or, as I believe I recall Armin's extension does today, copying
> the reference rather than the value), the copy process would...:
I haven't seen Armin's code, but I don't believe that the type alone
gives enough information about whether they should be copied.
Consider a generator that uses a dict as a cache or memo for values it
computes. Multiple instances of the generator share the dict, but for
efficiency the generator references it in a local variable. This dict
should not be copied when copying the generator's stack frame.
But consider another generator that uses a dict to hold some of its
state. This dict should be copied.
> > > > Cool! Why don't you try copy.copy on types you don't automatically
> > > > recognize and know how to deal with, BTW? That might make this
> and Armin said that __copy__ seems weak to him but __deepcopy__
> might not be:
> > > I will try. Note that only __deepcopy__ makes sense, as far as I can
> > > tell, because there is too much state that really needs to be copied and
> > > not shared in a generator (in particular, the sequence iterators from
> > > 'for' loops).
> So, now he went on to ask about __deepcopy__ and you answered:
> > > I'm not sure about how deep-copying should be defined for built-in
> > > types. Should a built-in __deepcopy__ method try to import and call
> > > copy.deepcopy() on the sub-items? This doesn't seem to be right.
> > Almost -- you have to pass the memo argument that your __deepcopy__
> > received as the second argument to the recursive deepcopy() calls, to
> > avoid looping on cycles.
> Now, if Armin's code can only provide __deepcopy__ and not __copy__,
> then it's probably of little use wrt the __copy__ functionality I
> talk about in PEP 323 (which I still must revise to take into
> account your feedback and Raymond's -- plan to get to that as soon
> as I've cleared my mbox) -- the memory and time expenditure is
> likely to be too high for that.
> It's still going to be a cool hack, well worth "publishing" as such,
As a third-party module? Sure.
> and probably able to be "user-installed" as the way deepcopy deals
> with generators even though generators themselves may not sprout a
> __deepcopy__ method themselves (fortunately, copy.copy does a lot of
> "ad-hoc protocol adaptation" -- it's occasionally a bit rambling or
> hard to follow, but often allows plugging in "third party copiers"
> for types which their authors hadn't imagined would be copied or
> deep copied, so that other innocent client code which just calls
> copy.copy(x) will work... essentially how "real" adaptation would
> work, except that registering a third-party protocol adapter would
> be easier:-).
--Guido van Rossum (home page: http://www.python.org/~guido/)
More information about the Python-Dev