Being able to specify "copy mode" to copy.deepcopy
This is about the `copy.deepcopy` function. With the __deepcopy__ method, user-defined objects can specify how they will be copied. But it is assumed that you will always want to copy them the same way. What if sometimes you want to copy them in one way and sometimes in another? I am now being held back by this limitation. I will give some background to what I'm doing: I'm developing a simulations framework called GarlicSim. You can see a short video here: http://garlicsim.org/brief_introduction.html The program handles world states in simulated worlds. To generate the next world state in the timeline, the last world state is deepcopied and then modified. Now sometimes in simulations there are big, read-only objects that I don't want to replicate for each world state. For example, a map of the environment in which the simulation takes place. So I have defined a class called `Persistent`, for which I have defined a __deepcopy__ that doesn't actually copy it, but gives a reference to the original object. So now I can use `Persistent` as a sub-class to these big objects that I don't want to replicate. But in some cases I do want to replicate these objects, and I can't! So I suggest that it will be possible to specify a "mode" for copying. User defined objects will be able to specify how they will be deepcopied in each mode. What do you think? Ram.
cool-RR wrote:
This is about the `copy.deepcopy` function.
With the __deepcopy__ method, user-defined objects can specify how they will be copied. But it is assumed that you will always want to copy them the same way. What if sometimes you want to copy them in one way and sometimes in another?
I am now being held back by this limitation. I will give some background to what I'm doing:
I'm developing a simulations framework called GarlicSim. You can see a short video here: http://garlicsim.org/brief_introduction.html The program handles world states in simulated worlds. To generate the next world state in the timeline, the last world state is deepcopied and then modified.
Now sometimes in simulations there are big, read-only objects that I don't want to replicate for each world state. For example, a map of the environment in which the simulation takes place. So I have defined a class called `Persistent`, for which I have defined a __deepcopy__ that doesn't actually copy it, but gives a reference to the original object. So now I can use `Persistent` as a sub-class to these big objects that I don't want to replicate.
But in some cases I do want to replicate these objects, and I can't!
So I suggest that it will be possible to specify a "mode" for copying. User defined objects will be able to specify how they will be deepcopied in each mode.
What do you think?
My own feeling is that this is a misuse of __deepcopy__: if you ask for a copy (of a mutable object) then you should get a copy (for immutable objects copying isn't necessary).
MRAB <python@...> writes:
cool-RR wrote:
What do you think?
My own feeling is that this is a misuse of __deepcopy__: if you ask for a copy (of a mutable object) then you should get a copy (for immutable objects copying isn't necessary).
I agree it that the Persistent.__deecopy__ thing does smell like misuse on my part. However I'd be happy to hear any alternative suggestion you have on how to solve the problem I have. Meanwhile, I thought of a nice backwards-compatible way to implement what I suggest, but I want to know whether this idea makes sense at all to the people here. Ram.
Ram Rachum wrote:
I agree it that the Persistent.__deecopy__ thing does smell like misuse on my part. However I'd be happy to hear any alternative suggestion you have on how to solve the problem I have.
Meanwhile, I thought of a nice backwards-compatible way to implement what I suggest, but I want to know whether this idea makes sense at all to the people here.
It is already quite easy to abuse the "memo" dict argument of copy.deepcopy to pass this kind of flag to the __deepcopy__ methods. What else do you need? - Jacob
2009/12/15 Ram Rachum <cool-rr@cool-rr.com>:
MRAB <python@...> writes:
cool-RR wrote:
What do you think?
My own feeling is that this is a misuse of __deepcopy__: if you ask for a copy (of a mutable object) then you should get a copy (for immutable objects copying isn't necessary).
I agree it that the Persistent.__deecopy__ thing does smell like misuse on my part. However I'd be happy to hear any alternative suggestion you have on how to solve the problem I have.
Deepcopy is a very simple operation conceptually, there's no need to make it more complicated. How about implementing __deepcopy__ in your world state objects? Specify attributes that don't need copying. You can even use the Persistent class to signal that. Something like this (untested!): def __deepcopy__(self): new = self.__class__() for k,v in self.__dict__.iteritems(): setattr(new, k, v if isinstance(v, Persistent) else deepcopy(v)) return new Vitor
Vitor Bosshard <algorias@...> writes:
Deepcopy is a very simple operation conceptually, there's no need to make it more complicated. How about implementing __deepcopy__ in your world state objects? Specify attributes that don't need copying. You can even use the Persistent class to signal that. Something like this (untested!):
def __deepcopy__(self): new = self.__class__() for k,v in self.__dict__.iteritems(): setattr(new, k, v if isinstance(v, Persistent) else deepcopy(v)) return new
Vitor
And what happens when State refers to another object which refers to a Persistent? Ram.
2009/12/15 Ram Rachum <cool-rr@cool-rr.com>:
Vitor Bosshard <algorias@...> writes:
Deepcopy is a very simple operation conceptually, there's no need to make it more complicated. How about implementing __deepcopy__ in your world state objects? Specify attributes that don't need copying. You can even use the Persistent class to signal that. Something like this (untested!):
def __deepcopy__(self): new = self.__class__() for k,v in self.__dict__.iteritems(): setattr(new, k, v if isinstance(v, Persistent) else deepcopy(v)) return new
Vitor
And what happens when State refers to another object which refers to a Persistent?
Then that object would need to implement the same method, perhaps by inheriting form a common base. The point is that it can be done in a straightforward manner without needing to change the stdlib. Vitor
Vitor Bosshard <algorias@...> writes:
And what happens when State refers to another object which refers to a Persistent?
Then that object would need to implement the same method, perhaps by inheriting form a common base.
And what if the object is from a class defined by a third-party module that I can't change?
The point is that it can be done in a straightforward manner without needing to change the stdlib.
I guess so, yes. My method would be something like what Jacob said, abusing the memo dict to pass the copying mode. But I thought perhaps we can set a standard way for specifying different copy modes, because otherwise I'll do my memo hack and someone else will do his different memo hack and it won't be compatible. I'll detail my hack later today when I'll be back home. Ram.
On 12/15/2009 12:51 PM, Ram Rachum wrote:
Vitor Bosshard<algorias@...> writes:
And what happens when State refers to another object which refers to a Persistent?
Then that object would need to implement the same method, perhaps by inheriting form a common base.
And what if the object is from a class defined by a third-party module that I can't change?
The point is that it can be done in a straightforward manner without needing to change the stdlib.
I guess so, yes. My method would be something like what Jacob said, abusing the memo dict to pass the copying mode. But I thought perhaps we can set a standard way for specifying different copy modes, because otherwise I'll do my memo hack and someone else will do his different memo hack and it won't be compatible.
Perhaps you can post a recipe at the Python Cookbook. People who care about compatibility can follow the same recipe.
Terry Reedy wrote:
I guess so, yes. My method would be something like what Jacob said, abusing the memo dict to pass the copying mode. But I thought perhaps we can set a standard way for specifying different copy modes, because otherwise I'll do my memo hack and someone else will do his different memo hack and it won't be compatible.
Perhaps you can post a recipe at the Python Cookbook. People who care about compatibility can follow the same recipe.
Alternatively, this use case strikes me as being rather similar to the various flatten() recipes out there that accept a list of "atomic" types to avoid flattening iterable-but-not-really-a-container types such as strings. The analogy currently breaks due to copy.deepcopy() being set up with each __deepcopy__ method doing its own recursion rather than constructing a graph of mutable (to be copied) and immutable members (to be referenced) down the chain of the object graph. More flexible (but significantly harder) than adding a copy mode would be defining a protocol for exposing the object graph in a standardised fashion. __iter__ in conjunction with __dict__ would get you a fair way, but there would be a lot of complications. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Terry Reedy <tjreedy@...> writes:
I guess so, yes. My method would be something like what Jacob said, abusing the memo dict to pass the copying mode. But I thought perhaps we can set a standard way for specifying different copy modes, because otherwise I'll do memo hack and someone else will do his different memo hack and it won't be compatible.
Perhaps you can post a recipe at the Python Cookbook. People who care about compatibility can follow the same recipe.
(Just a closing comment about this: I tried this and it was really pretty simple, just making a dict subclass.) Ram.
cool-RR wrote:
With the __deepcopy__ method, user-defined objects can specify how they will be copied. But it is assumed that you will always want to copy them the same way. What if sometimes you want to copy them in one way and sometimes in another?
Then you need to define your own system of copying methods and implement them appropriately for the classes they apply to. The deepcopy mechanism is only designed to cover simple cases. It isn't, and can't be, all things to all people. -- Greg
participants (8)
-
cool-RR -
Greg Ewing -
Jacob Holm -
MRAB -
Nick Coghlan -
Ram Rachum
-
Terry Reedy -
Vitor Bosshard