[Twisted-Python] Clean pb solution for two-way object sync?
(Apologies for double send but I forgot the subject line) Dear Twisted users, I recently found myself implementing a design pattern that I think twisted.pb was specifically designed to address. I think I'm not using pb correctly so I'd like advice. This is a somewhat longish post because I need to describe the problem I'm trying to solve. I have done internet searches on Stack Overflow and this list but have not found the answer to my question. If I've missed something kindly direct me to the appropriate reference. I want to implement something functionally equivalent to a network chess game. I first consider how I would do this on a single computer with no network (maybe this is bad thinking). Each piece in the game is represented by an instance of class Agent. Each agent has a .graphics attribute which is an instance of a class from a GUI toolkit library or equivalent. Whenever an agent in the game needs to do something there will be business logic executed by the game objects proper (ie the agents) which will invoke methods on the .graphics objects to update the screen. This sort of structure seems natural as it allows easy integration of drag/drop, mouse click detection etc. It also nicely separates the real business logic from the GUI. Now I want to run over the network. The question is how should I set up references between the client and server objects? Surely the server will maintain a set of objects representing the pieces in the game. It seems reasonable that each user's program will have a corresponding set of objects (with .graphics attributes). The issue is, what do we mean by "corresponding" and how do these objects talk to one another? Following is my idea so far: Each instance of AgentClient has a .server attribute which is a remote reference to an instance of AgentServer, and each instance of AgentServer has a .clients attribute which is a list of remote references to instances of AgentClient. class AgentServer(pb.referenceable): def remote_move(self, targetSquare): """Handle move request from client""" if self.thisMoveIsLegal(targetSquare): self.position = targetSquare for client in self.clients: client.callRemote("move", targetSquare) def thisMoveIsLegal(self, targetSquare): <check that this is a legal move> class AgentClient(pb.referenceable): def requestMove(self, targetSquare): """Tell server we'd like to move""" self.server.callRemote("move", targetSquare) def remote_move(self, targetSquare): """Server told us we moved""" self.position = targetSquare self.graphics.setNewPosition(targetSquare) This isn't THAT bad. The client's requestMove is thin and unecessary (I put it there for illustration). Still I need to have two separate classes with corresponding methods to handle moving the piece. This seems like the kind of thing I could twisted.pb to solve more cleanly if I only would look in the right place. This problem gets even worse when I think about how to birth new in-game objects. It would have to look like this: class PlayerServer(pb.referenceable): def newAgent(self, asker): """Client told us it wants a new Agent""" if self.thisIsLegal(): a = AgentServer() self.agents.append(a) for client in self.clients: d = client.callRemote("newAgent", a) d.addCallback(lambda obj: a.clients.append(obj)) class PlayerClient(bp.referenceable): def requestNewAgent(self): """Tell the server we want to spawn a new Agent""" self.server.callRemote("newAgent", self) def newAgent(self, serverObj): a = AgentClient() self.agents.append(a) a.server = serverObj return a This just looks wrong. Any advice? Thank you in advance for your help. Regards, Daniel Sank
On Sep 25, 2013, at 11:05 PM, Daniel Sank <sank.daniel@gmail.com> wrote:
This isn't THAT bad. The client's requestMove is thin and unecessary (I put it there for illustration). Still I need to have two separate classes with corresponding methods to handle moving the piece.
That's OK. Don't try to reduce your number of classes just for the sake of having fewer classes. Each class should have a clearly defined responsibility. In this case, your responsibilities correspond directly to the things that have to happen on the server (validating the move) and the things that have to happen on the client (updating the graphical representation to correspond to the new game state). Having more, smaller classes means that it's easier for you to understand each class on its own, and programming is nothing if not the act of promoting local understanding :). If anything, you should have an additional class to separate out your remote_* responders and the actual internal state.
This seems like the kind of thing I could twisted.pb to solve more cleanly if I only would look in the right place.
I don't know if this is going to directly address any of your concerns, but have you considered using twisted.spread.flavors.Cacheable? That will atomically combine the propagation of initial state with the provision of the back-propagation channel for updates to that state. It's designed for exactly what you're doing, maintaining parallel simulated states on client and server. Does that help at all? -glyph
If anything, you should have an additional class to separate out your remote_* responders and the actual internal state.
Indeed.
I don't know if this is going to *directly* address any of your concerns, but have you considered using twisted.spread.flavors.Cacheable?
This is funny. I read the documentation on Cacheable a few times and eventually figured the warning about it being "hard to understand" was there for a reason. I'll check it out in earnest now that I know it's relevant. Many thanks. Regards, Daniel Sank P.S. Thanks for Twisted. It rocks.
Upon re-reading the Cacheable docs I still don't understand how to use it. Can we take my previous home-brewed example and use that as a launch point for illustrating how to use Cacheable? Here's the example. We assume the client has a PlayerClient instance with .server pointing to a PlayerServer instance. The PlayerServer has a .client list of PlayerClients. The code fragments are the methods needed to spawn a new Agent and hook up the client and server objects. class PlayerServer(pb.referenceable): def newAgent(self, asker): """Client told us it wants a new Agent""" if self.thisIsLegal(): a = AgentServer() self.agents.append(a) for client in self.clients: d = client.callRemote("newAgent", a) d.addCallback(lambda obj: a.clients.append(obj)) class PlayerClient(bp.referenceable): def requestNewAgent(self): """Tell the server we want to spawn a new Agent""" self.server.callRemote("newAgent", self) def newAgent(self, serverObj): a = AgentClient() self.agents.append(a) a.server = serverObj return a How does Cacheable help do this?
On 09/27/2013 05:48 AM, Daniel Sank wrote:
Upon re-reading the Cacheable docs I still don't understand how to use it.
Have you seen this: http://twistedmatrix.com/documents/current/core/howto/pb-copyable.html#auto9 Essentially, you move all attribute access to accessor methods and do callRemote to propagate the changes out to observers; new observers are passed to you in getStateToCacheAndObserveFor. Observers respond to observe_xxx methods, and implement a setCopyableState method. Then you map the cacheable and observer with pb.setUnjellyableForClass
Have you seen this:
http://twistedmatrix.com/documents/current/core/howto/pb-copyable.html#auto9
No, I hadn't. That example is extremely helpful, thank you. I just realized that the documentation pages I'd been reading are all linked from here: http://twistedmatrix.com/documents/current/core/howto/index.html but don't link to each other, which is why I didn't find the page you referenced in your post. I should learn to pay attention to URLs :P Regards, Daniel
Hi Daniel, If you're interested in PB, you may also be interested in Foolscap, the object-capability extension to PB. Foolscap lives at: http://foolscap.lothar.com/trac Feature overview: http://foolscap.lothar.com/trac/wiki/FoolscapFeatures cheers lvh
That will atomically combine the propagation of initial state with the provision of the back-propagation channel for updates to that state.
My understanding of Cacheable is that it propagates initial state to the RemoteCache and then sends subsequent updates also to the RemoteCache. What I was originally asking about was how to allow the side holding the RemoteCache to request changes on the Cacheable. Your phrase "back-propagation" leads me to think that maybe this kind of thing is built into Cacheable, but I have not discovered how to use it. All of that said, in the end the answer to my original question seems to be "read this howto page:" http://twistedmatrix.com/documents/current/core/howto/pb-cred.html and then use the mind argument. Sincere thanks for your help, Daniel
On 09/26/2013 02:05 AM, Daniel Sank wrote:
I want to implement something functionally equivalent to a network chess game. I first consider how I would do this on a single computer with no network (maybe this is bad thinking). Each piece in the game is represented by an instance of class Agent. Each agent has a .graphics attribute which is an instance of a class from a GUI toolkit library or equivalent. Whenever an agent in the game needs to do something there will be business logic executed by the game objects proper (ie the agents) which will invoke methods on the .graphics objects to update the screen. This sort of structure seems natural as it allows easy integration of drag/drop, mouse click detection etc. It also nicely separates the real business logic from the GUI.
I think you have the right idea but that's still a bit too much coupling between the logic and the UI for my taste. I don't want the game logic to have a .graphics attribute; I want the game logic to fling game events to one or more consumers, each of which may or may not be a GUI. (Maybe it's a headless AI player. Maybe it's a logging service. The server shouldn't care.)
Now I want to run over the network. The question is how should I set up references between the client and server objects?
There's more than one way to do it. Here's my game that uses PB: https://github.com/dripton/Slugathon I used PB (because AMP and Foolscap didn't exist yet), but I didn't use the fancy bits of PB like Cacheable, because I strongly prefer simple remote method calls to fancy remote objects. But if you grep for callRemote, remote_, and perspective_, you can see how I did it. As noted above, my game server flings events (see Action.py for what they look like) to both GUI and AI clients. The actions are just little value objects that happen to inherit from pb.Copyable and pb.RemoteCopy for convenience, though they just as easily be JSON blobs. Of course, it's probably much easier to just use Cacheable. It comes down to programmer preference. One piece of advice: do the network code first and always exercise it, even when playing on a single computer. Every time I've written a single-machine game first then tried to add networking later, the networking has been a mess to debug. -- David Ripton dripton@ripton.net
Here's my game that uses PB: https://github.com/dripton/Slugathon
This has been useful. I am learning from it and my own experimentation.
One piece of advice: do the network code first and always exercise it, even when playing on a single computer. Every time I've written a single-machine game first then tried to add networking later, the networking has been a mess to debug.
The way my classes are set up I can't run the game without the network bits. Glad to hear this was good thinking.
participants (5)
-
Daniel Sank -
David Ripton -
Glyph -
Laurens Van Houtven -
Phil Mayers