[Twisted-Python] Multiplex

Preface: Currently, Napkin blots (the things that get drawn on the napkin) are immutable (in practice, if not in name). They go out over the wire and they're forgotten, presumably having been rendered on the other end. But this evening, dash asked me to revisit the idea of moving around objects which had been previously placed on the napkin. The following is what I would like to have to implement that goal -- but I believe it will be useful for far more than just Napkin. Even now, the core Napkin service has practically nothing to do with the idea of "whiteboards", and is just concerned with the far more general task of distributing information. What I've done with Napkin so far has taught me enough about PB that I can see that this can be done. Of course, there's a possibility that Twisted already does this, and I have not yet uncovered it. If that's the case, speak up! ---- Dramatis personae: Bob -- runs a PB.Multiplex service on twistedmatrix.com. Alec, Amy, and Angus -- are each "Leaf nodes" of the service. They might be called peers, subscribers, listeners, or observers, although the last three terms, describing passive occupations, are probably not most appropriate. kermit -- don't look for him in the dramatis personae, he's just an instance of a Frog. ---- The Goal: is quite simple. When Amy moves the red circle, it moves on Alec and Angus's napkins as well. When Angus hits the "change artist" button, Amy and Alec's players switch tunes too. When Alec clicks on the frog, all three workstations croak. Get the idea? Example: Alec has a Frog object (referred to as 'kermit') from Bob's multiplex service. Alec invokes kermit.shout_croak(). The 'shout_' prefix indicates that this action does not take place locally, but rather it happens world-wide over the multiplexer. Somewhere in kermit's private parts, there is a Proxy referring to kermit's counterpart in Bob's service. kermit does getattr(self._proxy, 'croak'), and a short time later... ...kermit and his other avatars on Amy and Angus's machines get their .remote_croak() methods invoked. Amy and Alec happen to be running the same client, one which provides an audio environment, and a giant "Ribbit" echos through each of their rooms. Angus is running a minimalistic text client on his Wyse 85 terminal, and sees a discrete "kermit: *croak*" message. Somewhere on the far side of the galaxy, a Sirius Cybernetics Genuine People Personality Prototype gets the message, promptly self-terminates, and falls to the floor. ---- As you can see, while kermit's interface is the same in each of the clients, the local implementation of the rendering methods differs. (This is why I thought I might want a Referenced class object I could descend Frog from in each of the clients, but I'm not sure it's practical to broker classes.) This can be achieved through a mechanism very similar to that used for Copied/Copy, there's a tag identifing the copied serialized Frog, and the Copy is constructed with a local implementation of Frog, one of Amy's choosing. On Amy's end, the difference between MultiplexLeaf and Copy is that kermit must retain a link back to Bob's place, where his spreadable self resides in a form resembling Proxied. On Bob's end, the difference between Multiplexed and Proxied is that the Multiplexed kermit must retain references to all the nodes/leaves of himself, so that when someone shout_s at him, he can tell all his selves to do the same. That also means that aside from the construction, a MultiplexLeaf behaves much more like a Referenced object than a Copy. ---- So now we have an object which when we wiggle its toe here, it wiggles its toe there. That's pretty good, but the exciting part is when we use it as a container or a vechile. Say my Napkin is Multiplexed. Now I say napkin.shout_draw(blot), with the Blot class being Multiplexed as well. When the blot gets passed from Bob to Alec and friends, leaves of the blot are constructed in their clients -- and references to those leaves are held in the Multiplexed blot. Now, when I say blot.shout_rotate(90)... Isn't this fun? ---- Questions answered: Q1. Why does Alec have to use the shout_ prefix? Don't the adverb prefixes usually go where the method is defined, on the receiving end? A1. Remember, Alec's Frog is not just a Proxy or a Reference. kermit was constructed an instance of a class defined on Alec's side, a class that likely has methods of its own, which Alec's program would like to use locally. Say Angus has a blot. He'll want that blot drawn in his view locally, while he positions/sizes it, before he commits it to the world. Having an interface to manipulate the blot locally (rather than through the multiplexer) is essential here, as I'm making no distinction between the-local-object-on-my-foo-canvas and the network-object. Issues not addressed here (but not showstoppers): 1) Subscription -- how do you join ranks with Alec, Amy, and Angus? This is closely related to the question of 2) Construction -- do multiplex objects have to be issued by Bob's service, or do they just spring into being whenever someone passes a MultiplexLeaf through the serializer? Just what role *is* Bob's service playing, anyway? But you know something? Bob's service doesn't really have to know anything about Blots or Frogs or Napkins in particular... it just has to pass along the right copyTags, so that compatible objects are constructed in each of the peers. Another cool-thing point: I've found no reason why you shouldn't be able to chain Multiplexed objects. Just have it look like Angus's peer on one side, and one of Bob's multiplexed objects on the other, and it should function in both directions. This lets you build more interesting network-topologies if you don't want all the peers connected to one point. That's all I've got for tonight, boys and girls. Hope you found reading this worthwhile. So tell me, whaddya think? Will it work?

Ok, Glyph is givin' me funny looks. The idea that hooked me in to that whole mess was the bit about the peer getting a single object which has both locally-defined properties *and* can still act as a Reference. again, the idea analogous to subclassing. Python checks the subclass for the attribute first, and then checks its base classes. I want to have the object search its local class first, and then fall back to being a Reference. I dunno how to do that currently. The two-way aspect is important to me as well, but that's already covered.

This is what Cached was designed for :-). You do realize, however, that even in a Copied, you may return something like {'foo': self.foo, 'remote', Proxy(perspective, self)} from YourCopied.getStateToCopyFor, right? Then you can copy your cake and proxy it too... On Wed, 8 Aug 2001, Kevin Turner wrote:
Ok, Glyph is givin' me funny looks.
The idea that hooked me in to that whole mess was the bit about the peer getting a single object which has both locally-defined properties *and* can still act as a Reference.
again, the idea analogous to subclassing. Python checks the subclass for the attribute first, and then checks its base classes. I want to have the object search its local class first, and then fall back to being a Reference. I dunno how to do that currently.
The two-way aspect is important to me as well, but that's already covered.
______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph

On Wed, Aug 08, 2001 at 06:48:59AM -0500, Glyph Lefkowitz wrote:
This is what Cached was designed for :-).
I must be missin' something. Here's my attempt. Problems include not being sure if I can really talk in the Cache -> Cached direction, and the transparent automatic repeater creation alchemy doesn't even come close. (no, it doesn't run. yes, the names suck. but none of that matters if it doesn't work at all.) class PlexPerspective(Perspective): """This is the perspective given by Bob's repeater service.""" def perspective_setPlexTag(self, tag): """Register a copytag as being a plexed class.""" # Do some sanity checking, make sure tag has a safe prefix # like "PLEX_" or somesuch. pb.setCopyTags(tag, PlexRepeater) class PlexBase(Copied): """Plex base class -- always returns a consistant copytag (even if it's been through a repeater, where everything is a PlexRepeater, no matter what the tag.) """ def getStateToCopy(self): # Make sure _copytag gets into our state, which it otherwise # wouldn't do if our _copytag refers to the class attribute. state = pb.Copied.getStateToCopy(self) state['_copytag'] = self._copytag return state def getTypeToCopy(self): return self._copytag class PlexRepeater(PlexBase, Cached, Copy): """This is how 'plexes look in the repeater. Repeaters don't have unique properties as they're all instances of this class, but they must remember the copytag to give to their children.""" def setCopiedState(self, state): """Leaf -> Repeater alchemy. This is so new plexed objects may be created transparently, if a peer just passes through an instance of their leaf-class.""" # XXX: How do I find out which broker I arrived through? # There isn't a setCopiedStateFor() moms_perspective = moms_broker.getPerspective() # I need to convince that broker that Mom is really a # remote cache of me. puid = self.ProcessUniqueID() moms_broker.remotelyCachedLUIDs[puid] = moms_luid moms_broker.remotelyCachedObjects[moms_luid] = Local(self) # Add Mom as an Observer observer = CacheObserver(moms_broker, self, moms_perspective) # Don't need to get the state, actually, but that is the # method that keeps my observer list healthy. self.getStateToCacheAndObserveFor(moms_perspective, observer) # XXX: Now I have to convince Mom's broker on her end to make # a CacheProxy for me, with Mom as the cNotProxy. state['_receive_daughter'](self) pb.Copy.setCopiedState(self, state) def remoteMessageReceived(self, broker, message, args, kw): """Pass the word along to all the kids.""" # ??? Do I receive remote messages? # Or do I have to pass this method as part of my cached state? for o in self.observers: self.remoteCacheDo(bla, bla, args, kw) def getStateToCacheAndObserveFor(self, perspective, observer): """keep a list of my kids.""" self.observerlist.append(observer) return self.getStateToCopyFor(self, perspective) class PlexLeaf(PlexBase, Cache): """This is subclassed by Alec, Amy, and Angus. it will be the setCopierForClass for a PLEX_ copytag, and thus created whenever a peer (most likely the repeater) passes a plex to us. It receives updates from the plex object it caches, and can talk back to that object as well (causing it to broadcast, if it is a repeater). My subclass should define a _copytag attribute to identify me when I'm passed to my peer if I was initialized locally, and not by the broker as a Cache. """ def __getattr__(self, key): # Shout out through the plex if key[:6] == "shout_": return getattr(self.cached, key[6:])

On Sun, 12 Aug 2001, Kevin Turner wrote:
On Wed, Aug 08, 2001 at 06:48:59AM -0500, Glyph Lefkowitz wrote:
This is what Cached was designed for :-).
I must be missin' something. Here's my attempt. Problems include not being sure if I can really talk in the Cache -> Cached direction, and the transparent automatic repeater creation alchemy doesn't even come close.
Yes, you can. See the (woefully incomplete, I know) docs for twisted.spread.pb.Cached.getStateToCacheAndObserveFor and let me know if it makes sense...
(no, it doesn't run. yes, the names suck. but none of that matters if it doesn't work at all.)
Still not sure if I understand, though I'll give the code a more thorough read-through later... Is there a single object that you're replicating, or do you want *any* client to be able to publish a repeater to the server that any other client can observe? ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph
participants (2)
-
Glyph Lefkowitz
-
Kevin Turner