-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 23 Mar 2003, Brian Warner wrote: Thanks for the response! [snip: pb.Copyable copies all references found]
However, I'm sometimes getting an exception when the actual dictionary copying is done, as something other than a dict is being copied into __dict__. At this point the "jelType" is "dereference"...
[snip]
If it's something else, let us know (and provide a small test case??) so we can fix it at the sprint.
Here's the relavent end of the error: File "..\twisted\spread\flavors.py", line 386, in setCopyableState self.__dict__ = state exceptions.TypeError: __dict__ must be set to a dictionary I've included a test case at the end of the message.
Damn, it looks like this might be the cuplrit. "reference" jelyTypes are recursively descended into before they are stored, and if a dereference is found before it's stored... some sort of _Dereference object is created? An attempt is then made to copy this into __dict__, and boom.
Yes, the current Jelly code looks for objects that are referenced multiple times in the same jellying call and marks them with "reference" tags. When another reference to the same object is detected, it is jellied with a "dereference" tag that points to the earlier "reference" marker. The "cook", "prepare", and "preserve" methods are used to implement these multiple phases. Circular or recursive references are handled because the reference number is allocated when we start to jelly the object, even though the state is not yet known.
The number is allocated earlier on, but it isn't stored for use by "derference" jelTypes until after recursion, as I will attempt to describe: Here are the relavent parts of jelly._Unjellier. Note that the reference is stored after it's value is recursively computed -+ | def _unjelly_reference(self, lst): | refid = lst[0] | exp = lst[1] | o = self.unjelly(exp) <--------+ ref = self.references.get(refid) <--------+ if (ref is None): self.references[refid] = o elif isinstance(ref, NotKnown): ref.resolveDependants(o) self.references[refid] = o else: assert 0, "Multiple references with same ID!" return o This ends up causing the following method to return a _Dereference, rather than a state dictionary that unjelly() would normally return. This then results in the error I list above. def _unjelly_dereference(self, lst): refid = lst[0] print refid #!!! added x = self.references.get(refid) if x is not None: return x der = _Dereference(refid) self.references[refid] = der return der
This scheme will change on Tuesday. The "reference" tags will go away and be replaced by an implicit marker that is notionally inserted every time we start jellying a new mutable object. The "dereference" tags will then point to these implicit markers. This should improve performance quite a bit, and will pave the way to a combined jelly+banana extension module that should give an enormous speedup (doing everything in C).
Sounds good! No point in my suggesting a fix then. ;-) Will a new version of twisted be released along with these changes, or would I have to go through CVS? [snip] Here are 3 files for reproducing the "Unjellying a Circular Reference Bug". Just stick them all in the same place, and start the server then the client. - -Jasper Listing for objectBug.py =-=-=-=-=-=-=-=-=-=-=-=- from twisted.spread import pb class ClassA( pb.Copyable, pb.RemoteCopy ): def __init__( self ): self.ref = ClassB( self ) class ClassB( pb.Copyable, pb.RemoteCopy ): def __init__( self, ref ): self.ref = ref import sys pb.setCopierForClassTree( sys.modules[__name__], pb.Copyable ) Listing for serverBug.py =-=-=-=-=-=-=-=-=-=-=-=- #/usr/bin/env python from twisted.internet import app from twisted.spread import pb from twisted.cred import authorizer import clientBug, objectBug class MyPer( pb.Perspective ): def attached( self, client, identity ): self.client = client return pb.Perspective.attached( self, client, identity ) def perspective_receive( self, obj ): self.client.callRemote( "receive", obj ) class MyService( pb.Service ): perspectiveClass = MyPer def __init__( self, serviceParent, auth ): pb.Service.__init__( self, "MyService", serviceParent, auth ) def startServer(): myApp = app.Application("pbServer") auth = authorizer.DefaultAuthorizer( myApp ) s = MyService( myApp, auth ) s.createPerspective( "player1" ).makeIdentity( "password1" ) myApp.listenTCP( 9000, pb.BrokerFactory( pb.AuthRoot( auth ))) myApp.run( save=0 ) if __name__ == '__main__': startServer() Listing for clientBug.py =-=-=-=-=-=-=-=-=-=-=-=- #/usr/bin/env python from twisted.internet import reactor from twisted.spread import pb import objectBug class MyClient( pb.Referenceable ): def connect( self, ipAddress, port, user, password ): defer = pb.connect( ipAddress, port, user, password, "MyService", client=self, timeout=3 ) defer.addCallback( self.connected ) def connected( self, perspective ): perspective.callRemote( "receive", objectBug.ClassA() ) def remote_receive( self, obj ): print "Object received back" def startClient(): client = MyClient() client.connect( "localhost", 9000, "player1", "password1" ) reactor.run() if __name__ == '__main__': startClient() -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE+f45+8EpjZ7/X9bIRAr0yAJ9P2dwozcY478sVL6GKemwYoHAZmwCfYbhT ppjMznAw88wjXItJDczxTVU= =LvPx -----END PGP SIGNATURE-----