[Twisted-Python] pb objects unexpectedly change identity
Dear twisted users, I think I have found some surprising behavior in perspective broker. I define a subclass of pb.RemoteCache which has a method managed by a descriptor. The descriptor keeps track of the instances it manages in a set. When the RemoteCache is first received by the client, it accesses the descriptor-ified method inside setCopyableState. At that time the RemoteCache's id is a certain value X. Then later when the RemoteCache is notified of a change by the server side Cacheable, it again accesses the descriptor-ified method, but at this time it's id is Y, and Y!=X. I attach to this post a simple working example which displays the behavior described above. To run, first run server.py and then run client.py. You will see a little bit of output, the most important part being === Client received RemoteCache: id=45952496 Asking server to update TD 45921680 (test.RemoteCache.add) accessed by <test.RemoteCache instance at 2BD2DC8>: id=45952456 RemoteCache id=45952496 === To understand the details of the output please see descriptor.py. The id of the RemoteCache is first reported as 45952496 when the client receives it. Then, when the descriptor is accessed, the id of the accessing instance is reported as 45952456, which is different. Then, when in the last line we print out the id of the RemoteCache we're back to 45952496. Is there some reason that a RemoteCache's id can change during its life time? -- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
Dear Twisted users, To my amazement I have reproduced this strange behaviour in an even simpler program. Attached are a pb client/server which illustrate unexpected behaviour of the id of a pb.RemoteCache. To run the example, run server.py and then client.py (with resources.py in the same working directory). You will see output like this (line numbers added by me) ==== 1. RemoteCache 29403544 initialized 2. Client received RemoteCache: id=29403544 3. RemoteCache: while responding to observe_add I think my id is 29403760 4. Client: The RemoteCache's ide is 29403544 ... ==== 1. When the RemoteCache is initialized it thinks its id is 29403544. 2. The client agrees that it has received a RemoteCache with id 29403544. 3. When the RemoteCache is inside observe_add it thinks its id is 29403760. 4. The client still thinks the RemoteCache's id is 29403544. If you let the program run it will continue to loop between the RemoteCache reporting its id while in observe_add, and the client reporting the id of the RemoteCache. The two reported ids are always the same unequal numbers. The question appears to be "Why does the id reported by a RemoteCache's while inside an observe_* method differ the id reported by objects owning references to that RemoteCache?" Thank you for your time, Daniel On Wed, Apr 30, 2014 at 11:00 AM, Daniel Sank <sank.daniel@gmail.com> wrote:
Dear twisted users,
I think I have found some surprising behavior in perspective broker. I define a subclass of pb.RemoteCache which has a method managed by a descriptor. The descriptor keeps track of the instances it manages in a set.
When the RemoteCache is first received by the client, it accesses the descriptor-ified method inside setCopyableState. At that time the RemoteCache's id is a certain value X. Then later when the RemoteCache is notified of a change by the server side Cacheable, it again accesses the descriptor-ified method, but at this time it's id is Y, and Y!=X.
I attach to this post a simple working example which displays the behavior described above. To run, first run server.py and then run client.py. You will see a little bit of output, the most important part being
=== Client received RemoteCache: id=45952496
Asking server to update TD 45921680 (test.RemoteCache.add) accessed by <test.RemoteCache instance at 2BD2DC8>: id=45952456 RemoteCache id=45952496 ===
To understand the details of the output please see descriptor.py. The id of the RemoteCache is first reported as 45952496 when the client receives it. Then, when the descriptor is accessed, the id of the accessing instance is reported as 45952456, which is different. Then, when in the last line we print out the id of the RemoteCache we're back to 45952496.
Is there some reason that a RemoteCache's id can change during its life time?
-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
Dear twisted users, I have discovered the cause of this strange behavior. This is a good one. The object exposed to you as a client when you receive a RemoteCache is not the same object which is passed in as "self" to observe_* methods. In the source code, there is a pair of objects created when a RemoteCache is received. One of them is given to the client, and the other remains in the shadows of pb for reasons I don't yet understand. For some reason remoteMessageReceived delegates observe_* methods to the object which is not the one given to the client. This means that normal methods and observe_* methods get different self arguments. The authors seemed to try to cover this up by forcing the two objects to share the same __dict__ and __class__. Of course this does not give the objects the same id, which was the source of my problem. I have reported this issue here: https://twistedmatrix.com/trac/ticket/7274 I am interested in fixing this bug. What is the right way to get help from people who know the code? I have some simple questions I'd like to ask so that I go in the right direction while fixing this. -Daniel On Wed, Apr 30, 2014 at 8:09 PM, Daniel Sank <sank.daniel@gmail.com> wrote:
Dear Twisted users,
To my amazement I have reproduced this strange behaviour in an even simpler program. Attached are a pb client/server which illustrate unexpected behaviour of the id of a pb.RemoteCache. To run the example, run server.py and then client.py (with resources.py in the same working directory).
You will see output like this (line numbers added by me)
==== 1. RemoteCache 29403544 initialized 2. Client received RemoteCache: id=29403544 3. RemoteCache: while responding to observe_add I think my id is 29403760 4. Client: The RemoteCache's ide is 29403544 ... ====
1. When the RemoteCache is initialized it thinks its id is 29403544. 2. The client agrees that it has received a RemoteCache with id 29403544. 3. When the RemoteCache is inside observe_add it thinks its id is 29403760. 4. The client still thinks the RemoteCache's id is 29403544.
If you let the program run it will continue to loop between the RemoteCache reporting its id while in observe_add, and the client reporting the id of the RemoteCache. The two reported ids are always the same unequal numbers.
The question appears to be "Why does the id reported by a RemoteCache's while inside an observe_* method differ the id reported by objects owning references to that RemoteCache?"
Thank you for your time, Daniel
On Wed, Apr 30, 2014 at 11:00 AM, Daniel Sank <sank.daniel@gmail.com>wrote:
Dear twisted users,
I think I have found some surprising behavior in perspective broker. I define a subclass of pb.RemoteCache which has a method managed by a descriptor. The descriptor keeps track of the instances it manages in a set.
When the RemoteCache is first received by the client, it accesses the descriptor-ified method inside setCopyableState. At that time the RemoteCache's id is a certain value X. Then later when the RemoteCache is notified of a change by the server side Cacheable, it again accesses the descriptor-ified method, but at this time it's id is Y, and Y!=X.
I attach to this post a simple working example which displays the behavior described above. To run, first run server.py and then run client.py. You will see a little bit of output, the most important part being
=== Client received RemoteCache: id=45952496
Asking server to update TD 45921680 (test.RemoteCache.add) accessed by <test.RemoteCache instance at 2BD2DC8>: id=45952456 RemoteCache id=45952496 ===
To understand the details of the output please see descriptor.py. The id of the RemoteCache is first reported as 45952496 when the client receives it. Then, when the descriptor is accessed, the id of the accessing instance is reported as 45952456, which is different. Then, when in the last line we print out the id of the RemoteCache we're back to 45952496.
Is there some reason that a RemoteCache's id can change during its life time?
-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899
participants (1)
-
Daniel Sank