[Twisted-Python] Request for ideas

Warning - fairly long post follows for what I hope is a simple question that only needs (possibly) a short answer. (I'm not looking for any code, just one of a couple possible ideas.) Environment: A LAN with a server and multiple PCs. Full control is available over all the PCs. I have the ability to run whatever I want on each - in this case it means Python 2.4 / Twisted 2. Background / Use Case: Imagine a simultaneous silent auction. (More like an RFP-type situation.) Any of the PCs can put an item up for bid. Other PCs can respond (or ignore!) that item. There are two timeout requirements - an "acknowledgement" timeout, where the PC has a certain period of time to respond "I want to make a bid", and a response timeout that restricts the clients to how much time they have to make the bid. These timeouts can change depending on the item going up for bid. (How and 'what' goes up for bid is irrelevant to my question.) What I've done so far: I've created a class called "CommTrans" to manage the auction. Each time an item goes up for bid, an instance of CommTrans is created. CommTrans creates two objects, one each to manage the acknowlegements and the other to manage the replies (AckMgr, RepMgr). The RepMgr is a Viewable that can be called by the client to submit its bid. (RepMgr is passed to the client in the callRemote call.) CommTrans issues the callRemote, passing RepMgr to the client. Then it sets a function in the AckMgr as the callback. The client receives the request, and decides whether or not to respond to the request. If yes, it issues a callLater on the function to determine the bid and submit it to the RepMgr. Either way, the original function returns either "Yes" or "No", triggering the callback on the CommTrans. Both the AckMgr and RepMgr issue callLater to schedule the timeout. If all possible machines reply before the timeout occurs, the timeout is cancelled. If the timeout occurs first, a flag is set to have the objects reject all further information from the clients. (It's simply ignored, nothing special occurs.) QUESTION #1: Did I miss something in the Twisted API that would handle this type of situation? I don't think a Deferred list would work in this situation, because it's not going to fire until all deferrs complete - something which might not occur. But this isn't the problem - the code that I have written for these functions all work. The specific problem occurs when objects are reused. I've had situations where a client fails to reply for significantly longer than the timeout period. In that interval, the original CommTrans, AckMgr and RepMgr have been deleted and replaced by new instances. The client finally replies to the original request, and the reply is caught by those new objects! (There is what I think is a very small, but non-zero probability that I'm misinterpreting what I'm seeing.) From what I can see in the source code, the internal ID is what's set by the builtin 'id' function. I'm printing these at key times and I can see where they're being duplicated. The only time I see this problem occur is when a new object has been created with the same id as a previous object. So this leads me to QUESTION #2: Anyone have ideas on how to resolve this? (Note: I can't channel everything through a single CommTrans object, because multiple bids might be happening concurrently with overlapping timeouts.) About the only thought I've had so far is to generate a GUID-like object for each CommTrans, and include that in every communication to and from the clients. The server would check each ack and each reply to ensure that the ack and reply are for the proper auction. Sending another message to the client isn't going to be a good solution, because the client might be tied up doing other things - starving the reactor - so by the time the message is processed, it may be too late. This also brings up QUESTION #3, although it hasn't happened yet: Server does a callRemote to the client from a commTrans object. Before the remote function returns (client is _extremely_ busy) the commTrans object goes away. What will happen when the function exits? (I know, I ought to generate a test-case for this - I'm more wondering if anyone knows right off-hand.) All thoughts, ideas, comments, recommendations are encouraged. Thanks to any/everyone who managed to get this far through my writing. Finally, I'm most pleased to announce that my proposal to provide a "Getting Started with Twisted" tutorial at PyCon has been accepted! I know I can't turn people into Twisted gurus in 3 hours, but I hope I can get a few over that initial hurdle. Thanks, Ken

At 2005-12-03 04:24 PM -0500, you wrote:
The specific problem occurs when objects are reused. I've had situations where a client fails to reply for significantly longer than the timeout period. In that interval, the original CommTrans, AckMgr and RepMgr have been deleted and replaced by new instances. The client finally replies to the original request, and the reply is caught by those new objects! (There is what I think is a very small, but non-zero probability that I'm misinterpreting what I'm seeing.)
From what I can see in the source code, the internal ID is what's set by the builtin 'id' function. I'm printing these at key times and I can see where they're being duplicated. The only time I see this problem occur is when a new object has been created with the same id as a previous object.
Sounds like an object aliasing problem. And I don't think that it's *objects* that are being re-used, but *memory locations* for same-class objects. Don't use the built-in id() function. I'm not sure if this is correct, but it seems to me something like the memory address of the object. When the memory gets re-used, the same ID sometimes recurrs. ***** If you need a unique identifier for your application logic, create and assign it yourself. Don't use the id() function. ***** One possibility is to give each AckMgr/RepMgr pair an auction ID. Create a simple ID generator (object or function). When you create a new AckMgr and RepMgr, get a new ID from the generator and assign it to each one *before* you use it. The AckMgr then could simply ignore any response that doesn't have the expected ID. More generally, I would put the AckMgrs in a dictionary, keyed by the action ID. I would have *one* function/method (ProcessAck) that gets responses and dispatches them to the appropriate AckMgr, or drops them if no matching AckMgr is found. This would make testing, debugging, and tuning easier, too, by providing a single location for instrumentation. - Sam __________________________________________________________ Spinward Stars, LLC Samuel Reynolds Software Consulting and Development 303-805-1446 http://SpinwardStars.com/ sam@SpinwardStars.com
participants (2)
-
Ken Whitesell
-
Samuel Reynolds