[Twisted-Python] Using deferreds when writing across unreliable network
I have no concrete solution to what you describe, but I can try and explain how Deferreds work.
The crucial thing is that "someone" or "something" must call the callback method on your Deferred. The general rule to follow is that it is the code that creates a Deferrred which has the reponsibility of calling the callback method at the correct time.
So when you use the getPage function you get a Deferred back. Behind the scenes the getPage function has setup things in such a way that the callback method of the returned Deferred is called when the page has arrived -- you must not invole callback yourself in this case.
But if you create a Deferred yourself, then it is up to you to make a call to the callback method when ready.
I hope this helps a bit -- it helped me to look at the implementation in defer.py and see that it was less scary than I had imagined :-)
-- Martin Geisler
Martin Thanks for the reply. That does clarify that I need to fire off my own deferreds. The problem it leaves me at the moment is: How do I tell when my transport.write has completed. The ACKs for my messages happen at the TCP level. Does this mean that for writes and reads I have to work at the application level to register when the write has been successful and fire my callback then. By the way I did find your example, but I could not reconcile why some deferred examples had 'automatic' callbacks and other fired their own. Yes. I have looked through the defer.py code to see what was going on. It is certainly well laid out and easy to follow - just the basic concept needed clearing up. Thanks John Aherne
"John Aherne" <johnaherne@rocs.co.uk> writes:
Martin
Thanks for the reply.
That does clarify that I need to fire off my own deferreds.
Excellent! :-)
The problem it leaves me at the moment is:
How do I tell when my transport.write has completed. The ACKs for my messages happen at the TCP level. Does this mean that for writes and reads I have to work at the application level to register when the write has been successful and fire my callback then.
I think so -- you wont be told when the actual bytes have been pushed onto the network, nor when they have arrived succesfully at their destination. You have to add such logic yourself, maybe by having the recepient send back a confirmation. So one way to structure this is to let the code that calls transport.write return a Deferred, call it d. Before returning it, the code stores d in a well-known place, something like transport.write(...) d = Deferred() outstanding requests[some_id] = d return d The code that handles incoming data will then have to recognize the confirmation and will invoke the callback method on d: def dataReceived(self, data): id, payload = unpack(data) d = outstanding_requests[id] d.callback(payload) This is sort of how my code works in the VIFF project.
By the way I did find your example, but I could not reconcile why some deferred examples had 'automatic' callbacks and other fired their own.
That is because of the different ways they are used: Defered is the basic building block, but DeferredList combines several Deferreds into one. -- Martin Geisler VIFF (Virtual Ideal Functionality Framework) brings easy and efficient SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
participants (2)
-
John Aherne -
Martin Geisler