Mailman 3 October 2008 - Twisted

[Twisted-Python] DOUBTS PLEASE SOLVE
by shweta mani Oct. 10, 2008

Oct. 10, 2008

Hi al i am new to twisted. I had been assigned a project on SSH. the project is as follows. i have to login to a remote Linux machine through Windows i have to execute a shell script and print the output back in Windows machine this needs to be done through SSH. Any clues . please help. i need it urgently

6 12

[Twisted-Python] Using deferreds when writing across unreliable network
by John Aherne Oct. 10, 2008

Oct. 10, 2008

I think so -- you wont be told when the actual bytes have been pushed onto the network, nor when they have arrived succesfully at their destination. You have to add such logic yourself, maybe by having the recepient send back a confirmation. So one way to structure this is to let the code that calls transport.write return a Deferred, call it d. Before returning it, the code stores d in a well-known place, something like transport.write(...) d = Deferred() outstanding requests[some_id] = d return d The code that handles incoming data will then have to recognize the confirmation and will invoke the callback method on d: def dataReceived(self, data): id, payload = unpack(data) d = outstanding_requests[id] d.callback(payload) This is sort of how my code works in the VIFF project. > By the way I did find your example, but I could not reconcile why > some deferred examples had 'automatic' callbacks and other fired > their own. That is because of the different ways they are used: Defered is the basic building block, but DeferredList combines several Deferreds into one. Martin Thankyou very much for your replies. I have a feeling that I now know what I am doing or going to do. I also think your min-deferred example will be very useful. Thanks once again. John Aherne

1 0

[Twisted-Python] Using deferreds when writing across unreliable network
by John Aherne Oct. 10, 2008

Oct. 10, 2008

> I have no concrete solution to what you describe, but I can try and > explain how Deferreds work. > > The crucial thing is that "someone" or "something" must call the > callback method on your Deferred. The general rule to follow is that > it is the code that creates a Deferrred which has the reponsibility of > calling the callback method at the correct time. > > So when you use the getPage function you get a Deferred back. Behind > the scenes the getPage function has setup things in such a way that > the callback method of the returned Deferred is called when the page > has arrived -- you must not invole callback yourself in this case. > > But if you create a Deferred yourself, then it is up to you to make a > call to the callback method when ready. > > > I hope this helps a bit -- it helped me to look at the implementation > in defer.py and see that it was less scary than I had imagined :-) > > > -- > Martin Geisler > Martin Thanks for the reply. That does clarify that I need to fire off my own deferreds. The problem it leaves me at the moment is: How do I tell when my transport.write has completed. The ACKs for my messages happen at the TCP level. Does this mean that for writes and reads I have to work at the application level to register when the write has been successful and fire my callback then. By the way I did find your example, but I could not reconcile why some deferred examples had 'automatic' callbacks and other fired their own. Yes. I have looked through the defer.py code to see what was going on. It is certainly well laid out and easy to follow - just the basic concept needed clearing up. Thanks John Aherne

2 1

[Twisted-Python] Using deferreds when writing across unreliable network
by John Aherne Oct. 10, 2008

Oct. 10, 2008

I have been trying out twisted(2.5 for python 2.4 XPSP2) over the past few weeks and have some code running connecting with some mobile devices over GPRS. It works. But I am having problems with deferreds - which I find tricky to understand. I have setup a tcpserver which the mobile devices connect to and I send answers back and occasionally unsolicited messages out. I have made an assumption that I should use a defer to write back across the network. Maybe I am wrong. I have a function in which I set up the transport.write to send a message out. This I make a deferred and return the deferred and add my callBack and errBack functions. I can see them when I print out the callbacks so I know they are there. As far as I can see from my logs, the deferred runs but neither the callback nor the errback are fired. Possibly an error occurs that stops them but I have not yet found a way to pin this down. I see there is a setDebugging option but I have not yet worked out how I would use this. So before I start trying any more incorrect assumptions, what I would like to confirm is that I should expect my callback or errback functions to be called automatically when my deferred has executed and returned. If successful, it would return fairly promptly. If it fails it could take a number of minutes to fail. But that is another problem. So should my callback or errback fire without me having to initiate any action other than setting up the deferred. And should I be using a deferred at all for this situation. If I know that the basis of what I am doing is correct and I am still having problems I can try and put together a small bit of code that reproduces the problem on the assumption that my code is probably wrong. Thanks for any pointers. I'm afraid that everytime I read the documentation on deferreds and go through the examples on using them I come to different conclusions on what I should be doing. So any help would be much appreciated. One example I looked at was the getPage example. Apart from doing a read it seems to be much along the lines of what I want to do. I am writing using straight TCP rather than HTTP. But that example fires the callback and errback without any problems. So it makes me think that my deferreds have something wrong with them. John Aherne

2 1

[Twisted-Python] Buildbot history
by Jean-Paul Calderone Oct. 10, 2008

Oct. 10, 2008

Hey all, Due to some unfortunate behavior which began after I upgraded our build master to 0.7.8, I've just removed all of the history from our buildbot. This means all old links to build results are broken now. Hopefully this won't cause any major problems. Jean-Paul

2 3

[Twisted-Python] Translation for the Documentation
by Fantix King Oct. 9, 2008

Oct. 9, 2008

Hi, Just new to here, but been using Twisted for one year. Some of the articles in the documentation is really good. So I started some work to translate the Core Documentation into my main language. The project was hosted on googlecode, but obviously it is not the best place to deal with translation works, especially team work. So I want to move it to launchpad. Problem here is the launchpad people prefer a whole twisted project other than seperate translation projects. So here I am to request some help, mostly about turning on the translation module on launchpad for the project of twisted. Also, RFC on the method I intend to deal with the translation work. Here's some rough idea about translating the documentation. Now that launchpad uses PO format for translation, thus PO would be the better way. I found a tool named 'po4a' a really good stuff. It can "gettextize" the original xhtml document (aka master doc) to generate the POT file needed by launchpad, and 'translate' the master doc with PO files which are translated though launchpad. Meanwhile, po4a is pretty configurable and easy to use. Best regards, Fantix.

1 1

[Twisted-Python] Another tutorial?
by Kevin Anderson Oct. 9, 2008

Oct. 9, 2008

Greetings, Stepping through the mailing list archives a little, I can see that I am not the first who has read about Twisted and has gained some appreciation of how powerful and useful it is, but finds the tutorial documentation to be kind of lacking. I would take a stab at trying to improve the finger tutorial myself (that is, to explain the rationale behind some of the decisions and why they're useful), but is someone already in the process of doing so? (By the way, is there some way to submit patches for the documentation? I still see depreciated references to mktap, particularly in the "Using Twisted Web" howto, even though there are already old tickets about removing references to it.) -K

3 2

Re: [Twisted-Python] Packet Fragmentation
by Nima Ghanavatian Oct. 9, 2008

Oct. 9, 2008

Thanks for the detailed information! I tried it out with LineReceiver and everything worked great! -Nima ----- Original Message ---- From: "glyph(a)divmod.com" <glyph(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, October 8, 2008 4:13:18 PM Subject: Re: [Twisted-Python] Packet Fragmentation On 10:51 pm, nimag(a)rogers.com wrote: >I was wondering if there's any way in Twisted/Python to handle packet >fragmentation. Can I access the last fragment flag in the TCP packet >somehow? Or is there a total packet size sent with the packet >automatically? >If not, what would be the best way to solve this? What you're dealing with here is not packet fragmentation. TCP packet fragmentation is a very specific phenomenon that occurs at a lower level than Twisted (and your application). When you send a large message over a TCP stream, it can get broken up into multiple segments. That's not the same as an individual packet getting fragmented. There's no way to re-assemble the message to discover the original boundaries of the "write" call that you used to emit it. In other words, your data stream needs to have message boundaries in it. TCP, at the application level, is a stream of octets, *not* a sequence of packets. There are numerous examples of this in Twisted; for example, twisted.protocols.basic includes NetstringReceiver, LineReceiver and (various subclasses of) IntNStringReceiver, each of which is a different strategy for breaking up a TCP stream into a discrete sequence of messages. _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

1 0

[Twisted-Python] Packet Fragmentation
by Nima Ghanavatian Oct. 9, 2008

Oct. 9, 2008

Hi, I'm using TCP to pass large files back and fourth. So far I've follow and expanded on the twisted code examples for a simple server/client. Everything works fine for me until the packets get large enough that fragmentation kicks in. >From one side, I'm sending packets like so: class EchoClient(protocol.Protocol): def connectionMade(self): data = "" while len(data) < 10000: data=data+'a' self.transport.write(data) and on the receiving end: def dataReceived(self, data): print data I noticed after multiple iterations the data buffer fills up to the total 10000 bytes, but this takes 3 iterations of dataReceived(). Common sense suggests to have a simple "if len(data) == totalPacketSize" to solve my problem, but I have different sized packets coming in all the time, and can't hardcode this part. I was wondering if there's any way in Twisted/Python to handle packet fragmentation. Can I access the last fragment flag in the TCP packet somehow? Or is there a total packet size sent with the packet automatically? If not, what would be the best way to solve this? thanks, Nima

5 5

[Twisted-Python] alternative to tac
by Vasil Vangelovski Oct. 7, 2008

Oct. 7, 2008

Which is the simplest way to put a service under and application to listen on a port and behave similarly to the way it behaves when run with twistd -noy script.tac but directly from a standard python module? Lets assume I don't have access to the tac command and want to emulate the behaveour.

3 3