Mailman 3 July 2005 - Twisted

[Twisted-Python] spawnProcess and deferreds
by Nathaniel Haggard 20 Jul '05

20 Jul '05

I would like to spawn seperate processes from the methods program. What's the equivalent of spawnProcess for internal functions? Is it twisted threads? Nate

1 0

[Twisted-Python] how to invoke an action in twisted matrix from a process
by Daniel Isenegger 20 Jul '05

20 Jul '05

Hi, a newby question: i try to invoke an action in a twisted matrix program from a process running on my machine: for example: i start a program (test.py), which generates output. test.py should call the twisted matrix program (which is already running and waiting for input) and hand over its output to the twisted matrix program. to do this i examined the examples process.py and quotes.py (on the HOWTO pages of the twisted matrix homepage). quotes.py seems to be an example which is quite close. When i start it on the shell, nothing happens. if i understand right, quotes.py waits for a tcp-connection on port 10999. Is it also possible to read the data from test.py from stdout (like in process.py) without invoking this program in the twisted matrix program Thanks Dani

1 0

[Twisted-Python] protocol.Protocol and transport.write
by Michele Petrazzo 19 Jul '05

19 Jul '05

I have a very simple application that try to open a connection on a port on the local computer and if it has no program listening, it execute a program file, else if it find a answer program, it send a string ( about 50/80 characters ). After the upgrade to 2.0.x, it don't want to work well, because if it has no response, it open the program correctly, but if it has response, it send the message, but the message don't arrive to the listener. After some tries, I see that into my code, I call reactor.stop before, I think, reactor has ended to send my message. I resolve this with a simple code: reactor.callLater(1, reactor.stop). My question is: when I call reactor.stop, it close all, also if there are some communication in progress? Is there another method to close the communication (and in my case the program) when I'm sure that all the works are ended? Thanks, Michele My code: class ConneExist(protocol.Protocol): #The server is listening def connectionMade(self): self.transport.write(file_name_path) out() def out(): # reactor.stop() <- here is a problem reactor.callLater(1, reactor.stop)

2 1

[Twisted-Python] question about twisted conch (SSH) and wxPython
by Astan Chee 19 Jul '05

19 Jul '05

Hi Guys, Im trying to make a psudo SSH client GUI, but it seems that I although I can 'ls' to a remote machine using twisted.conch, I cant `ls -al`. Also I cant seem to to ssh commands that have spaces in them. Although Im using a windows environment to SSH, i dont think that is the problem. Here is a snippet of my code. Obviously I've already called reactor.run() and included appropriate modules and variables. I dont understand why ls works and ls -al doesnt. Also I was wondering if there are any existing command line SSH clients written with twisted similar to putty, but written in python. Thanks! class SimpleTransport(transport.SSHClientTransport): def verifyHostKey(self, hostKey, fingerprint): print 'host key fingerprint: %s' % fingerprint return defer.succeed(1) def connectionSecure(self): self.requestService( SimpleUserAuth(USER, SimpleConnection())) class SimpleUserAuth(userauth.SSHUserAuthClient): def getPassword(self): return defer.succeed(PASSWORD) def getGenericAnswers(self, name, instruction, questions): print 'Name %s' % name print 'Instruction %s' % instruction answers = [] for prompt, echo in questions: if echo: answer = raw_input(prompt) else: answer = getpass.getpass(prompt) answers.append(answer) return defer.succeed(answers) def getPublicKey(self): path = os.path.expanduser('~/.ssh/id_dsa') # this works with rsa too # just change the name here and in getPrivateKey if not os.path.exists(path) or self.lastPublicKey: # the file doesn't exist, or we've tried a public key return return keys.getPublicKeyString(path+'.pub') def getPrivateKey(self): path = os.path.expanduser('~/.ssh/id_dsa') return defer.succeed(keys.getPrivateKeyObject(path)) class SimpleConnection(connection.SSHConnection): def serviceStarted(self): self.openChannel(TrueChannel(2**16, 2**15, self)) self.openChannel(FalseChannel(2**16, 2**15, self)) self.openChannel(LsChannel(2**16, 2**15, self)) class TrueChannel(channel.SSHChannel): name = 'session' # needed for commands def openFailed(self, reason): print 'true failed', reason def channelOpen(self, ignoredData): self.conn.sendRequest(self, 'exec', common.NS('true')) def request_exit_status(self, data): status = struct.unpack('>L', data)[0] #print 'true status was: %s' % status self.loseConnection() class FalseChannel(channel.SSHChannel): name = 'session' def openFailed(self, reason): print 'false failed', reason def channelOpen(self, ignoredData): self.conn.sendRequest(self, 'exec', common.NS('false')) def request_exit_status(self, data): status = struct.unpack('>L', data)[0] #print 'false status was: %s' % status self.loseConnection() class LsChannel(channel.SSHChannel): name = 'session' def openFailed(self, reason): print 'false failed', reason def channelOpen(self, ignoredData): self.conn.sendRequest(self, 'exec', common.NS('ls -al')) self.data = '' def dataReceived(self, data): self.data += data def closed(self): print 'got data from ls: %s' % repr(self.data) self.loseConnection() reactor.stop() class CatChannel(channel.SSHChannel): name = 'session' def openFailed(self, reason): print 'echo failed', reason def channelOpen(self, ignoredData): self.data = '' d = self.conn.sendRequest(self, 'exec', common.NS('cat'), wantReply = 1) d.addCallback(self._cbRequest) def _cbRequest(self, ignored): self.write('hello conch\n') self.conn.sendEOF(self) def dataReceived(self, data): self.data += data def closed(self): print 'got data from cat: %s' % repr(self.data) self.loseConnection()

6 8

[Twisted-Python] High Level Twisted Question [Multiple Protocol Usage]
by J French 19 Jul '05

19 Jul '05

I'm working with Twisted and getting my head around it but I still have some general paradigm questions. When using Twisted to create an application to run under twistd, is it possible to implement more than one protocol? For instance, If I write a server which uses PB to interface with its python client, can I then also check pop3 boxes from the same application / twistd process? Put another way: When incorporating functionality into a server app requiring multiple protocols (PB, SMTP, HTTP, etc.) what is the standard way of effecting this? Sorry for another newbie question and thanks for your time.

3 2

[Twisted-Python] XMPP/Jabber server using Twisted
by Jeremy Redburn 19 Jul '05

19 Jul '05

I am looking into coding up a simple XMPP server in Python to accept simple XML-RPC calls. I was wondering if there was any sort of "Hello World" client and server examples, or if there is active work being done on XMPP support in Twisted? Thanks much, Jeremy

2 1

[Twisted-Python] IOCP and CANCELLED asynchronous I/O operation
by Justin Johnson 18 Jul '05

18 Jul '05

I am wondering if someone would be willing to assist me in debugging a problem I'm experiencing with my current attempt at an implementation of spawnProcess for the IOCP proactor. A patch with my implementation can be found at http://twistedmatrix.com/bugs/issue1008 along with comments and a log file describing my problem. The problem occurs sporadically when running twisted.test.test_process.ProcessTestCase.testProcesstwisted.test.test_process. First I create 3 pipes that will be used to write to the child process's stdin and read its stdout and stderr. Then I initiate a read on all 3 pipes. The reads makes sense for stdout and stderr. The stdin pipe is created in duplex mode and the read attempt on it from the parent is only there so we can detect when the child closes its end of the pipe so the parent can close its end as well. The problem is that occasionally (often, but not consistently) the read operations on stdin (and sometimes stderr) seem to not run at all and FileMon reports their status as CANCELLED when I get to the end of the program. If I run trial run through pdb and step through it, the result of CANCELLED for these 1 or 2 (if stderr fails as well) reads doesn't appear until the very end of the program. The callbacks (ReadInOp.ovDone and ReadErrOp.ovDone don't get called for either of these when the failure occurs, so FileMon's results seem to accurately reflect what is happening. However, from my print statements it is clear that the reads are initiated ( ReadInOp.initiateOp and ReadErrOp.initiateOp). The closest thing I can find to a description of what CANCELLED might mean is at http://www.cs.princeton.edu/~appel/smlnj/basis/posix-error.html which says it indicates that the associated asynchronous operation was cancelled before completion. This sounds correct, but doesn't help illuminate *why* this is happening. Does anyone have any suggestions as to why this is happening? Thanks in advance. Justin

1 0

[Twisted-Python] threadedselectreactor - shutting down the foreign event loop
by Toby Dickenson 18 Jul '05

18 Jul '05

Im looking at moving my PyQt app to threadedselectreactor. Previously I was keeping twisted and Qt in seperate threads. Avoiding the need to cross that thread boundary has been a great simplification, but Im having some problems relating to the shutdown of my foreign event loop. The examples such as http://svn.twistedmatrix.com/cvs/trunk/doc/core/examples/threadedselect/pyg… handle this by: a. Calling reactor.stop() when there is a GUI shutdown request, b. Using reactor.addSystemEventTrigger to shut down the foreign event loop after reactor shutdown. That approach seems untenable for me, for several reasons: 1. Too invasive. I dont want to pollute Qt code with Twisted details. 2. If my app is runing as a COM server then Pythoncom handles the main loop, not Qt. More pollution. 3. The reactor doesnt get shutdown if the foreign event loop shuts down for some other reason. This leads to deadlocks when the threading module tries to join the threads managed by the threadpool module, because the threadpool does know to close those threads. Im having moderate success with the following alternative structure which closes the reactor *after* whatever foreign event loop has exited, but it feels like I am relying on undocumented reactor implementation accidents here: from twisted.internet.threadedselectreactor import install as twisted_install reactor = twisted_install() reactor.interleave(trigger) try: my_event_loop_in_here() finally: # Request that the reactor stops itself. Internally the # reactor uses self.callLater, so shutdown is not complete # when this method returns. reactor.stop() # One call to reactor.iterate is sufficient to complete # All reactor shutdown events. reactor.iterate() # Reactor is now fully shutdown Is there a better approach? Should something like my finally block be included as a method of threadedselectreactor? Is it right for reactor.stop to return before the shutdown is complete? -- Toby Dickenson

2 2

[Twisted-Python] Any tutorial/sample-code for khashmir or DHT?
by Eric Hsu 18 Jul '05

18 Jul '05

Hi all, I'm trying to learn the DHT (distributed hash table) which could be used to build up a decentralize network. I knew khashmir (http://khashmir.sourceforge.net) which has implemented DHT with twisted. However, the project page of khashmir presents no release, no docs, no sample code, no tutorial... but, it has be included into the Bittorrent sources to implement the "trackless" feature. I'm wondering if someone could provide me or point me to some sample-code or tutorial helping me to learn khashmir? TIA :) Best Regards - Eric

1 0

[Twisted-Python] Is it possible to build a .dll for other win32 program/language to invoke?
by Eric Hsu 18 Jul '05

18 Jul '05

Hi all! This question may not be suitable to ask here, but please help me if you can, thanks! I have written a server with twisted. The server contains some classes which are responsible for handling protocol related stuffs (e.g. packing/unpacking messages). We've already has an app which was written in delphi, we want to extend it to be able to communicate with this server. Here comes the question: is it possible to build the protocol related classes from the server to be some kind of .dll, so that the app could invoke them, reuse them to handle the protocol related stuffs? Any infomation would be warmly welcome, thanks in advance! - Eric

5 5