Mailman 3 February 2006 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] Problem: spawnProcess() never ending when installSignalHandlers=False
by Matt Feifarek 03 Feb '07

03 Feb '07

Hello. I'm using Twisted 1.3.0 on python2.4 on an Ubuntu-Hoary x86 Linux box. I'm using twisted and spawnProcess() with a Queue structure to chain processes that need to be called (specifically, imagemagick commands). I use the "processEnded" event/method of t.i.p.ProcessProtocol to go from one task to the next. Twisted shares a python process with a multi-threaded web-app-kit (Webware) so it has to be run in a thread. This worked well on other older versions of twisted and older versions of python; also on linux. I've tracked down the problem to be twisted itself (or my incorrect use of it) rather than my code. This example illustrates it: ------------------------------------------------- from twisted.internet import reactor from twisted.internet.protocol import ProcessProtocol class MyProt( ProcessProtocol ): def processEnded(self, reason): print "ended.", reason executable = '/bin/touch' args = ( '/tmp/foo.txt', ) args = (executable,) + tuple(args) reactor.spawnProcess( MyProt(), executable, args ) reactor.callLater( 5, reactor.stop ) #reactor.run() reactor.run(installSignalHandlers=False) ------------------------------------------------- As is, this code never runs the "processEnded()" method of MyProt(). If I change the reactor.run() parameter to leave the signal handlers on, it works. Note that I need to leave the signal handlers turned off because I can't run the reactor in the main thread; Webware needs that. I also know that the subprocess is giving up its file descriptors; I can see that if I put more print statements into MyProt(). And I know that the subprocess is working; /tmp/foo.txt is there. It seems that somehow the reactor is using signals to notice when a process has ended; this makes a certain ammount of sense intuitively, though I don't understand the nitty-gritty of processes and signals at all. And more importantly, this has worked before on other combinations of twisted/python. Is this a bug? If so, has it been fixed? If not, what should I do to detect when a spawned process is finished? Thanks very much. Twisted rocks.

6 14

[Twisted-Python] Service dependencies
by Matt Goodall 10 Jul '06

10 Jul '06

Hi, I have a couple of services. The second is dependent on the first being fully initialised before the second gets set up. I thought that the order of Service startup was typically handled by organising services in a hierarchy, and that the startup of the hierarchy understood deferreds. I was wrong - service.Application is a MultiService and only understands deferreds in stopService. There's a DependentMultiService class in twisted.internet.app but that entire module has been deprecated. Is there a reason why the MultiService does not support deferreds in startService? Is there a reason that DependentMultiService was not moved across to the twisted.application package? In other words, what is the recommended way of starting services in the correct order, including waiting for them to get initialised fully, these days? Thanks. - Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt(a)pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer.

5 4

[Twisted-Python] Status of wxPython support in Twisted 2.1: use wxreactor
by Itamar Shtull-Trauring 01 Jul '06

01 Jul '06

wxreactor in Twisted 2.1 uses threadedselectreactor. This means that wx apps ought to use wxreactor, not TSR directly; there are some issues with vanilla TSR wx integration that are also present in wxreactor, but will be fixed in future versions of wxreactor. What's broken (there may be more): 1. reactor shutdown events aren't handled correctly (e.g. Deferred returned from during-shutdown event). 2. Signal handling for e.g. Ctrl-C is caught by wx, not twisted. In order to get these fixed I need your help. I have a version of wxreactor that *probably* fixes these in Subversion trunk. You can try it out by running doc/core/examples/wxdemo.py. Hitting Ctrl-C, choosing Exit from the menu or closing the menu all should have the same effect - the application logs "shutting down in 0.3 seconds...", "2", "1", "0" then exits. In addition, "Hello world" should continue to be printed while the dialog box is open or you are holding down a menu. Please try this out, and report success or failure at <http://twistedmatrix.com/bugs/issue1235>. I'm especially interested to hear if it works on Windows or Mac OS X. Please include your OS and version of wxPython/wxWidgets when submitting reports.

3 7

[Twisted-Python] Missing feature in detecting stalled PB connections?
by Luc Stepniewski 07 Mar '06

07 Mar '06

While using pb.PBClientFactory() with reactor.connectTCP(), I noticed that the specified timeout parameter is only for testing if there is no connection, not if the remote PB replies correctly. I noticed this when one of my remote PB instance was stuck. It was accepting requests, but did not answer anything at all. So the connection is established, and stays indefinitely! You can verify this by yourself by just putting netcat ("nc -vv -l -p 9999" for example) instead of a PB instance. The PB on the other side will connect, and then, nothing else. No timeout that netcat didn't reply (not even correctly). I know I could add a timer (callLater) that would wake up and kill the connection after some time (because I know one transaction will not exceed some precise time), but it's really really trashy, and one not always knows what will be the duration of the transaction. Is there really no other alternative to detect the correct connection to another PB instance? I attached an example code that would show this behaviour. Thanks, Luc -- Luc Stepniewski <luc.stepniewski(a)adelux.fr> <sip:724766@fwd.pulver.com> Adelux - Securite, Linux Public key: <http://lstep.free.fr/pubkey.txt> Key BC0E3C2A fingerprint = A4FA466C68D27E46B427 07D083ED6340BC0E3C2A

3 5

[Twisted-Python] Twisted & SOAP
by Richard Townsend 01 Mar '06

01 Mar '06

I've been experimenting with the Twisted web/soap.py integration with SOAPpy which appears to do the sort of I stuff I need at the moment. However, a recent message on the Pywebsvcs-talk mailing list said: "Any objection to our diabolical plan to integrate wstools into ZSI and shove SOAPpy to a second class status? We'd like to converge on a single python web services solution, but we're working hard to make sure SOAPpy users can do in ZSI what they can do in SOAPpy..." What effect will this development have on support for SOAP in Twisted? regards Richard

3 2

[Twisted-Python] Sending commands when using Telnet client
by Don Smith 28 Feb '06

28 Feb '06

Yes, I'm sort of a Twisted newbie, so maybe this has an obvious answer, but I'm just not seeing it. I have a Twisted program that I need to add in the ability to make a client telnet connection to a remote server and send some commands to it and deal with the data received. In my main program I have this line of code: reactor.connectTCP("mytelnethost", 24, Connections.TelnetConnection("myname of object",events,"command to run")) My Connections.TelnetConnection class looks like this: from twisted.internet.protocol import Protocol, ReconnectingClientFactory from twisted.conch.telnet import Telnet class TelnetClient(Telnet): def connectionMade(self): print "connection made" self.write("\r\n") def write(self, data): print data Telnet._write(self, data+"\r\n") def dataReceived(self, data): print "received:", data if "User Name:" in data: self.write("user") if "Password:" in data: self.write("password") if ">" in data: time.sleep(2) self.write("connect") if "Connector Name:" in data: time.sleep(2) self.write("another command") class TelnetConnection(Connection, ReconnectingClientFactory): """ Telnets to host:port and executes cmd. cmd """ protocol = TelnetClient def __init__(self, name, eventQueue, cmd=None): Connection.__init__(self, name) def clientConnectionFailed(self, connector, reason): print 'connection failed:', reason.getErrorMessage() def clientConnectionLost(self, connector, reason): print 'connection lost:', reason.getErrorMessage() This all works great, except that the command sequence is hard coded in the TelnetClient class. I want to be able to reference the "cmd" parameter, which could be a list of commands to iterate over,etc. But I don't see how I can access the "cmd" parameter that gets passed into the TelnetConnection class from within the TelnetClient class. I'm really desparate for some help, I've been racking my brain on this since yesterday morning. Thanks, Don

3 5

[Twisted-Python] Re: [Twisted-commits] r16103 - Make loopbackTCP and loopbackUNIX return Deferreds
by Jean-Paul Calderone 28 Feb '06

28 Feb '06

On Tue, 28 Feb 2006 05:37:54 -0700, Jonathan Lange <jml(a)wolfwood.twistedmatrix.com> wrote: >Author: jml >Date: Tue Feb 28 05:37:52 2006 >New Revision: 16103 > >Modified: > branches/deprecated-trial-1509/twisted/mail/test/test_imap.py > branches/deprecated-trial-1509/twisted/mail/test/test_pop3.py > branches/deprecated-trial-1509/twisted/mail/test/test_pop3client.py > branches/deprecated-trial-1509/twisted/mail/test/test_smtp.py > branches/deprecated-trial-1509/twisted/protocols/loopback.py > branches/deprecated-trial-1509/twisted/test/test_loopback.py > branches/deprecated-trial-1509/twisted/test/test_tpfile.py >Log: >Make loopbackTCP and loopbackUNIX return Deferreds > >Modified: branches/deprecated-trial-1509/twisted/protocols/loopback.py >============================================================================== >--- branches/deprecated-trial-1509/twisted/protocols/loopback.py (original) >+++ branches/deprecated-trial-1509/twisted/protocols/loopback.py Tue Feb 28 05:37:52 2006 >@@ -122,14 +123,9 @@ > clientF.noisy = noisy > reactor.connectTCP('127.0.0.1', serverPort.getHost().port, clientF) > >- # this needs to wait until: >- # A: the client has disconnected >- # B: the server has noticed, and its own socket has disconnected >- # C: the listening socket has been shut down >- spinUntil(lambda :clientF.disconnected) # A >- spinUntil(lambda :not server.transport.connected) # B >- serverPort.stopListening() >- spinUntil(lambda :not serverPort.connected) # C >+ d = clientF.deferred >+ d.addCallback(lambda x : defer.maybeDeferred(serverPort.stopListening)) >+ return d > This now has a race, where the Deferred may fire before the server protocol is notified that the connection has been lost. Jean-Paul

2 2

[Twisted-Python] twisted TCP frame size
by Thys Meintjes 27 Feb '06

27 Feb '06

Hi, I've recently written a threaded client server application using a custom message protocol. Each message is 49 bytes long and needs to arrive at the client as soon as possible. In the current app I send() each message as soon as all 49 bytes are available, I also recv(49) bytes at the client side (buffering as necessary). The threaded application thus always reads 49 bytes from the network buffers as soon as it is possible to do so. This application consumes about 9% cpu time at full throttle. Because I already use the twisted.enterprise adbapi on the client side I decided to get rid of all the client threads and use twisted for the TCP stuff as well. I've written a small test script to determine basic loads and performance: ----------------- #! /usr/bin/env python import psyco psyco.full() import array import sys from twisted.internet.protocol import Protocol,ReconnectingClientFactory from twisted.internet import reactor f = open("wakka", "w") class TRAUReceiver(Protocol): def dataReceived(self, data): print len(data) f.write(data) # f.flush() def connectionMade(self): """ * connection were made, send signon""" print "Signing on...", signon = array.array('B', [ord('T'), 1 ,255, 1, 255]) self.transport.write(signon.tostring()) print "Done" class TRAUClientFactory(ReconnectingClientFactory): def startedConnecting(self, connector): print 'Started to connect.' def buildProtocol(self, addr): print 'Connected.' print 'Resetting reconnection delay' self.resetDelay() return TRAUReceiver() def clientConnectionLost(self, connector, reason): print 'Lost connection. Reason:', reason ReconnectingClientFactory.clientConnectionLost(self, connector, reason) def clientConnectionFailed(self, connector, reason): print 'Connection failed. Reason:', reason ReconnectingClientFactory.clientConnectionFailed(self, connector, reason) reactor.connectTCP('localhost', 55555, TRAUClientFactory()) reactor.run() ------------------ Connecting the above to the server yielded somewhat surprising results: The length of the dataReceived() data between runs varies between 49 and multiples of 49 bytes. I understand this (I think) as the ethernet packet length sweet spot is about 1.3kB and Protocol is propably optimized arround that. This does imply that I need a more elaborate frame caching scheme on the client side than the one I currently have, as another socket connection signals if messages from the current connection must be stored or discarded. Surprisingly, the script data length is reported as continuous 49 bytes at leas 4/10 times it's run, on other runs the print len(data) line looks something like this: 49 12691 49 7938 49 Each second dataReceived() callback is 49 bytes, the rest multiples of 49. When the script reports all frames as 49; cpu consumption is ~33%, when staggered it's ~4%. The dataReceived() call seems overly expensive when compared with the 'raw' synchronous recv(). Paradoxically, padding the 49 byte message with 1000 pad bytes improves the script's performance 8 times; due to the decrease in dataReceived() calls probably. So, 1) is there a way to force dataReceived() to return when a certain data length has been received ? 2) Why is dataReceived() so expensive (if it is) ? 3) Is Protocol the correct tree or are there other ways to handle small time sensitive messages in twisted. Apologies for the longish post. Regards Thys

3 2