Mailman 3 June 2007 - Twisted

Re: [Twisted-Python] Have I installed twisted correctly - part II
by Simon Pickles 08 Jun '07

08 Jun '07

"It looks like your version of ZopeInterface doesn't want to play nicely with your version of Twisted. Try getting the latest release of each." Jean-Paul Well, I just installed python 2.5 and latest twisted... I don't even know what zope is! I downloaded zope but that doesn't look like it will help me...... Any more ideas? Here's my error again: Traceback (most recent call last): File "<string>", line 74, in run_nodebug File "F:\_Dev\twisted server 20070608\twistedServer.py", line 5, in <module> from twisted.internet import reactor File "C:\Python25\lib\site-packages\twisted\__init__.py", line 19, in <module> from zope.interface import Interface File "C:\Python25\lib\site-packages\zope\interface\__init__.py", line 58, in <module> _wire() File "C:\Python25\lib\site-packages\zope\interface\interface.py", line 936, in _wire classImplements(Attribute, IAttribute) File "C:\Python25\lib\site-packages\zope\interface\declarations.py", line 464, in classImplements spec = implementedBy(cls) File "C:\Python25\lib\site-packages\zope\interface\declarations.py", line 330, in implementedByFallback if isinstance(spec, Implements): TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types >>> Si _________________________________________________________________ Txt a lot? Get Messenger FREE on your mobile. https://livemessenger.mobile.uk.msn.com/

3 2

[Twisted-Python] Have I installed twisted correctly - part II
by Simon Pickles 08 Jun '07

08 Jun '07

Sorry, how lazy of me! *** Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32. *** >>> >>> Traceback (most recent call last): File "<string>", line 74, in run_nodebug File "F:\_Dev\twisted server 20070608\twistedServer.py", line 5, in <module> from twisted.internet import reactor File "C:\Python25\lib\site-packages\twisted\__init__.py", line 19, in <module> from zope.interface import Interface File "C:\Python25\lib\site-packages\zope\interface\__init__.py", line 58, in <module> _wire() File "C:\Python25\lib\site-packages\zope\interface\interface.py", line 936, in _wire classImplements(Attribute, IAttribute) File "C:\Python25\lib\site-packages\zope\interface\declarations.py", line 464, in classImplements spec = implementedBy(cls) File "C:\Python25\lib\site-packages\zope\interface\declarations.py", line 330, in implementedByFallback if isinstance(spec, Implements): TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types >>> btw, the line from twisted.internet import reactor is the ONLY line in the module currently! Thanks Simon _________________________________________________________________ Could you be the guest MSN Movies presenter? Click Here to Audition http://www.lightscameraaudition.co.uk

2 1

[Twisted-Python] Taking a look at pending deferToThread actions
by Allen Bierbaum 08 Jun '07

08 Jun '07

I am using deferToThread in my application to put a number of web service queries into the background and then respond to them when they are complete. This is all working very well, but once in a while I seem to overload the web service with queries and I would like to track this down by checking to see how many queries are in flight and how many are still waiting to be processed. One I have put an action into the background using deferToThread, is there a way to check to see how many functions are currently processing and similarly how many functions are waiting to be processed? Thanks, Allen

7 11

[Twisted-Python] Have I installed twisted correctly?
by Simon Pickles 08 Jun '07

08 Jun '07

Hi, I have just been recommended twisted from the python mailing lists. I thought I'd installed it, but I get an error with the first line! from twisted.internet import reactor error: TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types Can an expert help me? Simon http://www.simonpickles.com --- http://www.squirtualreality.com _________________________________________________________________ Play your part in making history - Email Britain! http://www.emailbritain.co.uk/

3 2

[Twisted-Python] where to getPeerCertificate()
by Eli Criffield 07 Jun '07

07 Jun '07

I'd like to check the CN of incoming certs for a xmlrpc server. I can't seen to find where to get the clients cert information. In the protocol.Protocol (in my case the http.HTTPCHannel) in the connectionMade() i put in a print self.transport.getPeerCertificate() but it returns a None. print self.transport.socket.get_peer_certificate() Does the same. I'm sure the client is connecting with a cert because its verifying it fine. I also have no idea where to check on client program to get the cert info of the server I'm connecting to. Eli Criffield --example-- #!/usr/bin/env python from twisted.web import xmlrpc, server, http, resource from twisted.internet import reactor, ssl from twisted.python.log import startLogging from sys import stdout import OpenSSL class Example(xmlrpc.XMLRPC): """An example object to be published.""" def xmlrpc_add(self, a, b): """Return sum of arguments.""" return a + b class myHTTPChannel(http.HTTPChannel): def connectionMade(self): print "connection from %s"%(self.transport.getHost().host,) # HERE i get no cert print self.transport.socket.get_peer_certificate() print self.transport.getPeerCertificate() self.setTimeout(self.timeOut) class mySite(server.Site): protocol = myHTTPChannel def makeSSLContext(myKey,trustedCA): '''Returns an ssl Context Object @param myKey a pem formated key and certifcate with for my current host the other end of this connection must have the cert from the CA that signed this key @param iTrustCA a pem formated certificat from a CA you trust you will only allow connections from clients signed by this CA and you will only allow connections to a server signed by this CA ''' fd = open(myKey,'r') theCert = ssl.PrivateCertificate.loadPEM(fd.read()) fd.close() fd = open(trustedCA,'r') theCA = ssl.Certificate.loadPEM(fd.read()) fd.close() ctx = theCert.options(theCA) # The SSL protocol to use, one of SSLv23_METHOD, SSLv2_METHOD, # SSLv3_METHOD, TLSv1_METHOD. Defaults to TLSv1_METHOD. ctx.method = ssl.SSL.TLSv1_METHOD # If True, verify certificates received from the peer and fail # the handshake if verification fails. Otherwise, allow anonymous # sessions and sessions with certificates which fail validation. ctx.verify = True # Depth in certificate chain down to which to verify. ctx.verifyDepth = 1 # If True, do not allow anonymous sessions. ctx.requireCertification = True # If True, do not re-verify the certificate on session resumption. ctx.verifyOnce = True # If True, generate a new key whenever ephemeral DH parameters are used # to prevent small subgroup attacks. ctx.enableSingleUseKeys = True # If True, set a session ID on each context. This allows a shortened # handshake to be used when a known client reconnects. ctx.enableSessions = True # If True, enable various non-spec protocol fixes for broken # SSL implementations. ctx.fixBrokenPeers = False return ctx if __name__ == '__main__': r = Example() ctx = makeSSLContext('server.pem','cacert.pem') reactor.listenSSL(7080,mySite(r),ctx) startLogging(stdout) reactor.run()

1 0

[Twisted-Python] Deterministic "program counter" for asynchronious events
by Martin Geisler 07 Jun '07

07 Jun '07

Hello, I'm writing a runtime system for doing secure multiparty computation, and here I have used deferreds with great success. All players run the same program (but with different input) and each player communicate with other players as needed to share intermediate results. These results need to be associated with the right operation and for this I use a program counter. This minimal example shows my solution without network traffic: import sys From twisted.internet.defer import Deferred, DeferredList, gatherResults pc = (0,) def inc_pc(pc): pc = pc[:-1] + (pc[-1]+1,) return pc def sub_pc(pc): pc = pc + (1,) return pc def finish_mul(arg, pc): print "finish_mul(%s): pc = %s" % (arg, pc) return arg def mul(a, b): global pc pc = inc_pc(pc) results = gatherResults([a,b]) results.addCallback(finish_mul, sub_pc(pc)) return results x = Deferred() # Declare variables y = Deferred() z = Deferred() w = Deferred() mul(x,y) # Schedule multiplications mul(z,w) x.callback(2) # The value for x arrives over the network z.callback(4) if len(sys.argv) > 1: y.callback(3) # y gets a value, enabling mul(x,y) to proceed. w.callback(5) else: w.callback(5) # z and w are ready and mul(z,w) can proceed. y.callback(3) Here mul() schedules calls to finish_mul(), and it is this call which I would like to have a deterministic labelling of. I have solved it by having a "two-dimensional" program counter: the calls to mul() get labels (1,) and (2,). Their calls to the sub-operation finish_mul() get labels (1,1) and (2,1) respectively. This works, invoking the example with and without a command line argument gives the same labels for finish_mul(). But it is cumbersome to drag this program counter around all the code, and (what is worse) it is easy to forget to increment it and to create sub program counters at the right times. So have anybody done something similar where you wanted to start a bunch of operations which might start sub-operations themselves and where everything needed to be deterministically labelled? -- Martin Geisler --- <mgeisler(a)mgeisler.net> --- http://mgeisler.net Read, write, create Exif data in PHP with PEL: http://pel.sf.net Take control of your webserver with PHP Shell: http://phpshell.sf.net

1 0

[Twisted-Python] Please help me with process spawning
by Alexey Maksimov 07 Jun '07

07 Jun '07

Hi, Please help me with the following: I want to write a simple app that should remotely control long running shell process. I have a C++ program that does some network capturing I need. I want to create a simple web interface that can start and stop this program via shell on linux. I already wrote a web part of this but failed to spawn my process. I think my problem is that I want to use myserver.tac and don't want directly call reactor.run() as shown in documentation. How can I spawn my long-running process without using reactor.run()? Thanks in advance, Alexey.

2 1

[Twisted-Python] Problem with trial using the reactor
by Alex Lang 07 Jun '07

07 Jun '07

Hello, Here is a problem regarding the distributed trial test runner. The slave instances receive data from the master, telling them which tests to run. This requires use of the reactor on the slave-side. Unfortunately, when the testCase does the cleanup of the reactor, it cleans up everything including the trial slave. Any ideas on what should be done about this? Thank you, Alex

4 4

[Twisted-Python] Twisted TCP Server Maximum Simultaneous Connections
by Alvin Delagon 06 Jun '07

06 Jun '07

First of all, I want to send my gratitude to the people who developed Twisted. ^_^ We have this XMPP server that is implemented using twisted (using the epoll reactor). I was tasked to know how many simultaneous connections it can handle. So I wrote an XMPP benchmark tool using twisted that spawns multiple clients. When we used the benchmark tool on two PCs (each spawning 20,000 XMPP clients), the number of successful connect/authorize varies between 37,000~39,000. Adding more clients we get the, "Connection Closed cleanly" error on the XMPP server. Spawning two XMPP servers didn't help to increase the simultaneous connections. Did we hit the ceiling? Or is there anyway we can increase this further? We already configured the ulimit parameters and tinkered with some kernel configurations.

4 4

[Twisted-Python] udp
by Bruce Coram 06 Jun '07

06 Jun '07

I know the answer to this will be soo simple but is it necessary to use makeConnection when using udp. I understood that udp is a connectionless protocol. I have used TCP and factories to pass data between a server and clients and I wanted to use udp to send 'management' data on an ad hoc basis. Here is the code for a very simple client but I'm clearly missing the point. 'from twisted.internet.protocol import DatagramProtocol from twisted.internet import reactor host = '127.0.0.1' class testcomms(DatagramProtocol): def datagramReceived(self, data, (host, port)): print 'Received ', data, 'from ', host def makeconnection(self, transport): self.transport.write('this is a test', (host, 8000)) t = testcomms() t.makeconnect(host, 8000) reactor.listenUDP(8000, testcomms()) reactor.run()' Thanks in anticipation Bruce Coram

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 03 Jun '07

03 Jun '07

Bug summary ______________________ Summary for 2007-05-27 through 2007-06-03 Bugs opened: 9 Bugs closed: 0 Total open bugs: 969 (+9) |== Type Changes |== Priority Changes |== Component Changes |Defect: +7 |Highest: +2 |Conch: +2 |Enhancement: +3 |High: +2 |Core: +4 |Normal: +6 |Mail: +2 |Web: +2 New / Reopened Bugs ______________________ ===== Highest ===== [#2682] twisted.conch.ssh.userauth is poorly tested. (opened by z3p) defect conch http://twistedmatrix.com/trac/ticket/2682 [#2685] factor secure random implementation (opened by antoine) enhancement core http://twistedmatrix.com/trac/ticket/2685 ===== High ===== [#1105] IMAP Client does not seem to handle unsolicited server responses (opened by cablehead) defect mail http://twistedmatrix.com/trac/ticket/1105 ===== Normal ===== [#2680] abdapi ConnectionPool doesn't allow for synchronous close (opened by jml) defect core http://twistedmatrix.com/trac/ticket/2680 [#2681] twisted.web.distrib non-functional (opened by exarkun) defect web http://twistedmatrix.com/trac/ticket/2681 [#2683] twisted/web/http.py has self._header instead of self.__header (opened by jvtm) defect web http://twistedmatrix.com/trac/ticket/2683 [#2684] t.p.t.ThreadPool.dispatch should be deprecated (opened by indigo) enhancement core http://twistedmatrix.com/trac/ticket/2684 [#2686] AMP implementation: elisp (opened by glyph) enhancement core http://twistedmatrix.com/trac/ticket/2686 [#2687] Conch blows up when processes end due to signals (opened by jml) defect conch http://twistedmatrix.com/trac/ticket/2687 Closed Bugs ______________________

1 0

[Twisted-Python] Distributed Trial Test Runner
by Alex Lang 02 Jun '07

02 Jun '07

Hello, For my Summer of Code project, I am implementing a distributed trial test runner (http://code.google.com/soc/psf/appinfo.html?csaid=770D0FCD95DAFF9E). The following is basically what needs to be done to get distributed trial working, given a master machine and slaves: 1. Transfer the relevant code to all slave machines. (with conch) 2. Do test loading at least on the master (possibly on each of the slaves as well) 3. Find a good test-to-unique-id mapping (probably using FQPNs + something else for test_suite-generated tests?) 4. Implement a visitor which will run through the tests on the master and split them among the slaves. 5. Figure out how to get SSH command shells open to the slaves and get individual tests to run 6. Gather results from slaves and give them to a reporter. I think that is all. Any thoughts? Comments? Thanks, Alex

3 2