Mailman 3 November 2013 - Twisted

[Twisted-Python] DirtyReactorAggregateError: Reactor was unclean. - What to do?
by Jonas Brunsgaard 13 Nov '13

13 Nov '13

Dear list I may need som advise to get som testing wokring. I have a server for service a, which creates a client for service b to get som data: class StateHandler(object): implements(State.Iface) def __init__(self): #do stuff self.loaddata() @inlineCallbacks def loaddata(self): try: conn = yield ClientCreator( reactor, TTwisted.ThriftClientProtocol, dsClient, TBinaryProtocol.TBinaryProtocolFactory()).connectTCP("10.70.10.30", 7246) data = yield conn.client.getAllAuthenticationData() #process data conn.transport.loseConnection() I use trail to test this service, code is bolow: class TestStateStore(unittest.TestCase): @defer.inlineCallbacks def setUp(self): self.handler = StateHandler() self.processor = State.Processor(self.handler) self.pfactory = TBinaryProtocol.TBinaryProtocolFactory() self.server = reactor.listenTCP( 0, TTwisted.ThriftServerFactory(self.processor, self.pfactory), interface="127.0.0.1") self.portNo = self.server.getHost().port self.txclient = yield ClientCreator( reactor, TTwisted.ThriftClientProtocol, State.Client, self.pfactory).connectTCP("127.0.0.1", self.portNo) self.client = self.txclient.client @defer.inlineCallbacks def tearDown(self): self.txclient.transport.loseConnection() yield self.server.stopListening() def test_dummy(self): self.assertEquals(True, True) When I run test it fails with the following error message. ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug) <DelayedCall 0x29d99e0 [29.9931809902s] called=0 cancelled=0 Client.failIfNotConnected(TimeoutError('',))> oc.tests.test_state_service.TestStateStore.test_dummy =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. Selectables: <<class 'twisted.internet.tcp.Client'> to ('10.70.10.30', 7246) at 29d2c10> oc.tests.test_state_service.TestStateStore.test_dummy ------------------------------------------------------------------------------- Ran 1 tests in 0.010s To me it seems like Trail shuts down the reactor and throws an error, due to the fact that the reactor is unclean, because the server client is still active, or at least not cleaned up. How do I work around this problem, any help is very appreciated. -- Jonas Brunsgaard

1 0

[Twisted-Python] FreeBSD buildslaves: change proposal
by Tobias Oberstein 12 Nov '13

12 Nov '13

Hi, we (Tavendo) currently operate 2 Twisted/FreeBSD buildslaves: a) FreeBSD 8.2, i386 with 1 builder (CPy 2.7) => part of the list of "officially supported" builders b) FreeBSD 9.1, amd64 with 2 builders (CPy 2.7 and PyPy) => only listed under "unsupported" We would like to change that into the following: c) one FreeBSD amd64 host with 3 builders c1) CPy 2.7 (supported) c2) PyPy 2.2 (supported) c3) PyPy Nightly (unsupported) Here are some reasons: - FreeBSD 8.2 is very old (read, unsupported) now, FreeBSD 10 is around the corner, and 9.2 the current official stable release - i386 is .. getting obscure. Does _anybody_ run FreeBSD i386 anymore? - there are ongoing workarounds to support that thing .. time that could be spent better - we want to run a PyPy slave for the PyPy project on that same host (this allows to produce the PyPy/Nightlies .. on a daily basis .. and immediately run Twisted trials against that) - someone needs to run a PyPy slave anyway .. the current one is broken - the current Twisted 13.2 has run 100% clean on above b) (CPy) : http://buildbot.twistedmatrix.com/builders/freebsd-9.1-amd64-python2.7/buil… So, I'd like to ask Twisted developers if above makes sense, and you would support this way forward. I've been in contact with PyPy developers, and there seems to be clear interest in improving support for both Twisted and FreeBSD (hence the cross-post). In general, Tavendo is committed to contribute time & material making Twisted + CPy|PyPy + FreeBSD a combination fully (even better) supported and actually first-class in terms of features, compatibility and performance. We think < Twisted + PyPy + FreeBSD > in particular has great potential and can provide an extremely powerful stack: "The _complete_ power to serve." .. stealing from FreeBSD's motto here - heh, and sorry the the marketing blurb .. I just couldn't resist;) Thanks! /Tobias

1 0

[Twisted-Python] Determine TLS version from within request handler ? (twisted web)
by Rob Meijer 11 Nov '13

11 Nov '13

I'm using twisted web for an https connection. At first I start with creating and configuring a ssl.DefaultOpenSSLContextFactory derived class that disables SSLv2 and SSLv3. So (basically we're left with TLS1.0 and up) and forces the use of a single strong (non RC4 but BEAST vulnerable) cipher. Than I have a http.HTTPFactory derived class with a 'buildProtocol'that return a http.HTTPChannel derived object, that in turn returns a http.Request derived object. Than reactor.listenSSL is invoked with both the SSL factory and the HTTP factory. Now my problem. I don't really understand how these two are connected once the request handler gets invoked. What I would like to do is that I would want to allow TLS1.0 users to use the server, but only after explicitly agreeing that they understand the risks involved with using an old BEAST vulnerable browser. Is there any way to retrieve info on what TLS version has been negotiated from within the HTTP request handler? I've been walking the 'self' object a bit, and self.transport.getHandle() seems to have some tls stuff like 'get_cipher_list' and 'get_peer_certificate', but the simple TLS version number does not seem to be available. Is the twisted web API simply insuficient to retreive this information from within the request handler, or am I missing something here? T.I.A, Rob

5 6

[Twisted-Python] twisted irc client + packet capturing
by Edmund Wong 11 Nov '13

11 Nov '13

Hi, I've created a script that would log on to an irc server, while capturing packets. I came across the following link: http://dound.com/2009/09/integrating-twisted-with-a-pcap-based-python-packe… But I'm using pycap (http://pycap.sourceforge.net/), but I'm having some difficulties with getting it to work. Here's the code: # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. # twisted imports from twisted.words.protocols import irc from twisted.internet import reactor, protocol from twisted.python import log from twisted.internet.defer import Deferred # system imports import time, sys import pycap.capture def run_pcap(f): p = pycap.capture.capture('eth0') p.filter('src host ! 192.168.1.100 and dst host ! 192.168.1.100 and dst port 25') packet = None print "Listening...\n" while 1: if packet: print "Received packet." reactor.callFromThread(f, packet) else: print "no packet\n" packet = p.next() class LogBot(irc.IRCClient): """A logging IRC bot.""" nickname = "testbot" def packetShow(self, packet): """ booga """ msg = "Port 25 hit | From:[%s] To:[%s]" % (packet[1].source, packet[1].destination) self.msg(self.channel, msg) def connectionMade(self): irc.IRCClient.connectionMade(self) print "Setting up callInThread\n" reactor.callInThread(run_pcap, self.packetShow) print "Finished setting up callInThread\n" def connectionLost(self, reason): irc.IRCClient.connectionLost(self, reason) # callbacks for events def signedOn(self): """Called when bot has succesfully signed on to server.""" print "Signing on to %s.\n" % self.factory.channel # self.join(self.factory.channel) self.sendLine("JOIN %s" % (self.factory.channel,)) def joined(self, channel): """This will get called when the bot joins the channel.""" print "Joining channel %s." % channel def privmsg(self, user, channel, msg): """This will get called when the bot receives a message.""" user = user.split('!', 1)[0] # Check to see if they're sending me a private message if channel == self.nickname: msg = "It isn't nice to whisper! Play nice with the group." self.msg(user, msg) return # Otherwise check to see if it is a message directed at me if msg.startswith(self.nickname + ":"): msg = "%s: I am a log bot" % user self.msg(channel, msg) def action(self, user, channel, msg): """This will get called when the bot sees someone do an action.""" user = user.split('!', 1)[0] # irc callbacks def irc_NICK(self, prefix, params): """Called when an IRC user changes their nickname.""" old_nick = prefix.split('!')[0] new_nick = params[0] # For fun, override the method that determines how a nickname is changed on # collisions. The default method appends an underscore. def alterCollidedNick(self, nickname): """ Generate an altered version of a nickname that caused a collision in an effort to create an unused related name for subsequent registration. """ return nickname + '^' class LogBotFactory(protocol.ClientFactory): """A factory for LogBots. A new protocol instance will be created each time we connect to the server. """ def __init__(self, channel): self.channel = channel def buildProtocol(self, addr): p = LogBot() p.factory = self return p def clientConnectionLost(self, connector, reason): """If we get disconnected, reconnect to server.""" connector.connect() def clientConnectionFailed(self, connector, reason): print "connection failed:", reason reactor.stop() if __name__ == '__main__': # create factory protocol and application f = LogBotFactory("#testx") # connect factory to this host and port reactor.connectTCP("192.168.1.170", 6667, f) # run bot reactor.run() I was told that perhaps I could use twisted.pair, but I have no idea how to use the module. The documentation is lacking and I've only just started programming in Twisted. Any help appreciated. Edmund

6 6

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 10 Nov '13

10 Nov '13

1 0

Re: [Twisted-Python] Bristol (England!) sprint hosted by HybridCluster
by Richard Wall 09 Nov '13

09 Nov '13

On 5 November 2013 22:25, <exarkun(a)twistedmatrix.com> wrote: > I'm pleased to invite you to join Tom Prince (2013 Twisted Fellow), Luke > Marsden (founder of HybridCluster), and the HybridCluster team (including > myself!) for the first Twisted sprint hosted by HybridCluster! Count me in. I'll probably try and do some more work on Twisted Names. -RichardW.

1 0

[Twisted-Python] Twisted 13.2.0 Release Announcement
by HawkOwl 08 Nov '13

08 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 13.2! The highlights of this release are: * Twisted now includes a HostnameEndpoint implementation which uses IPv4 and IPv6 in parallel, speeding up the connection by using whichever connects first (the 'Happy Eyeballs'/RFC 6555 algorithm). (#4859) * Improved support for Cancellable Deferreds by kaizhang, our GSoC student. (#4320, #6532, #6572, #6639) * Improved Twisted.Mail documentation by shira, our Outreach Program for Women intern. (#6649, #6652) * twistd now waits for the application to start successfully before exiting after daemonization. (#823) * SSL server endpoint string descriptions now support the specification of chain certificates. (#6499) * Over 70 closed tickets since 13.1.0. For more information, check the NEWS file (link provided below). You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>) . The NEWS file is also available at <http://twistedmatrix.com/Releases/Twisted/13.2/NEWS.txt>. Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted! Twisted Regards, HawkOwl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) iQEcBAEBAgAGBQJSfTI0AAoJECMItHnTkkoRDzgH/0hbsTLrBVklSjFXINgjNZcE 9IEwFcXsWGElv066Nmt7Nm2CQ0idIJHosx58re8LKK9F/xK3CWlCceJOeS+npNfE y3CnuBRxJYWC+xdy/WTgLV26LkFi2TzqYJT+r6AfS8KUhOJCUJ3H+x8G2SeDlin+ eED64X4ftbs22tSz3RkCkpT++L6inPvVWhwpQhn+GqnS4oBqjbjcYpwJ8+wFfy72 6qe6OcXsoUxM72nbZ1vLg4WE5nbv3It+HnR9PYJTY9VFv8sNeGzb/VEfLpeFJG7f bZImycCh2Flp3OKVzaELmWc+hYDsWT/TuaQYYw6IJYqvFCoTuEQFsdtJuuVBZoQ= =Attd -----END PGP SIGNATURE-----

2 2

Re: [Twisted-Python] Bristol (England!) sprint hosted by HybridCluster
by exarkun＠twistedmatrix.com 08 Nov '13

08 Nov '13

On 04:42 pm, jamesbroadhead(a)gmail.com wrote: >Sounds exciting -- there have been some changes to trial that I've been >thinking about for a little while, so this would be a great >opportunity. > >Would you mind if I distributed this email farther afield? To >python-uk(a)python.org for instance (or are you waiting until you have a >fixed venue?) Hi James, Please feel free to widen the distribution. I'm going to post a follow- up to the original message with information about the venue in just a minute. Hope to see you there! Jean-Paul > >On 5 November 2013 22:25, <exarkun(a)twistedmatrix.com> wrote: >>Greetings all, >> >>I'm pleased to invite you to join Tom Prince (2013 Twisted Fellow), >>Luke >>Marsden (founder of HybridCluster), and the HybridCluster team >>(including >>myself!) for the first Twisted sprint hosted by HybridCluster! >> >>The sprint will take place on December 7th in central Bristol (exact >>location to be determined) from 11AM until no one remains capable of >>higher >>brain function(*). >> >>As usual sprint topics will surely be wide ranging - bring your >>favorite >>pet project, a personal loathing for unresolved bugs, or just find >>someone >>interesting to work with when you show up (personally, I'll probably >>be >>trying to add APIs to support more easily testing Agent- using >>applications). >> >>HybridCluster is a small Bristol-based company building high density >>web >>hosting platform using Twisted for all manner of tasks - from SSH, >>HTTP, >>SMTP, POP3, and IMAP4 to internal cluster coordination and >>asynchronous SQL >>access. >> >>Watch this space for a follow-up post giving details about the venue >>(and >>free food!). In the meantime feel free to contact me or post to the >>list >>with any other questions. >> >>Jean-Paul >> >>(*) Due to excessive hacking. Certainly not because of any kind of >>mind >>control technology deployed by the TSF. >> >>_______________________________________________ >>Twisted-Python mailing list >>Twisted-Python(a)twistedmatrix.com >>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >

1 0

[Twisted-Python] Strange DataLoosing Problem after some time of connection
by Toni Rosemann 07 Nov '13

07 Nov '13

2 2

[Twisted-Python] (newbie!) smtp to message object
by Fabio Sangiovanni 06 Nov '13

06 Nov '13

Hello list, I'm new to twisted and to python in general, so still deep in the learning process. Please forgive any faulty assumptions you'll find in this post :) I'm trying to accomplish to the following task: realize a tool that acts as a proxy between the SMTP domain and the 'objects' domain. That is: a daemon that listens for SMTP connections (relayed sessions from Postfix) and bundles everything that belongs to a single session (envelope sender, envelope recipients, headers, body) into an object (it will probably become a JSON one day, not decided yet). The aim is to allow further processing (es. publishing to a queue) of a self-contained item. My (draft) implementation can be reached at http://pastebin.com/mbRztuid My main concerns are: 1) am I using the right classes? My implementation leverages twisted.mail.smtp, is this correct? 2) I've overridden smtp.SMTP._cbToValidate; I really don't like this very much (I'd like to leave the protocol untouched) but it's the only way I've come up with (after several days of experiments :) ) to have one single payload down the line (in smtp.SMTP.do_DATA), and not one copy of the message for every single recipient. May this have bad consequences? 3) what do you think about the overall approach? could it have been done differently/better? Thanks a lot for your help! Fabio

3 4

Twisted November 2013