Mailman 3 July 2012 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 22 Jul '12

22 Jul '12

1 0

[Twisted-Python] i want to update 1000 devices running as servers
by Luka Rahne 20 Jul '12

20 Jul '12

I have like 1000 devices that need update that takes like 1 minute each. I want them to update as quick as possible, sending lines from sx file, using UDP and for each line i got reponse "OK". Here is code that works for up to 500 servers, but not so well when i go more (it just does not finish) my current simplified client and server_emulator code is here: https://gist.github.com/3144857 currently i have only one server for simulation, but in real world it will be 1000 different IP-s. I was trying to stop and run reactor but did not work (reactor is not rerunnable exception or somthing like that). I want to run lets say 100 updates at time and once some update is finished i want to remove this protocol out of job and schedule new one. Can somebody give some pointers? I am running this on windows if that is an issue.

4 6

[Twisted-Python] Effects of backlog parameter on listenTCP
by Tobias Oberstein 19 Jul '12

19 Jul '12

Could someone shortly elaborate on the semantics / effect of the parameter "backlog" with listenTCP? The docs say: "size of the listen queue". Is that a tunable within Twisted? I mean, there are kernel parameters for queue sizes of not yet completely TCP handshaked connections and so on. But those are kernel tunables. What does the "backlog" parameter? The reason I am asking: I am stress testing a Twisted server and I see client connections denied upon high connection rates .. does the parameter help? Thanks! Tobias

4 5

[Twisted-Python] unit testing question: twisted with tk
by Russell E. Owen 18 Jul '12

18 Jul '12

I have a bit of code that combines twisted with Tkinter and now I'd like to write some unittests for it. I can't seem to figure out how to get TwistedTrial to handle this case. I've written a unit test like this: from twisted.trial import unittest from twisted.internet.defer import Deferred import twisted.internet.tksupport root = Tkinter.Tk() twisted.internet.tksupport.install(root) from twisted.internet import reactor class TestTkSocket(unittest.TestCase): def test...(...): .... the one test so far returns a deferred and calls errback on that deferred if the test fails, and callback if it succeeds. When I run the unit test with trial I get this error error: twisted.trial.util.DirtyReactorAggregateError I have an identical unit test for a pure-twisted version of my communication code (no Tk) and it works fine. I am guessing the problem is cleanly getting rid of Tkinter's root at the end of the test and disentangling it from the reactor, but I have no idea how to do that. Any suggestions on how to proceed (e.g. a way to get TwistedTrial to work, or a good alternative)? -- Russell

3 3

[Twisted-Python] A few simple questions
by Russell E. Owen 18 Jul '12

18 Jul '12

I'm adding Twisted support to some existing communication code and I have a few questions which I hope are simple. I've been reading the API documentation, but if the answers are there I'm missing them. What happens if one tries to write to a protocol.transport that is not connected or in an error state? (I'm hoping that this case has defined behavior and that it will raise an exception). Is there some way to query a protocol to see if it is in a good state (connected and no errors)? I realize with callbacks this is not usually necessary, but I have a case where it would be helpful (though I can probably manage without it). When reading raw bytes, is there any way to get the bytes from the protocol? I assume not: that if one wants to buffer the data one must do that in the callback. If so, does Twisted provide a suitable buffer class? -- Russell

6 8

[Twisted-Python] bug or my code? (getPage + Twisted Webserver = Exception)
by Me Myself 17 Jul '12

17 Jul '12

Hi All, I have a flask application that I am running on a https twisted webserver and forwarding the port to a url. Everything works well. I can goto the url using firefox, accept the certificate and get in. I have a small nose test to test if the url is up. Here is the test: @deferred(timeout = 31) @defer.inlineCallbacks def test_if_forwarding_url_is_ready(self): """ furl: Check if url is accessible """ # First wait for 10 seconds yield task.deferLater(reactor, 10, lambda:None) delay = 2 secureurl = Config.General.url # Every two seconds do a query and find out if you can get # data from the mapped url. If you can, then exit for i in range(15): got_data = False print "Trial :", i try: output = yield getPage(secureurl) got_data = True except Exception as E: print "Exception occurred:", E got_data = False if got_data == True: assert "secure magic" in output return yield task.deferLater(reactor, delay, lambda:None) # All 15 trials to get data from the url failed assert False When the test is run, it fails "most of the time" and outputs: Exception occurred: [('SSL routines', 'SSL23_READ', 'ssl handshake failure')] But if I run the test and right away goto the secureurl manually using firefox or chrome, as soon as I am on the webpage, the test passes! Sometimes I've seen the test will pass after 8 trials, if that is when I can type the url in firefox and press enter! Any ideas what is causing this and how to fix it? Maybe I am doing something stupid and perhaps someone can help me rewrite this code. Thanks, --Ram

4 8

[Twisted-Python] Twisted 12.0.0 - Facing Issue in SSL/TLS connection setup
by Guru Prasad 17 Jul '12

17 Jul '12

Hello All, I am new to Twsited and I am facing SSL/TLS connection setup issue with Twisted 12.0.0. Version of Ubuntu (unstable), I am using is: 12.10 During the SSL/TLS connection setup, there is ClientHello request from the client and for this request there is no response from the server where Twisted 12.0.0 is running. I see only TCP FIN and RST sent by the server end for the ClientHello request. When I checked the syslog, I see below messages. Traceback (most recent call last): : File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 84, in callWithLogger : return callWithContext({"system": lp}, func, *args, **kw) : File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 69, in callWithContext : return context.call({ILogContext: newCtx}, func, *args, **kw) : File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext : return self.currentContext().callWithContext(ctx, func, *args, **kw) : File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext : return func(*args,**kw) : File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 599, in _doReadOrWrite : self._disconnectSelectable(selectable, why, inRead) : File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 266, in _disconnectSelectable : selectable.connectionLost(failure.Failure(why)) : File "/usr/local/lib/python2.7/dist-packages/gnutls/interfaces/twisted/__init__.py", line 328, in connectionLost : tcp.Server.connectionLost(self, reason) : File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 277, in connectionLost : protocol.connectionLost(reason) : File "/usr/lib/python2.7/dist-packages/twisted/web2/channel/http.py", line 853, in connectionLost : self.readConnectionLost() : File "/usr/lib/python2.7/dist-packages/twisted/web2/channel/http.py", line 842, in readConnectionLost : self.transport.loseConnection() : File "/usr/local/lib/python2.7/dist-packages/gnutls/interfaces/twisted/__init__.py", line 322, in loseConnection : tcp.Server.loseConnection(self, reason) : TypeError: loseConnection() takes exactly 1 argument (2 given) Anyone else has faced this issue, earlier? I am not having any clue like what went wrong. Have I missed any of the dependent packages required for Twisted 12.0.0? Please help me to come out of this issue. Best Regards, bgurup

2 1

[Twisted-Python] How to best log server/client interactions from a test?
by Dan Milstein 16 Jul '12

16 Jul '12

'lo all, I've been doing some work with the twisted.mail.imap4 module, and its associated unit tests (in order to learn the code more, I'm tackling a few of the outstanding tickets). When running those tests, I'd often like to log all the server/client interaction. I managed to do so by way of hacking the code in twisted.protocols.loopback, but that feels pretty wrong. What's a good way to get all the interaction logged? To be more specific, the tests in question (e.g. test_imap.NewFetchTestCase), do a lot of: d = loopback.loopbackTCP(self.server, self.client, noisy=False) d.addCallback(lambda x : self.assertEqual(self.result, self.expected)) return d What's a Good Way to get all the traffic going over that loopback logged? Setting noisy to True merely gets it to log some protocol startup/teardown-type messages. Or: given that I've figured out how to hack loopback.lookupbackTCP, if there's general utility, I can certainly add a keyword arg to trigger logging of all messages, e.g. logAll=False (I'm trying to find something which feels meaningfully different from 'noisy', because it would seem Very Wrong Indeed to make noisy suddenly produce vastly more log output). Is that something people might be interested in? Thanks, -Dan

4 5

[Twisted-Python] Twisted in a multicore environment
by Augusto Mecking Caringi 15 Jul '12

15 Jul '12

Hi, I'm researching the best way to implement/use a Twisted-based server in a multicore environment... There is the Ampoule project, that I realize is considered the best way to do that. Right? I'm also reading about the internals of Nginx HTTP server. This server utilizes the same reactor pattern of Twisted (epoll based)... "What resulted is a modular, event-driven, asynchronous, single-threaded, non-blocking architecture which became the foundation of nginx code." http://www.aosabook.org/en/nginx.html But to maximize the use of processors in a multicore environment, Nginx do this: "nginx doesn't spawn a process or thread for every connection. Instead, worker processes accept new requests from a shared "listen" socket and execute a highly efficient run-loop inside each worker to process thousands of connections per worker" My question: There is something similar in Twisted? Or do you think that is easy to implement something like that? Thanks! -- Augusto Mecking Caringi

8 9

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 15 Jul '12

15 Jul '12

1 0