Mailman 3 October 2010 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Oct. 10, 2010

Oct. 10, 2010

1 0

[Twisted-Python] Conch for file transfer?
by Laurens Van Houtven Oct. 9, 2010

Oct. 9, 2010

Hi, Is there any documentation about using Conch for file transfer anywhere? I'm interested in doing a continuous deployment thing using it. That means two parts: 1) the ability to submit changes: the hard part here is file transfer of the binary diffs 2) the ability to control a subprocess: basically like supervisord but not written with medusa 3) manual intervention would be nice, so manhole I can't find any documentation or examples, but these classes do exist. I've considered giving up and just using HTTPS instead for file transfer and control. Good idea or not? thanks in advance lvh

2 2

[Twisted-Python] recommended starting point for http debugging/logging proxy.
by Gelonida Oct. 8, 2010

Oct. 8, 2010

Hi, Can anybody recommend a good starting point for a twisted http proxy, or comment on the links that I'll mention lateron in this post? I'd like to use a ddebugging it for playing/debugging/filtering Initially I'd just like to log all requests. lateron I'd like to save certain responses on the disk. Please note: Im familiar with python and http / html in general. I'm not experienced with twisted, but thought, that this might be a good example / excuseto learn a little. I found following links and wondered what you could recommend: http://defrang.com/python mentions a HTTP Debugging Proxy --------------------------------------------------------- http://wiki.python.org/moin/Twisted-Examples has following example, but I dont't know where to hook in: I found one twisted example for a web proxy, but it was so simple, that I don''t have any idea of where to hook in. from twisted.web import proxy, http from twisted.internet import reactor from twisted.python import log import sys log.startLogging(sys.stdout) class ProxyFactory(http.HTTPFactory): protocol = proxy.Proxy reactor.listenTCP(8080, ProxyFactory()) reactor.run() ------------------------------------------------------------- http://sujitpal.blogspot.com/2010/03/http-debug-proxy-with-twisted.html seems also interesting

2 1

[Twisted-Python] patch for TLS
by jason-sage＠creativetrax.com Oct. 8, 2010

Oct. 8, 2010

(cc to both twisted and sage-devel lists) Hi all, In the Sage project (http://sagemath.org), we apply a patch to the Twisted version we distribute. I'd like to clean out our custom patches and push them upstream, if possible. So I have a couple of questions (I'm not terribly familiar with Twisted): 1. Is the patch below generally useful? 2. Is there a better way to do this than to patch the file? 3. What do we need to do to incorporate it upstream in Twisted? PATCH (against the Twisted git mirror, as of today): diff --git a/twisted/application/internet.py b/twisted/application/internet.py index 8437568..0397f08 100644 --- a/twisted/application/internet.py +++ b/twisted/application/internet.py @@ -193,7 +193,7 @@ on arguments to the reactor method. } import new -for tran in 'TCP UNIX SSL UDP UNIXDatagram Multicast'.split(): +for tran in 'TCP UNIX SSL TLS UDP UNIXDatagram Multicast'.split(): for side in 'Server Client'.split(): if tran == "Multicast" and side == "Client": continue diff --git a/twisted/application/strports.py b/twisted/application/strports.py index 3654413..2b6ca08 100644 --- a/twisted/application/strports.py +++ b/twisted/application/strports.py @@ -79,9 +79,25 @@ def _parseSSL(factory, port, privateKey="server.pem", certKey=None, return ((int(port), factory, cf), {'interface': interface, 'backlog': int(backlog)}) +def _parseTLS(factory, port, privateKey='server.pem', certKey=None, + sslmethod=None, interface='', backlog=50): + from gnutls.interfaces.twisted import X509Credentials + from gnutls.crypto import X509Certificate, X509PrivateKey + + if certKey is None: + certKey = privateKey + + cert = X509Certificate(open(certKey).read()) + key = X509PrivateKey(open(privateKey).read()) + cf = X509Credentials(cert, key) + + return ((int(port), factory, cf), + {'interface': interface, 'backlog': int(backlog)}) + _funcs = {"tcp": _parseTCP, "unix": _parseUNIX, - "ssl": _parseSSL} + "ssl": _parseSSL, + "tls": _parseTLS} _OP, _STRING = range(2) def _tokenize(description): Thanks for the excellent software! Jason [1] http://sagemath.org

3 2

[Twisted-Python] ConFoo 2011 Call for Speakers
by Yannick Gingras Oct. 7, 2010

Oct. 7, 2010

Greetings Twisters, We, Montréal-Python, are the coordinators of the Python track at ConFoo 2011 and we are very proud to announce our call for speakers. PHP-Québec, Montréal-Python, Montreal.rb, W3Qc, and OWASP Montréal are organizing the first edition of the ConFoo conference, which will be held in Montréal on March 9th through 11th at the Hilton Bonaventure Hotel. With over 500 expected attendees, ConFoo is one of the largest Web development conference in North America. ConFoo is about the Web, but it's also about showcasing the strengths of different technologies. Do you think that Python beats all the other languages out there for Web development? Do you think that Twisted knocks Tornado's socks off? Come and tell us why! Sessions are one hour long and you can present in French or in English, which ever your prefer. Submissions are due for November 26; for more details, visit the ConFoo website: http://confoo.ca/en/call-for-papers By the way, we are perfectly aware that there is a slight clash with PyCon. You should not worry about that since all Python talks will happen before Friday, giving you plenty of time for the commute towards Atlanta. Share the word! -- Yannick Gingras http://montrealpython.org -- lead organizer

1 0

[Twisted-Python] Limiting a task.Cooperator to N work units / sec
by Phil Mayers Oct. 3, 2010

Oct. 3, 2010

Is there an easy way to make a task.Cooperator instance only execute N ticks / sec, summed across all iterators it's driving? So if you add two iterators, they each run at N/2 per sec, 3 at N/3, etc. It seems like this ought to do it: N = <rate> def myScheduler(x): # reschedule N times per second reactor.callLater(1.0/N, x) class myTermination: def __call__(self): # stop immediately after one iterator.next() return True myCoop = task.Cooperator( terminationPredicateFactory=myTermination, scheduler=myScheduler, ) ...but this doesn't seem to work when I try it: def mytask(name, limit): for i in range(limit): print name, time.time(), i yield i def running(): finish_a = myCoop.coiterate(mytask('a', 40)) def start_b(): finish_b = myCoop.coiterate(mytask('b', 10)) reactor.callLater(6.5, start_b) def main(): reactor.callWhenRunning(running) reactor.run() if __name__=='__main__': main() ...shows that, once the "b" iterator is added, each iterator is running 1/sec, rather than the whole cooperator: a 1285925864.05 0 a 1285925865.05 1 a 1285925866.05 2 a 1285925867.05 3 a 1285925868.05 4 a 1285925869.05 5 << running fine 1/sec up until here b 1285925870.05 0 a 1285925870.55 6 << now running 2/sec b 1285925871.05 1 a 1285925871.55 7 b 1285925872.05 2 a 1285925872.55 8 b 1285925873.05 3 I've had a look at the source code, and it looks like the logic should cause what I want to happen, but obviously it's not. Version is Twisted 8.2.0 on python 2.6 (Fedora 12)

2 4

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Oct. 3, 2010

Oct. 3, 2010

1 0

[Twisted-Python] getPage generates a traceback in abstract.py
by Jason J. W. Williams Oct. 2, 2010

Oct. 2, 2010

I'm really pulling my hair out here. The code in question uses getPage: http://gist.github.com/607124 If I remove the postdata argument to getPage no problem (except the postdata is required by server I'm calling). However, no matter what I put in postdata, I get this: http://gist.github.com/607126 Any help is greatly appreciated. -J

3 4

[Twisted-Python] Unable to start twistd service on Ubuntu 10.04 when using pseudo terminal
by Garret Heaton Oct. 1, 2010

Oct. 1, 2010

To me this seems like a very strange problem and I've tried to debug it best I can but I don't know much about many of the components at work. Here's what happened: I've been happily using Capistrano to deploy many twistd services to Ubuntu 9.04 machines. It restarts them via an init.d script based off this template: http://twistedmatrix.com/trac/attachment/ticket/3434/twistdplugin. I'm running a self-installed Twisted 8.2.0 on both systems with some small tweaks to logging. On newer Ubuntu 10.04 machines I've been deploying to the twistd services don't restart properly. There are no errors in the log. The only log lines I'd get were: [-] Log opened. [-] twistd 8.2.0 (/usr/bin/python 2.6.5) starting up. [-] reactor class: twisted.internet.epollreactor.EPollReactor. But they were only failing to restart when run via Capistrano. I realized that the only difference was that Capistrano was using a pseudo terminal when it issued its commands. To recreate the problem without Capistrano I can run: > ssh -t user@host "/etc/init.d/service start" But it works fine when I run: > ssh user@host "/etc/init.d/service start" The 3 log lines I see in the failed case are written by twistd before it forks (clone() syscall) to create the daemon process. The strace dump of twistd when run with and without pseudo terminal were identical. I tried running strace with the -f option to get some info about the child process but it didn't fail in that case since I guess it doesn't actually fork. Running in a pseudo terminal with --nodaemon passed to twistd also works fine. I've tested this with multiple twistd services and am having the same issue with all of them. Any ideas? Did I completely misdiagnose the problem? Has there been some change in this area between Twisted 8.2.0 and 10.x? I'm certainly confused. :) - Garret

2 3

[Twisted-Python] how to write a safe catch-all
by Chris Withers Oct. 1, 2010

Oct. 1, 2010

Hi All, The attached .py file demonstrates a problem I'm having with a little scheduler I need to maintain. The loop() function is supposed to be a "catch all and report" error handler. However, when the async code initiated by doStuff throws the AttributeError, what actually gets logged is: 2010-09-28 14:41:15,706 ERROR : log (14757|7f691ba676e0): Unhandled Error Traceback (most recent call last): File "test_looping.py", line 41, in <module> reactor.run() File "Twisted-8.2.0-py2.5-linux-x86_64.egg/twisted/internet/base.py", line 1128, in run self.mainLoop() File "Twisted-8.2.0-py2.5-linux-x86_64.egg/twisted/internet/base.py", line 1137, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "Twisted-8.2.0-py2.5-linux-x86_64.egg/twisted/internet/base.py", line 757, in runUntilCurrent call.func(*call.args, **call.kw) File "test_looping.py", line 24, in __call__ del self.connector exceptions.AttributeError: Break instance has no attribute 'connector' 2010-09-28 14:41:15,707 ERROR : log (14757|7f691ba676e0): Unhandled scheduled exception Traceback (most recent call last): File "test_looping.py", line 41, in <module> reactor.run() File "Twisted-8.2.0-py2.5-linux-x86_64.egg/twisted/internet/base.py", line 1128, in run self.mainLoop() File "Twisted-8.2.0-py2.5-linux-x86_64.egg/twisted/internet/base.py", line 1137, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "test_looping.py", line 35, in loop yield doStuff() exceptions.GeneratorExit: But, more crucially, the looping call then appears to stop. What can I do to get the exception logged and then everything handled sanely such that the looping call can continue and my lopp function will keep getting called once every second rather than stopping? cheers, Chris

5 33