Mailman 3 October 2012 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince 03 Mar '22

03 Mar '22

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika 01 May '19

01 May '19

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html>

8 10

[Twisted-Python] [ANN] txZMQ released
by Andrey Smirnov 21 Jul '14

21 Jul '14

Hi! I would like to announce txZMQ, ZeroMQ bindings for Twisted. txZMQ is based on pyzmq and requires recent ØMQ version (>=2.1). txZMQ uses ØMQ APIs to get file descriptor that is used to signal pending actions from ØMQ library IO thread running in separate thread. txZMQ should be usable with any Twisted reactor. ØMQ is: * The socket library that acts as a concurrency framework. * Carries messages across inproc, IPC, TCP, and multicast. * Connect N-to-N via fanout, pubsub, pipeline, request-reply. * Fast enough for clustered products and supercomputing. * Asynch I/O for scalable multicore message-passing apps. Bridging ØMQ and Twisted makes a nice match: fast simple messaging between Twisted instances solving complex problems :) Links: * PyPi: http://pypi.python.org/pypi/txZMQ/ * Source code: https://github.com/smira/pyzmq * pyzmq: http://pypi.python.org/pypi/pyzmq * ØMQ: http://www.zeromq.org/ -- Andrey Smirnov, Qik Web Team Lead

8 11

[Twisted-Python] regarding xml elements
by Raj kumar 02 Oct '13

02 Oct '13

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml file as a child of an existing element? Thanks in advance.. Raj.. Forgot the famous last words? Access your message archive online at http://in.messenger.yahoo.com/webmessengerpromo.php

9 14

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] GSoC 2012 Project, TwistedChecker
by Raphael Shu 29 Nov '12

29 Nov '12

Hi everyone, I'm a student of GSoC 2012, from University of Tsukuba, Japan. I'm glad to be selected as a practicer in Twisted project this year. Then in these serveral months, my project is to create a code standard checker for Twisted, Basically, I will follow "Twisted Coding Standard", it can be found at: http://twistedmatrix.com/trac/browser/trunk/doc/core/development/policy/cod… I hope finally this project could be integrated into our main ticket system, so it would automatically check patches or other things. Please let me know if you have any good advise or idea to this project. My name on IRC is zomux. ( This project is currently hosted on launchpad, http://launchpad.net/twistedchecker ) Thanks. Best Regards, -- *Raphael Shu , UACA 2012*

5 7

[Twisted-Python] Create client in Server factory
by Grégoire Leroy 06 Nov '12

06 Nov '12

Hi, I have a server who receive data from clients A/B/C (remote machines). I want to use a client D (on the same machine than the server) to send this data to another server (remote). The difficulty is, I want to use the same client D connection for any client. I don't want open an new connection each time. First thing I would think is to create the connection in my server factory, and use client's methods in my server protocol, for example with [...] class LocalProxyFactory(Factory): def __init__(self): f = LocalProxyClientFactory() reactor.connectTCP("retenodus.net", 4242, f) reactor.run() [...] But in this case, code execution will stay in the reactor, right ? I have read that : http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinput… But I don't know how it can be applied to my situation. Thanks, Grégoire Leroy

3 5

[Twisted-Python] deferToThread and reactor loop
by Tobias Oberstein 02 Nov '12

02 Nov '12

Hi, I've got a question regarding deferToThread / runWithConnection I have a network server that accepts RPCs and forwards those to a relational database calling stored procedures. The call of the stored procedures is happening via "runWithConnection" .. that is on a background thread pool (since that DBs driver is blocking). Now, when there is a large number of RPCs coming in roughly at once, the latency of each call forwarded to the RDBMs increased into the 10's of ms. I have created a self-contained test program without DB etc that I think reproduces the "issue": https://github.com/oberstet/scratchbox/blob/master/python/twisted/threadpoo… ************** Am I correct that "deferToThread" does not immediately forward the call to a background thread, but only the next time the reactor loop runs? ************** At least that is what I conclude from the test program's option https://github.com/oberstet/scratchbox/blob/master/python/twisted/threadpoo… which produces totally different timing results: $ python test.py 800 no 20 2012-10-29 18:41:11+0100 [-] Log opened. 2012-10-29 18:41:11+0100 [-] Using Twisted reactor class <class 'twisted.internet.iocpreactor.reactor.IOCPReactor'> 2012-10-29 18:41:11+0100 [-] Thread pool size suggested: 20 2012-10-29 18:41:11+0100 [-] Worker count: 800 2012-10-29 18:41:11+0100 [-] release to reactor between deferToThread: False 2012-10-29 18:41:11+0100 [-] total run-time: 121 ms 2012-10-29 18:41:11+0100 [-] avg. thread switch IN time: 61177 us 2012-10-29 18:41:11+0100 [-] max. thread switch IN time: 79436 us 2012-10-29 18:41:11+0100 [-] avg. thread switch OUT time: 16021 us 2012-10-29 18:41:11+0100 [-] max. thread switch OUT time: 23150 us 2012-10-29 18:41:11+0100 [-] Main loop terminated. $ python test.py 800 yes 20 2012-10-29 18:41:14+0100 [-] Log opened. 2012-10-29 18:41:14+0100 [-] Using Twisted reactor class <class 'twisted.internet.iocpreactor.reactor.IOCPReactor'> 2012-10-29 18:41:14+0100 [-] Thread pool size suggested: 20 2012-10-29 18:41:14+0100 [-] Worker count: 800 2012-10-29 18:41:14+0100 [-] release to reactor between deferToThread: True 2012-10-29 18:41:14+0100 [-] total run-time: 201 ms 2012-10-29 18:41:14+0100 [-] avg. thread switch IN time: 127 us 2012-10-29 18:41:14+0100 [-] max. thread switch IN time: 5131 us 2012-10-29 18:41:14+0100 [-] avg. thread switch OUT time: 166 us 2012-10-29 18:41:14+0100 [-] max. thread switch OUT time: 7886 us 2012-10-29 18:41:15+0100 [-] Main loop terminated. === Could someone pls enlighten me rgd the "issue"? What's the point, and how to interpret the timings? Any hints greatly appreciated! Thanks, Tobias Above timings are on Windows. Here other platforms: Results on FreeBSD ================ [oberstet@www ~/scm/scratchbox/python/twisted/threadpoollatency]$ python test.py 800 no 20 2012-10-29 17:53:37+0000 [-] Log opened. 2012-10-29 17:53:37+0000 [-] Using Twisted reactor class <class 'twisted.internet.kqreactor.KQueueReactor'> 2012-10-29 17:53:37+0000 [-] Thread pool size suggested: 20 2012-10-29 17:53:37+0000 [-] Worker count: 800 2012-10-29 17:53:37+0000 [-] release to reactor between deferToThread: False 2012-10-29 17:53:37+0000 [-] total run-time: 469 ms 2012-10-29 17:53:37+0000 [-] avg. thread switch IN time: 244311 us 2012-10-29 17:53:37+0000 [-] max. thread switch IN time: 300846 us 2012-10-29 17:53:37+0000 [-] avg. thread switch OUT time: 2347 us 2012-10-29 17:53:37+0000 [-] max. thread switch OUT time: 25703 us 2012-10-29 17:53:37+0000 [-] Main loop terminated. [oberstet@www ~/scm/scratchbox/python/twisted/threadpoollatency]$ python test.py 800 yes 20 2012-10-29 17:53:44+0000 [-] Log opened. 2012-10-29 17:53:44+0000 [-] Using Twisted reactor class <class 'twisted.internet.kqreactor.KQueueReactor'> 2012-10-29 17:53:44+0000 [-] Thread pool size suggested: 20 2012-10-29 17:53:44+0000 [-] Worker count: 800 2012-10-29 17:53:45+0000 [-] release to reactor between deferToThread: True 2012-10-29 17:53:45+0000 [-] total run-time: 707 ms 2012-10-29 17:53:45+0000 [-] avg. thread switch IN time: 410 us 2012-10-29 17:53:45+0000 [-] max. thread switch IN time: 6184 us 2012-10-29 17:53:45+0000 [-] avg. thread switch OUT time: 310 us 2012-10-29 17:53:45+0000 [-] max. thread switch OUT time: 1331 us 2012-10-29 17:53:45+0000 [-] Main loop terminated. Results on Linux ============= [webmq@webmqpgcentos threadpoollatency]$ python test.py 800 no 20 2012-10-29 18:56:13+0100 [-] Log opened. 2012-10-29 18:56:13+0100 [-] Using Twisted reactor class <class 'twisted.internet.epollreactor.EPollReactor'> 2012-10-29 18:56:13+0100 [-] Thread pool size suggested: 20 2012-10-29 18:56:13+0100 [-] Worker count: 800 2012-10-29 18:56:13+0100 [-] release to reactor between deferToThread: False 2012-10-29 18:56:13+0100 [-] total run-time: 178 ms 2012-10-29 18:56:13+0100 [-] avg. thread switch IN time: 77332 us 2012-10-29 18:56:13+0100 [-] max. thread switch IN time: 123225 us 2012-10-29 18:56:13+0100 [-] avg. thread switch OUT time: 22048 us 2012-10-29 18:56:13+0100 [-] max. thread switch OUT time: 43779 us 2012-10-29 18:56:13+0100 [-] Main loop terminated. [webmq@webmqpgcentos threadpoollatency]$ python test.py 800 yes 20 2012-10-29 18:56:14+0100 [-] Log opened. 2012-10-29 18:56:14+0100 [-] Using Twisted reactor class <class 'twisted.internet.epollreactor.EPollReactor'> 2012-10-29 18:56:14+0100 [-] Thread pool size suggested: 20 2012-10-29 18:56:14+0100 [-] Worker count: 800 2012-10-29 18:56:15+0100 [-] release to reactor between deferToThread: True 2012-10-29 18:56:15+0100 [-] total run-time: 370 ms 2012-10-29 18:56:15+0100 [-] avg. thread switch IN time: 694 us 2012-10-29 18:56:15+0100 [-] max. thread switch IN time: 3204 us 2012-10-29 18:56:15+0100 [-] avg. thread switch OUT time: 483 us 2012-10-29 18:56:15+0100 [-] max. thread switch OUT time: 7991 us 2012-10-29 18:56:15+0100 [-] Main loop terminated.

2 2

[Twisted-Python] Transplating a request's transport (SockJS)
by Laurens Van Houtven 30 Oct '12

30 Oct '12

Hi, I've been trying to play with SockJS + Twisted. There's an implementation here: https://github.com/Fugiman/sockjs-twisted However, I'm already running a web server, and as the README says sockjs-twisted is *not* intended to be run together with a web server. It doesn't really use any of the mechanics in twisted.web. So, I tried to make it work using Resources anyway: reconstructing the equivalent bytes in render, and replaying them with the SockJSFactory: class SockJSResource(resource.Resource): """ A resource that defers to a SockJS factory. """ isLeaf = True def __init__(self, factory, options=None): self._factory = SockJSFactory(factory, options) def render(self, request): transport, request.transport = request.transport, None protocol = self._factory.buildProtocol(transport.getPeer()) protocol.makeConnection(transport) path = "/".join([""] + request.postpath) lines = ["{0} {1} HTTP/1.1".format(request.method, path)] for name, values in request.requestHeaders.getAllRawHeaders(): lines.append("{0}: {1}".format(name, ",".join(values))) lines += ["", request.content.read()] data = "\r\n".join(lines) protocol.dataReceived(data) return server.NOT_DONE_YET This only kind of works. There's an external sockjs test suite (more like an acceptance test suite): https://github.com/sockjs/sockjs-protocol, that has multiple failing tests. If anyone needs help running these, I'll gladly assist there. In an attempt to begin debugging this, I've found that basically none of the protocol methods get called on that protocol instance I make on the third line of render. This took me a while to figure out because dataReceived *was* being called -- except then I realized I'm calling it myself in that render method :) This leads me to believe I'm essentially just screwing up transplanting this transport entirely. Is that the case? -- cheers lvh

4 7

[Twisted-Python] a question about monotonic clock
by gelin yan 30 Oct '12

30 Oct '12

Hi All I am quite concerned the ticket 2424 due to our system use reactor.callLater almost anywhere. I know this issue is hard to be finxed otherwise It wouldn't exist for such a long time. So any work around before we getting fix it? The only way I can do is to disable NTP or stop process, sync the time manually and then restart the process. The latter way is hard to be accepted due to we have many machines. Regards gelin yan

7 17

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 27 Oct '12

27 Oct '12

1 0

[Twisted-Python] Avoiding network when testing Perspective Broker
by Maxim Lacrima 26 Oct '12

26 Oct '12

Hi! I am learning to develop TDD way. I want to create a server that understands PB protocol. Initially I thought it would be a good idea to avoid real network connections in my tests, so I tried to use `proto_helpers.StringTransport`: ---------- import cStringIO from twisted.spread import pb from twisted.trial import unittest from twisted.test import proto_helpers class Document(pb.Root): def remote_convert(self, props): self.props = props class DocTestCase(unittest.TestCase): def setUp(self): # set up server self.doc = Document() factory = pb.PBServerFactory(self.doc) self.broker = factory.buildProtocol(('127.0.0.1', 0)) tr = proto_helpers.StringTransport() self.broker.makeConnection(tr) # this is what a client sends self.props = {'name': 'MyDoc', 'path': '/path/'} # prepare data serialized_props = self.broker.serialize(self.props) msg = ('message', 1, 'root', 'convert', 1, ['tuple', serialized_props], ['dictionary']) io = cStringIO.StringIO() self.broker._encode(msg, io.write) self.chunk = io.getvalue() def test_convert(self): # data arrived self.broker.dataReceived(self.chunk) self.assertEqual(self.props, self.doc.props) ---------- However, `Document.remote_convert` is never executed so the test above fails. After debugging I discovered that `Broker._encode` produces different results depending on whether `self.broker.makeConnection(tr)` is called or not. And I created a test case that shows a difference. Here `test_convert1` succeeds while `test_convert2` fails: ---------- class DocTestCase(unittest.TestCase): def setUp(self): self.doc = Document() factory = pb.PBServerFactory(self.doc) self.broker = factory.buildProtocol(('127.0.0.1', 0)) self.props = {'name': 'MyDoc', 'path': '/path/'} serialized_props = self.broker.serialize(self.props) self.msg = ('message', 1, 'root', 'convert', 1, ['tuple', serialized_props], ['dictionary']) def test_convert1(self): self.broker.currentDialect = 'pb' self.broker.setPrefixLimit(64) self.broker.transport = proto_helpers.StringTransport() io = cStringIO.StringIO() self.broker._encode(self.msg, io.write) self.broker.dataReceived(io.getvalue()) self.assertEqual(self.props, self.doc.props) def test_convert2(self): self.tr = proto_helpers.StringTransport() self.broker.makeConnection(self.tr) io = cStringIO.StringIO() self.broker._encode(self.msg, io.write) self.broker.dataReceived(io.getvalue()) self.assertEqual(self.props, self.doc.props) ---------- I wonder what causes this behavior and, in general, if `StringTransport` is suitable for testing PB protocol. Thanks in advance. For your convenience I attached files with these test cases. -- with regards, Maxim

4 5

[Twisted-Python] Is it a typo in abstract.py ?
by gelin yan 26 Oct '12

26 Oct '12

Hi All I just noticed abstract.py from internet/iocpreactor/ has a possible typo: look at the method* *loseConnection * * if self.connected and not self.disconnecting: if self._writeDisconnected: # doWrite won't trigger the connection close anymore self.stopReading() self.stopWriting self.connectionLost(_connDone) else: self.stopReading() self.startWriting() self.disconnecting = 1 in the first part I found self.stopWriting Is it a typo for self.stopWriting()? Regards gelin yan

2 1

[Twisted-Python] Silly txoauth with Twisted Client Question
by Andrew Francis 25 Oct '12

25 Oct '12

Hi Folks: I am implementing a client that uses Twisted. I have to use oauth 2.0. I have started to read up on Oauth and Twisted cred. I noticed that there is a txoauth. For now, my client is of the "it can't keep secrets" nature as described by Google. From my understanding Twisted Cred is used for implementing servers. What is used for clients? Or do I roll my own headers and implement the handshakes myself? Cheers, Andrew P.S - I just started using Agent and CookieAgent (as opposed to getPage). Wow!

4 4

[Twisted-Python] a possible solution for ticket 5562
by gelin yan 25 Oct '12

25 Oct '12

Hi All A few months ago, I reported a bug about IOCP. Last night I spent several hours on its implementation and finally found out a possible solution for that. when sending some small chunks data continuously, the buffer will pile them up and send to IOCP; however there is a SEND_LIMIT (128K) that means only 128K will be handled. Now the problem is when we try to trigger the next writing, IOCP will raise ERROR_IO_PENDING immediately and then connection Lost. So I got a idea: if the size of data is larger than SEND_LIMIT, we can wait a little bit time for the next writing instead of do it immediately. in twisted\internet\iocpreactor\abstract.py there is a method def _cbWrite(self, rc, bytes, evt): if self._handleWrite(rc, bytes, evt): self.doWrite() now I modified a bit, def _cbWrite(self, rc, bytes, evt): if self._handleWrite(rc, bytes, evt): if len(evt.buff) < self.SEND_LIMIT: self.doWrite() else: self.reactor.callLater(0.8,self.doWrite) 0.8 is a silly trial but I have no idea what is the right number for this place. After this modification, previous problematic scripts can pass. Maybe the better solution is to find a way to poll the complete port status when read/write will be recovered from IO PENDING. Simply wait a little is risky. Regards gelin yan

3 8

[Twisted-Python] Running twistd and tac files as Windows service
by John Aherne 24 Oct '12

24 Oct '12

I have been looking at running some twisted programs as a windows service. So in the end I took a copy of twistd.py and adapted it to load my tac file and run it as a windows service. Of course the windows service will run it in a separate thread. I now have it running on XP, 2003 Server Twisted 10 and 12 and it all seemed fine. However when I install it on Windows 7 I get an error 2012-10-23 07:54:49+0100 [-] Unhandled Error Traceback (most recent call last): File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 413, in fireEvent DeferredList(beforeResults).addCallback(self._continueFiring) File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 297, in addCallback callbackKeywords=kw) File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 286, in addCallbacks self._runCallbacks() File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 542, in _runCallbacks current.result = callback(current.result, *args, **kw) --- <exception caught here> --- File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 426, in _continueFiring callable(*args, **kwargs) File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1153, in _reallyStartRunning self._handleSignals() File "C:\Python27\lib\site-packages\twisted\internet\posixbase.py", line 277, in _handleSignals _SignalReactorMixin._handleSignals(self) File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1118, in _handleSignals signal.signal(signal.SIGINT, self.sigInt) exceptions.ValueError: signal only works in main thread Looking around it would seem I need to tell the reactor 'installsignalhandlers=0'. I am not sure if I should do this with windows or not. Has anyone got twistd.py to run as a service on windows because I would be glad to know what other people have done to solve the problem. Thanks for any info. *John Aherne* * * * * * * *www.rocs.co.uk * 020 7223 7567

6 10

[Twisted-Python] Running twisted test suite produces errors and hangs
by Maxim Lacrima 24 Oct '12

24 Oct '12

Hi! I tried to run twisted test suite. And there are some errors: http://pastebin.com/em3NTN2B Also tests in "twisted.test.test_udp" took too much time to run (trial hanged for a couple of minutes on these tests). I wonder if this is something wrong with my Twisted installation. Thank you. -- with regards, Maxim

3 6

[Twisted-Python] git repo maintenance
by Duncan McGreggor 22 Oct '12

22 Oct '12

Hey all, At the sprint today, Glyph mentioned that the github repo hasn't been updated recently and needs someone to maintain it regularly. He also mentioned that pull requests would be accepted from github, at which point I immediately volunteered to keep the repo up to date :-) (I can't bear using svn anymore...) I'll be putting things in place (infrastructure, scripts, etc.) to assist me with this, so let me know if you have any concerns, questions, ideas, etc. Thanks, d

10 29

Re: [Twisted-Python] Adding mock as a test suite dependency
by Julian Berman 22 Oct '12

22 Oct '12

From: exarkun(a)twistedmatrix.com > Subject: Re: [Twisted-Python] Adding mock as a test suite dependency > > On 03:00 am, glyph(a)twistedmatrix.com wrote: > >Also I don't particularly like the testing style associated with Mock. > >I think it might discourage us yet further from writing verified fakes, > >i.e. supported in-memory implementations of things like IReactorTCP, > >that have somewhat intricate behavior that's tedious to emulate with > >Mock. > > I'm also not a huge fan of the *unverified* mock style of testing. I > don't think anything says that mocks *have* to be unverified, though it > seems they're often used that way. > > The mock library that got added to the stdlib has the notion of > constructing a mock using another object as a template. I haven't used > this feature, but it seems like the intent is to at least take a step > towards verification. It'd be nice if someone who knows more about the > features of this library could give some examples. > > In case anyone isn't clear, the problem with unverified fakes is that > they either start out incompatible with the objects they're fakes of, or > else they become incompatible with them over time. Once they're > incompatible, the tests that use them become significantly less useful, > since they demonstrate little or nothing about what will happen when you > try to use the code for real. > > Verified fakes solve this problem by adding assertions that objects and > their fakes have the necessary overlap in either interface or > functionality in order for the tests using them to be valid. > > Beyond that, considering the particular example presented, I wouldn't > actually use mocks to test this. The real object, the debugger, should > be perfectly usable in unit tests. It doesn't allocate or depend on > expensive resources, it doesn't do network I/O, etc. Mocks are perhaps > an attractive nuisance that distract from coming up with a better test. > > Well it seems I sidestepped behavior vs. state but fell into isolationist vs integration :). The reason I think even this was a reasonable example is because in the code that this will test, there is no dependence on an actual debugger whatsoever. What this test should be testing is that an object whose interface is irrelevant for the purpose of the test was handed off to another method in the case that that is expected. In short, the isolationist view as I understand and have come to appreciate says that mocks (in a broader sense here since these aren't true mocks I guess) aren't just for cases where the real object is expensive or annoying to create – they also remove irrelevant details from the body of the test. That being said though, I'm still looking (read: I have not yet looked but will do so when I get home) for actual examples in the test suite I can point to and say that real, actual mocks would have helped if that's what you'd be looking for. To go back to your first point about verification, mock has a bunch of things there of which if I'm truthful I only use some of them with any regularity. The thing that sounds like what you're referring to is likely the `spec` argument, which will do something like: >>> import mock >>> from twisted.trial.itrial import ITestCase >>> testCase = mock.Mock(spec=ITestCase.names()) >>> testCase.run <Mock name='mock.run' id='4319309392'> >>> testCase.jump Traceback (most recent call last): File "<input>", line 1, in <module> File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/mock.py", line 647, in __getattr__ raise AttributeError("Mock object has no attribute %r" % name) AttributeError: Mock object has no attribute 'jump' As someone mentioned you also can hand it an implementor of ITestCase too but it will then pull off all attributes that that object has. There's plenty more granularity, but like anything else the design of the test requires common sense. If the test is meant to test how a specific method on the testCase mock is being used then assertions on how it was called generally seem most natural. As an alternate approach, I've learned (from a few places I think) that in cases where I want to verify that the places I've mocked are being used in a way that isn't going to differ from their actual use in the code, that a much wider scoped test that does integrate but doesn't need to be a unit test can be helpful, such that there are a whole bunch of isolated tests for each piece of logic and then one integrated test that actually does push the real object through such that anyone who does change the way that the two objects interacts still will have something telling them that the tests need updating as they would if all of the tests integrated. Jean-Paul > >Personally I'm -0. Don't let that stop you from cooking up a patch > >that would include it though, I might be in the minority here. > > > >-glyph >

1 0

[Twisted-Python] Adding mock as a test suite dependency
by Julian Berman 22 Oct '12

22 Oct '12

Hi. I'd like to gauge any interest in (or resistance towards) adding mock (http://www.voidspace.org.uk/python/mock/mock.html) as a dependency for the test suite. In short, while I don't necessarily feel strongly about behavior vs. state based verification being preferred in a general sense, I think both have their place when used with sensibility. (Sorry, I know that's not saying too much of substance, but I'm trying to ward off a discussion on whether or not it has a place at all if possible). The *benefit* though for me in having mock present is that it decreases the lines of code necessary to write stubs and mocks. While doing so is not really that difficult anyhow, it *is* just a bit more clutter to do so without mock, and the extra 3 or 4 lines mean that in more than one instance I have found myself pick a different strategy than I would have because of the extra lines of code that clutter the test method. To give a specific example, as I write this I'm writing tests for #6143 (which will hopefully let trial drop into a debugger on an error or failure rather than at the outset of a test run). To do so I want to write tests that look like: def test_itDebugsOnError(self): result = mock.Mock() exc_type, exc_value, tb = mock.Mock(), mock.Mock(), mock.Mock() decorated = DebugDecorator(result) with mock.patch.object(decorated, "debug") as debug: decorated.addError(exc_type, exc_value, tb) debug.assert_called_once_with(tb) result.addError.assert_called_once_with(exc_type, exc_value, tb) where I want to test that the decorated test result is being used as I expect it to, and that debug is being called. Another test will test that debug does what it's supposed to, also in isolation. Of course none of the objects I used there were very interesting, but mock has similarly terse syntax for creating mocks with attributes, return values, etc. amongst a bunch of other useful features which I'll be happy to elaborate on if anyone would like, I just had the above example at hand. To add a few more notes: - mock is in the stdlib starting with 3.3 as unittest.mock, which as I understand it is the first version of Py3 for which support is planned. So really all that'd be needed would be to add mock as a dependency on 2.X. - it's a single file with no external deps, so installation is not difficult, and it'd only be a dep for running the tests (and presumably only a subset thereof that chose to use it) - there are other mocking libraries, but I don't like any of the others as much personally for various reasons. - I'd be willing to do the small amount of work to add it if there'd be interest :) Cheers, Julian

8 9

[Twisted-Python] Managing twisted application as init service.
by Maxim Lacrima 22 Oct '12

22 Oct '12

Hi, I have been developing an application, which I would like to manage in a fashion similar to scripts placed in /etc/init.d. I.e. I want to start, stop, restart and monitor the application. Currently, I am trying to figure out a correct approach for this. Initially I thought to create a wrapper shell script like here [1] and just copy it into `/etc/init.d` directory. Then in that script I would make `twistd` start the application. However, today I discovered `twisted.runner` package, which seems to do the same thing. So my question is how to run and monitor processes using `twisted.runner`? I couldn't find any examples on how to use it. Also, in 11th part of the tutorial [2] there is a mention that I can package my application as rpm package, which, when installed, will correctly register my package as `init.d` script. So are there best practices in twisted about how to install twisted applications and manage them as `init.d` services? I am quite new to Twisted (and to Linux `init.d` scripts). Sorry, if my question is unclear. Thank you in advance. [1] http://www.sensi.org/~alec/unix/redhat/sysvinit.html [2] http://twistedmatrix.com/documents/current/core/howto/tutorial/configuratio… -- with regards, Maxim

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 20 Oct '12

20 Oct '12

1 0

[Twisted-Python] getpeername from verify callback
by Nathan Mower 19 Oct '12

19 Oct '12

The following sample code worked until Twisted began to prefer memory BIOs over socket BIOs. Now it produces this error... exceptions.AttributeError: 'NoneType' object has no attribute 'getpeername' ...on line 9 where getpeername() is called by the verify() callback. Is there any way to obtain the peer name, given the OpenSSL.SSL.Connection object passed into verify()? Anything that surfaces the underlying socket? (Perhaps something similar to what is done in connectionMade(), which does work.) Or alternatively, is there a way to tell the reactor to employ socket BIOs? Thanks, Nathan ---------------------------------------------- from OpenSSL import SSL from twisted.internet import reactor, ssl from twisted.internet.protocol import ClientFactory from twisted.protocols.basic import LineReceiver class VerifyContextFactory(ssl.ClientContextFactory): def verify(self, connection, x509, errnum, errdepth, ok): print connection.getpeername()[0] return ok def getContext(self): ctx = ssl.ClientContextFactory.getContext(self) ctx.set_verify(SSL.VERIFY_PEER|SSL.VERIFY_FAIL_IF_NO_PEER_CERT, self.verify) return ctx class MyClient(LineReceiver): def connectionMade(self): print "connected to", self.transport.socket.getpeername()[0] return def connectionFailed(self, reason): reactor.stop() def connectionLost(self, reason): reactor.stop() class MyClientFactory(ClientFactory): protocol = MyClient if __name__ == "__main__": reactor.connectSSL('www.example.com', 443, MyClientFactory(), VerifyContextFactory()) reactor.run()

3 7

[Twisted-Python] how to daemonize twisted...and detatch from shell process
by Stephan 18 Oct '12

18 Oct '12

Hi, I'm attempting to create an ubuntu service that when started will run in the background processing beanstalk jobs. I've taken the code from beanstalk-client twisted to act as my beanstalk client ( https://github.com/dustin/beanstalk-client-twisted/blob/master/beanstalk.py) The issue I'm having is how do I set up my service (ocapi.py) so that when the daemon (ocapi) starts it detaches from the process, which isn't' happening right now. ocapi.py (service) <--this is what should detach and I put it in /usr/sbin/ocapi.py code found at http://pastebin.com/29VnXnrm ocapi (daemon) <-- goes int /etc/init.d/ocapi code found at http://pastebin.com/0QgRfTfu stephan@oc:~$ sudo /etc/init.d/ocapi start * Starting ocapi... Removing stale pidfile /var/run/ocapi.pid Connected! ...problem is I don't get my shell prompt back when i start the service, meaning it didn't detach from the parent process...hopefully someone can shed some light on this. Thanks, Stephan

4 5

Re: [Twisted-Python] Simpler Twisted deferred code via decorated callbacks
by Terry Jones 15 Oct '12

15 Oct '12

Here's a description of what the 'errback' decorator in my code does. (BTW, I've just updated it.) Here's normal Twisted errback code, assuming you do from twisted.web.client import getPage and from twisted.python import log. def logGetPageError(url): def handleError(failure, url): log.msg('Could not fetch URL %s.' % url) return getPage(url).addErrback(handleError, url) Using the 'errback' decorator you could instead write: def logGetPageError(url): @errback def handleError(failure, url): log.msg('Could not fetch URL %s.' % url) return handleError(getPage(url), url) As with the 'callback' decorator, the syntactic difference between the two in this case is minimal. The decorator returns a wrapping function that essentially does this: 0. Return a failed Deferred if the decorated function is called with no positional arguments (line 69). The reason for this is below. 1. Look at all its arguments. For any that are Deferreds, put them into a deferred list, which I'll call DL for this explanation. This is done for positional args (line 76) and keyword args (line 82). In our example, DL will contain the Deferred returned by getPage. 2. Suppose none of the arguments were Deferreds. If any of the args were instances of Failure (line 96), return a deferred that will fire with the result of calling the original wrapped (errback) function with all the passed arguments (at least one of which is a Failure). If no argument was a Failure, the original wrapped function is NOT called because there hasn't been any error. Instead (line 99), an already-fired Deferred (obtained via succeed) is returned with the value of the first positional argument. This is how the wrapping errback function passes the result along to its caller (if any). The other arguments to the wrapped function were obviously intended for that function, and are unused (this mirrors the way args passed to addErrback are unused if the errback is never run due to no error occurring). 3. If, as in our example, some of the arguments are deferreds (line 87) we have a DeferredList called DL that will eventually fire when all the Deferreds have fired (i.e., when all arguments are available for the wrapped function). Note that due to the use of addBoth (lines 79 and 84) non-Failure and Failure arguments coming from arguments that were Deferreds are both collected. These are collected, as they arrive, into a list (fargs) and a dictionary (fkw). Args that were not deferreds are already put into fargs and fkw (lines 78 and 86). 4. If DL fires with no errors (in our case, that means getPage returns a page), we act as we did in step 2: If any of the args are instances of Failure (line 89), the 'finish' callback added to DL (line 93) will return the result of calling the original wrapped (errback) function (handleError in our case) with all the passed arguments. If no argument was a Failure, the original wrapped function is NOT called because there hasn't been any error, and the callback on DL returns the first positional arg (on line 92). This last case corresponds to getPage returning successfully and the handleError errback not being called. The 'errback' decorator differs from the 'callback' decorator in that its wrapper function: - Collects all Failures in all arguments (including any coming from Deferreds, of course) in order to call the wrapped errback function. The 'callback' decorator OTOH immediately returns a failed Deferred as soon as any error happens (which is why its DeferredList uses fireOnOneErrback=True). - Must be called with at least one positional argument, so that in the case where nothing goes wrong with any Deferred argument (and no other arg is already a Failure) it has a value to pass back to its caller (if any). Both the wrapper functions sometimes do not call the wrapped function. This is symmetric and is as you'd expect: the 'callback' decorator doesn't call the wrapped (callback) function if there's any error, whereas the 'errback' decorator doesn't call the wrapped (errback) function unless there's an error. This is very similar to regular Twisted call/errback processing: if you're on the callback chain, errback functions aren't called & vice versa. Just as in regular Twisted code, a callback can produce a Failure or an errback can produce a non-Failure and you're back on the other chain. If you've read all this, great :-) It should be clear that you can roll up much more interesting cases, combining decorated callbacks and errbacks that accept multiple Deferred and non-Deferred arguments. Those are the cases where the syntactic / code complexity savings are much more apparent because (at least in many cases) there's no need to write code with addCallback, addErrback, or to use DeferredList. Some people's immediate reaction to all this is to tell me about inlineCallbacks. But this is very different, even though both share the universal decorator goal of making code look simpler / easier. inlineCallbacks lets you explicitly yield values from Deferreds. Unfortunately, anything can happen while you're context switched out before the yield yields. The decorators I've posted just wire up regular Twisted callback/errback processing chains, with no use of yield. There are other differences, but this post is already way long... Terry

1 0

Re: [Twisted-Python] Simpler Twisted deferred code via decorated callbacks
by Terry Jones 15 Oct '12

15 Oct '12

Here's a description of what the 'callback' decorator in my code does. Here's some normal Twisted code, assuming you do from twisted.web.client import getPage (yes, getPage is kind-of obsolete, but it's a concrete and conceptually simple deferred-returning function I like to use in examples). def pageLength(url): def _length(page): return len(page) return getPage(url).addCallback(_length) Using the 'callback' decorator you could instead write: def pageLength(url): @callback def _length(page): return len(page) return _length(getPage(url) The syntactic difference in this case is minimal. The saving is more significant in other cases, but let's consider this simplest case. The decorator returns a wrapping function that essentially does this: 1. Look at all its arguments. For any that are Deferreds, put them into a deferred list, which I'll call DL for this explanation. This is done for positional args (line 25) and keyword args (line 31). 2. If no arguments are Deferreds, just call the original _length function with the original arguments, and return the result in a Deferred using maybeDeferred (line 44). So, any function that is called with a set of arguments that are not Deferred instances just returns a deferred that fires with the function result or fails if the function raises (maybeDeferred takes care of that for us). 3. The interesting part is when some of the arguments are deferreds (line 36). In that case, we have a DeferredList called DL that will eventually fire when all the arguments are available. The arguments are collected, as they arrive, into a list (fags) and a dictionary (fkw). Args that were not deferreds are already put into fargs and fkw (lines 30 and 35). 4. If DL fires normally, we can go ahead and call the wrapped original function with all the arguments it was supposed to receive. In our _length case, there was only one argument, a deferred returning the content of a web page, so the original _length gets called with the page content. A lambda to call the original function is added to DL as a callback (line 41), and DL is returned. Note that DL has already fired, so adding the callback function to call the original function results in the function being called immediately. By adding it as a callback to DL, we can return a deferred (DL) from the wrapper. 5. When DL is made, I pass fireOnOneErrback=True to it. So if any of the arguments to the wrapped function result in an error (i.e., in our example, getPage fails), then DL will fail immediately. In that case, an errback (getSubFailure) attached to DL will pull the original failure out of DL and return it. Because one of the arguments to the original function has failed to materialize, we obviously can't call the original function. By just returning DL (which has failed), the failure is propagated (in a deferred) to any caller we may have. If you look at the code for the 'callback' decorator in decorate.py, the above should help to make things clearer. I'm happy to answer any questions, of course. I hope the description makes it clear how the decorator works on a function called with multiple deferred arguments. It's quite like a DeferredList that you can put non-deferreds objects into and add a callback to (where the callback is the decorated function). I implemented that once years ago, but didn't bother to post it. The 'errback' decorator is similar in structure, but operates differently of course. I'll send a separate mail describing it. Terry

1 0

Re: [Twisted-Python] Simpler Twisted deferred code via decorated callbacks
by Terry Jones 15 Oct '12

15 Oct '12

Hi Naveen > This is a really nice approach. I'm glad you like it. I've learned over the years to avoid decorators, but I nevertheless think these ones are promising. > Are there any downsides or functionality that can't be accomplished using > this approach? The main thing that's a little different is that it doesn't (yet) help with situations where you might want finer-grained control over callbacks and errbacks, or to use addBoth. I didn't want to tackle everything at once. Because the approach is all just normal deferreds, there was no need to. I wasn't sure what to do about addBoth - add a new decorator for it, let people use both decorators, etc. You can still just do this: def both(result): # do something @callback def first(result): # do something @callback def second(result): # do something second(first(getPage('http://www.something.com')).addBoth(both) The regular non-decorated Twisted code would be: def both(result): # do something def first(result): # do something def second(result): # do something getPage('http://www.something.com').addCallback(first) .addBoth(both).addCallback(second) As another example, in regular Twisted code you might want to do this: d = getDeferred() d.addCallback(f1) d.addCallbacks(f2, e1) Many people don't understand that that's not the same as d = getDeferred() d.addCallback(f1) d.addCallback(f2) d.addErrback(e1) The decorator approach doesn't provide directly for the former. You get the latter of course by writing e1(f2(f1(getDeferred))). You could have the former via f1(getDeferred).addCallbacks(f2, e1) in which case the decorators aren't buying you much. So I think there's some way to go on that front, but I've not yet thought about what would be most elegant/simple etc. Given a nice way to do addCallbacks, it can be used to implement addBoth (as in defer.py), so these two examples are really just the same thing. Glyph suggested I post with a simple example explaining exactly what the decorators are doing. I plan do that. Terry

1 0

[Twisted-Python] Simpler Twisted deferred code via decorated callbacks
by Terry Jones 15 Oct '12

15 Oct '12

This morning I was thinking about deferreds and how people find them difficult to grasp, but how they're conceptually simple once you get it. I guess most of us tell people a deferred is something to hold a result that hasn't arrived yet. Sometimes, though, deferreds do have a result in them immediately (e.g., using succeed or fail to get an already-fired deferred). I wondered if it might work to tell people to think of a deferred as really being the result. If that were literally true, instead of writing: d = getPage(...) d.addErrback(errcheck, args) d.addCallback(cleanup, args) d.addCallback(reformat, args) return d We might write something like: result1 = getPage(...) result2 = errcheck(result1, args) result3 = cleanup(result2, args) return reformat(result3, args) And if you could write that, you could obviously instead write: return reformat(cleanup(errcheck(getPage(...), args), args), args) If we could write Twisted code that way, I think using deferreds would be simpler for people unfamiliar with them. In the style we're all used to, the programmer manually adds callbacks and errbacks. That's basically boilerplate. It gets worse when you then need to also use DeferredList, etc. It's a little confusing to read deferred code at first, because you need to know that the deferred result/failure is automatically passed as the first arg to callbacks/errbacks. It seems to take a year or more for people to finally realize how the callback & errback chains actually interact :-) Also, I wonder how comfortable programmers are with code ordered innermost function first, as in the normal d.addCallback(inner).addCallback(outer) Twisted style, versus outer(inner()), as in the line above. Anyway... I realized we CAN let people use the succinct style above, by putting boilerplate into decorators. I wrote two decorators, called (surprise!) callback and errback. You can do this: @errback def errcheck(failure, arg): ... @callback def cleanup(page, arg): ... @callback def reformat(page, arg): ... reformat(cleanup(errcheck(getPage(...), arg1), arg2), arg3) The deferred callback and errback chains are hooked up automatically. You still get a regular deferred back as the return value. And... the "deferred" aspect of the code (or at least the need to talk about or explain it) has conveniently vanished. You can also do things like func1(getDeferred1(), errcheck(func2(getDeferred2(), getDeferred3()))) This gets the result of deferreds 2 & 3 and (if neither fails) passes the result of calling func2 on both results through to func1, which is also called with the result of deferred 1. You don't need to use DeferredLists, as the decorator makes them for you. The errcheck function wont be called at all unless there's an error. That's nice compared to the verbose equivalent: d1 = DeferredList([getDeferred2(), getDeferred3()]) d1.addCallback(func2) d1.addErrback(errcheck) d2 = DeferredList([getDeferred1(), d1]) d2.addCallback(func1) Or the more compact but awkward: DeferredList([getDeferred(), DeferredList([getDeferred(), getDeferred()]).addCallback(func2).addErrback(errcheck)]).addCallback(func1) There's lots more that could be said about this, but that's enough for now. The code (surely not bulletproof) and some tests are at https://github.com/terrycojones/twisted-callback-decorators I'll add a README sometime soon. This is still pretty much proof of concept, and some it could be done slightly differently. I'm happy to discuss in more detail if people are interested. Terry

2 1

[Twisted-Python] If you can read this, the mailing list is working again
by Itamar Turner-Trauring 15 Oct '12

15 Oct '12

We've been having some delivery issues, if this arrives in your inbox then probably the mailing list is working again. -Itamar

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 13 Oct '12

13 Oct '12

1 0

[Twisted-Python] txtorcon 0.6
by meejah＠meejah.ca 12 Oct '12

12 Oct '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 txtorcon 0.6 is now tagged and released. txtorcon is a Twisted-based Python asynchronous implementation of the Tor control protocol. It includes a state-tracking abstraction, configuration abstraction, Twisted endpoint support for hidden services, 96%+ unit-test coverage and many examples. New in this release: . debian packaging (mmaker); . psutil fully gone; . *changed API* for launch_tor() to use TorConfig instead of args; . TorConfig.save() works properly with no connected Tor; . fix incorrect handling of 650 immediately after connect; . pep8 compliance; . use assertEqual in tests; . messages with embdedded keywords work properly; . fix bug with setup.py + pip; . issue #15 reported along with patch by Isis Lovecruft; . consolidate requirements (from aagbsn); . increased test coverage and various minor fixes; . https URIs for ReadTheDocs Code and built documentation may be obtained: https://github.com/meejah/txtorcon https://txtorcon.readthedocs.org/en/latest/index.html git clone git://github.com/meejah/txtorcon.git You may also install it directly: pip install txtorcon easy_install txtorcon The sha256sum output is: 4f939e217ea3149175bb1285fa7296edc02ac245a71a60ec77b9cf48511e991e txtorcon-0.6.tar.gz thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/> iQEcBAEBAgAGBQJQd5uqAAoJEMJgKAMSgGmnTCkIAOGT6QaE+Q8jaBOedE79j/B3 QYMXh1SSfrvxn2x/utmjIgRbvCrCE+lzkxv9T9GXXjAhKSRN2xyyhRxq/Mbxvucd FWb2zHyZIjsTcc//veHYNpRNEWORl1ZIG2kU0muHQPpxY5ZAvtEkW+RA5zMJbhHp LF/JyltsAEu1Ja6HMFsfPmlLZiOtWBkBi3MgGGenUVJsScrtGRNCjWutGEkXxKXe 9pw2W0IGH5YmBAUiAuSW6q4cnyPQroqjSgO+RFfKMVGJNAaSqgUE8UCkcCifyjjM 2cDtGNA50kU0obT8QLgsiRyavePCQSfn9Jr1y8/Ck45Toa9a4FOQbBlOOyQSgrY= =bORU -----END PGP SIGNATURE-----

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 06 Oct '12

06 Oct '12

1 0

[Twisted-Python] Consensus on how to handle "MySQL server has gone away"
by E S 05 Oct '12

05 Oct '12

I have a service running that occasionally connects to a MySQL database. When there is no activity on it for some time, I eventually get the the error _mysql_exceptions.OperationalError: (2006, 'MySQL server has gone away') I have found a few posts on this group related to this issue http://twistedmatrix.com/pipermail/twisted-python/2007-July/015788.html http://twistedmatrix.com/pipermail/twisted-python/2007-October/016178.html as well as some tickets on the twisted homepage: http://twistedmatrix.com/trac/ticket/4404 http://twistedmatrix.com/trac/ticket/4964 I have also seen this referenced as a potential fix: http://www.gelens.org/2009/09/13/twisted-connectionpool-revisited/ I currently have the cp_reconnect parameter set to True, but it does not appear to be doing the job. I don't really see much consensus on how to properly handle this issue. Some people seem to think that the cp_reconnect parameter should take care of it for you, other people say that cp_reconnect is only part of the solution and that you have to write your own error handling. I realize that I could increase the value of MySQL's wait_timeout parameter, but I would rather have the twisted code detect and handle the situation. So, how does one do that? Thanks, Dave

7 9

Re: [Twisted-Python] Introducing Frack, a (partial) replacement for Trac
by Allen Short 04 Oct '12

04 Oct '12

That's strange. I believe that's due to twistedmatrix.com using PB to connect different web server processes together, not anything directly related to the code in Frack. Try this URL? It's working for me currently. http://twistedmatrix.com/users/frack.twistd/ui/ticket.html?id=1956

1 0

[Twisted-Python] Introducing Frack, a (partial) replacement for Trac
by Allen Short 04 Oct '12

04 Oct '12

Twisted's issue tracker is based on Trac, which has had increasingly bad performance over the years. Also, its code and deployment setup are so bad that everyone who has the ability to improve it has tried as hard as possible to avoid touching it at all. So I started writing a new issue tracker compatible with Trac's database, based on Twisted and more modern web technology. But it's incomplete and I'm not a JavaScript expert, so I need your help. If you've ever tried to contribute to Twisted and had to wait on Trac in order to read, comment on, or submit a ticket, contribute to Frack! It's already much faster than Trac for what it does, and there's plenty of room for you to help. If you want to help out with Twisted, but like writing HTML or Javascript too, contribute to Frack! It's written using modern, modular JavaScript with client side templates. It uses Twisted on the backend so using stuff like websockets for updates would be pretty easy to add. Interested? Here are some things we'd like to do soon: - Add a nice front page - Add a search page - Display more info on a ticket for reviewers the code is at: http://launchpad.net/frack The current prototype is running at http://twistedmatrix.com/users/frack.twistd/ui/ticket.html?id=<ticket_id>. You can contact me directly or post on the list with any issues or questions you have.

1 0

Re: [Twisted-Python] dynamic FTP realm
by Tom Sheffler 01 Oct '12

01 Oct '12

Hi Paul - I'm very new to twisted (I've only just started using it, so I apologise if > anything I ask is seemingly obvious!). > I'm looking to try and set up a FTP server which will serve a file > structure and files which don't exists on the server (the file structure > information is stored on cassandra, and the files are on s3). The file > structure and files will also be unique to the user, so different logins > will generate the file structure for that particular user, but I guess I > can adress that later. Here is a quick sketch of some of the steps ... but beware, I haven't debugged this. You'll want to subclass twisted.protocols.ftp.FTPShell with a new command interpreter class PaulsFTPShell(FTPShell): .... The you'll need to write a Realm that returns an instance of your FTP Shell: class PaulsFTPRealm(FTPRealm): """ @type anonymousRoot: L{twisted.python.filepath.FilePath} @ivar anonymousRoot: Root of the filesystem to which anonymous users will be granted access. """ implements(portal.IRealm) def __init__(self, ....) .... def requestAvatar(self, avatarId, mind, *interfaces): for iface in interfaces: if iface is IFTPShell: if avatarId is checkers.ANONYMOUS: avatar = None else: avatar = PaulsFTPShell( ...) return IFTPShell, avatar, getattr(avatar, 'logout', lambda: None) raise NotImplementedError("Only IFTPShell interface is supported by this realm") Then you'll stich it all together like this: f = FTPFactory() r = PaulsFTPRealm() p = Portal.Portal(r) p.registerChecker(...) f.portal = p Hope this helps get you started.

4 7