Mailman 3 September 2007 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] 2006, 'MySQL server has gone away'
by Werner Thie 15 Oct '07

15 Oct '07

Hi all While using mySQL V 5.0.33 and MySQL-python-1.2.2 with twisted/adbapi with the following connection params DB_DRIVER = "MySQLdb" DB_ARGS = { 'host': 'xxx.xxx.xxx.xxx', 'db': 'db', 'user': 'dbuser', 'passwd': 'dbpasswd' } I noticed (2006, 'MySQL server has gone away') errors, which seem to be not recoverable from an adbapi standpoint. Further analysis and reading the mySQL manual showed, that the mySQL server in versions > 5 seems to close open connections after (default) 8 hours. The reconnect feature of the db is disabled by default and there is no parameter to preset it via MySQL-python. Further digging brought up this link http://sourceforge.net/tracker/index.php?func=detail&aid=1483074&group_id=2… which patches the (also 1.2.2) version such, that a reconnect keyword is introduced in the DB_ARGS. Questions: - are there any adverse effects in applying this patch and setting reconnect: 1 in DB_ARGS? - is there a better, safer way to avoid this nasty error? Thxs, Werner

3 3

[Twisted-Python] Discovering Twisted : reactor.stop()
by Steve MULLER 03 Oct '07

03 Oct '07

Hello, I discovered Python recently and even more recently the Twisted module. I have a problem with the Twisted reactor and I would like to understand for which reason my program freezes. Here the code which should loop forever : from twisted.internet import reactor from twisted.internet import task class reactorProb(object): def __init__(self): self.count = 5 self._callback = task.LoopingCall(self.Heartbeat) self._callback.start(1, now = False) def Heartbeat(self): print "Top" self.count -= 1 if (self.count == 0): print "Exit" self._callback.stop() del self._callback reactor.stop() while (True): rp = reactorProb() print "Enter into reactor.run()" reactor.run() print "Leave from reactor.run()" I got this: Enter into reactor.run() Top Top Top Top Top Exit Leave from reactor.run() Enter into reactor.run() Top Top Top Top Top Exit Thus I start first once the reactor, I stop it then I start again and it freezes instead of stopping it the reactor. Could somebody explain me it why? Thank you in advance. smu

4 6

[Twisted-Python] Aborting callback chain with an errback
by Andy Fundinger 01 Oct '07

01 Oct '07

What is the proper way to handle an uncorrectable error? I have code that looks like: def callXMLRPC(self,stringValue,intValue): return(xmlProxy.callRemote("myFunc",{...}) \ .addCallback(self.rpcSuccess).addErrback(self.rpcFailure)) and in a derived class: def specificQuery(self): return self.callXMLRPC("Ask Thing", 3).addCallback(self.handleResult) what I'd like to have happen is if an error is raised in callRemote, rpcFailure does some cleanup and no further callbacks are executed. I can't fix the failure, but I don't want a message logged into my error log everytime, it's not unhandled, just that there's nothing to be done and callbacks won't have the data to run. If I want to deal with it further I'd want to do it in the base class. How can I do that? Am I doing something nonsensical here? Andy Fundinger -- Blog: http://channel3b.wordpress.com Second Life Name: Ciemaar Flintoff

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 29 Sep '07

29 Sep '07

Bug summary ______________________ Summary for 2007-09-23 through 2007-09-30 Bugs opened: 4 Bugs closed: 4 Total open bugs: 1035 (+0) |== Type Changes |== Priority Changes |== Component Changes |Defect: +2 |Highest: -1 |Core: -2 |Enhancement: -2 |High: +1 |Trial: +1 |Normal: +0 |Web2: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2836] limits of web2.resource.PostableResource can't be customized (opened by therve) enhancement web2 http://twistedmatrix.com/trac/ticket/2836 ===== High ===== [#2837] TreeReporter setup/teardown error handling broke (opened by pjd) defect trial http://twistedmatrix.com/trac/ticket/2837 ===== Normal ===== [#2838] _dumbwin32proc does not IProcessTransport make (opened by Peaker) defect core http://twistedmatrix.com/trac/ticket/2838 [#2839] _pollingfile fails to call stopProducing when connection is lost (opened by Peaker) defect core http://twistedmatrix.com/trac/ticket/2839 Closed Bugs ______________________ ===== Highest ===== [#2684] t.p.t.ThreadPool.dispatch should be deprecated (opened by indigo, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2684 [#2216] Coding Standard update: Lines should not have trailing whitespace (opened by jml, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2216 ===== Normal ===== [#2636] stdio.StandardIO breaks, readhack code in process.py to blame (opened by gtalvola, closed by therve, duplicate) defect core http://twistedmatrix.com/trac/ticket/2636 [#2101] Coding standard should fallback to PEP 8 (opened by jml, closed by therve, duplicate) enhancement core http://twistedmatrix.com/trac/ticket/2101 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2839] _pollingfile fails to call stopProducing when connection is lost (since 2007-09-25 12:50:46). Mean open ticket age: 530 days, 11:12:33.587451. Median: 429 days, 23:27:03.200978. Standard deviation: 402 days, 14:36:13.748171. Interquartile range: 504 days, 22:01:06. Mean time between ticket creation and ticket resolution: 121 days, 13:43:11.437464. Median: 14 days, 13:40:23. Standard deviation is 224 days, 14:04:54.647686. The interquartile range is 128 days, 3:43:59. Mean time spent in review: 65 days, 18:18:25.859556. Median: 4 days, 5:45:01. Standard deviation: 145 days, 4:55:30.540939. Interquartile range: 29 days, 10:44:36. Mean number of times a ticket is reviewed: 1.95081967213. Median: 1 Standard deviation: 1.58592108825. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 12 unique ticket reporters 6 unique ticket reviewers 2 unique ticket resolvers In the last 24 weeks, 76 unique ticket reporters 15 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 155 unique ticket reporters 19 unique ticket reviewers 26 unique ticket resolvers

1 0

[Twisted-Python] How to stop reactor when execution finishes on more than one ssh server?
by James Deng 29 Sep '07

29 Sep '07

Hi, in the sample sshsimpleclient.py, it demostrated how to connect to a ssh server and issue a command. my question is what should I do if I want to connect to more than one ssh server. I did try to create more than one SimpleTransport by using the following code HOSTS=[host1, host2, host3] protocol.ClientCreator(reactor, SimpleTransport).connectTCP(host1, 22) protocol.ClientCreator(reactor, SimpleTransport).connectTCP(host2, 22) protocol.ClientCreator(reactor, SimpleTransport).connectTCP(host3, 22) reactor.run() this really behaves as what i expected, and executed the command on all remote hosts, but after it finishes, i just hangs there, I know I need a place to call something like reactor.stop() when all transport finishes the execution. but i do not know where to call it, need some deffereds? and how? thanks very much! James

2 1

[Twisted-Python] tests for web2/channel/http
by Eduardo Schettino 27 Sep '07

27 Sep '07

Hello all, I am trying to learn trial... I am looking for examples of trial testing web2 code. I could not find tests for web2/channel/http.py They exist? Regards, Eduardo

2 1

[Twisted-Python] tracebacks in twisted web2
by Paul Perez 25 Sep '07

25 Sep '07

Dear List: I am having a heck of a time trying to figure out why this traceback is happening, and I was hoping that you could shed some light on the situation. I am sorry for bothering you with this simple thing, but I can't seem to figure it out! The code and traceback are below. The traceback only happens when I call httprequest.writeResponse(httpresponse) from a callback. When the same code is called inline and not in a deferred, it works just fine. Even stranger is that the response is written to the client even though the traceback occurs. An explanation of what I am trying to do: A consumer requests data. If there is data in the queue when the consumer first checks, return the result right away. If there is no data present, return a deferred and wait. When the producer inserts data into the queue, check if there is a consumer waiting. If so, begin callback chain on consumer. The code works fine (no traceback) in the case where there is no deferred needed (in QueueCache.maybedefer). When there is a deferred, the traceback happens during the callback (in QueueCache.put). Again, the consumer actually gets the correct result, but the traceback still occurs! Any ideas? Thanks in advance! ~P from twisted.internet import reactor, defer from twisted.web2 import http, responsecode import Queue TIMEOUT = 10 def respond(result, httprequest): httpresponse = http.Response(responsecode.OK, {}, stream=result) httprequest.writeResponse(httpresponse) class QueueCache(object): def __init__(self): self.queue = {} self.defer = {} def get_data(self, xSig): return self.get_queue(xSig).get_nowait() def push_data(self, xSig, value): self.get_queue(xSig).put(value) def get_queue(self, xSig): try: return self.queue[xSig] except KeyError: q = Queue.Queue() self.queue[xSig] = q return q def defer_request(self, xSig, httprequest): if self.defer.has_key(xSig): raise Exception("Client %s already has a deferred request!" %(xSig)) def timeout(deferred, request): xSig = request.args['X-Signature'][0] try: self.defer.pop(xSig) except KeyError: print 'Timeout called and xSig `%s` was not in cache!' % xSig respond('ACK',request) d = defer.Deferred() d.addCallback(respond, httprequest) d.setTimeout(TIMEOUT, timeout, httprequest) self.defer[xSig] = d return d def put(self, data): xSig = data['X-Signature'] try: d = self.defer.pop(xSig) d.callback(data['message']) except KeyError: self.push_data(xSig,data) def maybedefer(self, httprequest): xSig = httprequest.args['X-Signature'][0] try: data = self.get_data(xSig) result = http.Response( responsecode.OK, {}, stream=data['message'] ) except Queue.Empty: result = self.defer_request(xSig, httprequest) return result ----------------------------------------------------------------------------------------------------- 2007-09-25 18:38:38-0700 [-] Original exception: 2007-09-25 18:38:38-0700 [-] Unhandled Error Traceback (most recent call last): File "C:\Projects\PyServer\twisted\internet\defer.py", line 304, in _startRunCallbacks File "C:\Projects\PyServer\twisted\internet\defer.py", line 317, in _runCallbacks File "C:\Projects\PyServer\twisted\internet\defer.py", line 281, in _continue File "C:\Projects\PyServer\twisted\internet\defer.py", line 277, in unpause --- <exception caught here> --- File "C:\Projects\PyServer\twisted\internet\defer.py", line 317, in _runCallbacks File "C:\Projects\PyServer\twisted\web2\server.py", line 518, in _cbFinishRender exceptions.TypeError: html is not a resource or a response 2007-09-25 18:38:39-0700 [-] Unhandled error in Deferred: 2007-09-25 18:38:39-0700 [-] Unhandled Error Traceback (most recent call last): File "C:\Projects\PyServer\twisted\internet\defer.py", line 317, in _runCallbacks File "C:\Projects\PyServer\twisted\web2\server.py", line 476, in _processingFailed File "C:\Projects\PyServer\twisted\internet\defer.py", line 200, in addErrback File "C:\Projects\PyServer\twisted\internet\defer.py", line 182, in addCallbacks --- <exception caught here> --- File "C:\Projects\PyServer\twisted\internet\defer.py", line 317, in _runCallbacks File "C:\Projects\PyServer\twisted\web2\server.py", line 492, in _processingReallyFailed File "C:\Python24\lib\site-packages\twisted\web2\http.py", line 446, in writeResponse self.chanRequest.writeHeaders(response.code, response.headers) File "C:\Projects\PyServer\twisted\web2\channel\http.py", line 431, in writeHeaders File "C:\Projects\PyServer\twisted\web2\channel\http.py", line 466, in _writeHeaders exceptions.AttributeError: 'HTTPChannelRequest' object has no attribute 'transport'

1 0

[Twisted-Python] reactor.stop() doesn't stop the select reactor?
by Sam Roberts 25 Sep '07

25 Sep '07

I have unittests where each test: def test_this .. setup reactor.run() # ... when test is done reactor.stop() will be called def test_that .. same as above does some setup, runs the reactor until test is complete, then stops the reactor, asserting all went as intended. However, second call to reactor.stop() doesn't stop the reactor. strace shows reactor going back into select(), blocking forever, but killable with sigquit. Is this really intentional? Example code and testrun follow. ~/w/engine % python t_reactor_stop.py 2.5.1 (r251:54863, May 2 2007, 16:27:44) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] [twisted, version 2.5.0] <twisted.internet.selectreactor.SelectReactor object at 0x2b684344a310> stopping reactor running? 1 reactor stopped stopping reactor running? 1 zsh: quit (core dumped) python t_reactor_stop.py ~/w/engine % cat t_reactor_stop.py import sys print sys.version from twisted.internet import reactor import twisted print twisted._version.version print reactor def stop_reactor(): print "stopping reactor running? " + repr(reactor.running) try: reactor.stop() except: print "stop died!" reactor.callWhenRunning(stop_reactor) reactor.run() print "reactor stopped" reactor.callWhenRunning(stop_reactor) reactor.run() # Unreachable, why? print "reactor stopped" reactor.callWhenRunning(stop_reactor) reactor.run() print "reactor stopped"

3 4

[Twisted-Python] waiting for data before shutdown
by Andreas Poisel 25 Sep '07

25 Sep '07

Hi! My client is supposed to wait for two seconds to receive data when a shutdown event is triggered. In buildProtocol() the trigger is added: self.sdt= reactor.addSystemEventTrigger('before', 'shutdown', protocol.shutdown) The method shutdown() is supposed to wait for 2 seconds before closing the connection: def shutdown(self): self.doSomeCleanup() # works self.sendSomeData() # works d= defer.Deferred() reactor.callLater(2.0, d.callback, None) return d The shutdown() method gets called, cleanup and sending of data work as expected. But the connection is closed by my client _before_ the two seconds are over. Isn't the shutdown process supposed to be halted until the deferred returned by shutdown() is fired? Thank you! -- Bye, Andreas

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 22 Sep '07

22 Sep '07

Bug summary ______________________ Summary for 2007-09-16 through 2007-09-23 Bugs opened: 3 Bugs closed: 5 Total open bugs: 1035 (-2) |== Type Changes |== Priority Changes |== Component Changes |Defect: -2 |Highest: -3 |Conch: -2 |Task: +0 |Normal: +1 |Core: +1 |Words: -1 New / Reopened Bugs ______________________ ===== Highest ===== [#2835] _pollingfile calls pauseProducing on registered Pull producers (opened by Peaker) defect core http://twistedmatrix.com/trac/ticket/2835 ===== Normal ===== [#2833] deprecate twisted.internet.gtkreactor (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2833 [#2834] Logging should deal nicely with problems logging Failures (opened by itamarst) defect core http://twistedmatrix.com/trac/ticket/2834 Closed Bugs ______________________ ===== Highest ===== [#1884] Document adjustment to review policy (opened by exarkun, closed by exarkun, invalid) task core http://twistedmatrix.com/trac/ticket/1884 [#2099] manhole telnet server broken (opened by exarkun, closed by exarkun, invalid) defect conch http://twistedmatrix.com/trac/ticket/2099 [#1967] twisted.conch.telnet dies in conjunction with PuTTY (opened by sam, closed by exarkun, fixed) defect conch http://twistedmatrix.com/trac/ticket/1967 [#2809] 99.9% cpu load when using the epoll reactor (opened by mvw, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2809 ===== Normal ===== [#2828] xmlstream.IQ shouldn't create timeout deferreds for every IQ (opened by jack, closed by ralphm, invalid) defect words http://twistedmatrix.com/trac/ticket/2828 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2835] _pollingfile calls pauseProducing on registered Pull producers (since 2007-09-22 00:36:54). Mean open ticket age: 524 days, 9:14:54.799999. Median: 422 days, 23:27:03.859903. Standard deviation: 401 days, 17:45:04.268446. Interquartile range: 503 days, 18:33:50. Mean time between ticket creation and ticket resolution: 121 days, 7:14:56.715258. Median: 14 days, 10:59:04. Standard deviation is 224 days, 16:56:44.223591. The interquartile range is 127 days, 8:25:41. Mean time spent in review: 65 days, 4:49:15.983357. Median: 4 days, 5:45:01. Standard deviation: 143 days, 15:51:23.365689. Interquartile range: 29 days, 10:44:36. Mean number of times a ticket is reviewed: 1.94600938967. Median: 1 Standard deviation: 1.58466700636. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 16 unique ticket reporters 5 unique ticket reviewers 3 unique ticket resolvers In the last 24 weeks, 78 unique ticket reporters 14 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 155 unique ticket reporters 19 unique ticket reviewers 26 unique ticket resolvers

1 0

Re: [Twisted-Python] Web Proxy that does something different once a day
by Fred Trotter 21 Sep '07

21 Sep '07

> It looks like this is just for the web browsers > (i.e., HTTP protocol). If so, then at step 2 > you would make your test. If you decide A should > go to the "required reading" URL instead of the > requested URL, simply skip steps 3-5 and return > a valid HTTP document with a "moved temporarily" > code (30x, I think). The browser will then request > the other URL. The one catch is that, depending on > the browser, the user may get the same page if they > request the original URL again in the same browser > window/session. If that's the case, they would need > to open a new window to get to the original URL. That sounds reasonable and good. Any clue how you would implement that in Twisted? -FT -- Fred Trotter http://www.fredtrotter.com

1 0

Re: [Twisted-Python] Web Proxy that does something different once a day
by Samuel Reynolds 21 Sep '07

21 Sep '07

>Hoping someone else chimes in with how to stick a redirect into a proxy, Nothing to do with Twisted, per se. A proxy is a two-way conduit: 1. Receive a request from A 2. Massage and/or log the request as necessary 3. Make a request to B. 4. Receive the response from B. 5. Massage and/or log the response as necessary 6. Return the response to A. It looks like this is just for the web browsers (i.e., HTTP protocol). If so, then at step 2 you would make your test. If you decide A should go to the "required reading" URL instead of the requested URL, simply skip steps 3-5 and return a valid HTTP document with a "moved temporarily" code (30x, I think). The browser will then request the other URL. The one catch is that, depending on the browser, the user may get the same page if they request the original URL again in the same browser window/session. If that's the case, they would need to open a new window to get to the original URL. - Sam

1 0

AW: Re: [Twisted-Python] Can a deferred catch a segmentation fault ?
by Andreas Kostyrka 21 Sep '07

21 Sep '07

Setting up such signal handler wouldn't be trivial (as in unsafe). The "normal" thing in such a situation is to longjmp (in C) or to let the process to die (which is the safe thing to do), Now presuming that you cannot fix the segfaults (which as sad it sounds is quite common when using commercial 3rd party DLLs. Practice shows that being even a huge customer often get a quick turn around on bugs especially if its a hard to reproduce random crash. Driver DLLs for hardware costing $10K upwards come to mind, *shudder*), your only safe way is to isolate the driver usage into a seperate process. (If you are unlucky not even that is enough because some hardware gets stuck in a state that needs manual cold reseting, but let's be optimistic here). With a seperate process, you can restart it should it die. Andreas -- Ursprüngl. Mitteil. -- Betreff: Re: [Twisted-Python] Can a deferred catch a segmentation fault? Von: Jean-Paul Calderone <exarkun(a)divmod.com> Datum: 20.09.2007 16:34 On Thu, 20 Sep 2007 09:14:11 -0700, "Gerald John M. Manipon" <geraldjohn.m.manipon(a)jpl.nasa.gov> wrote: >Hi, > >I'm using deferToThread to run some python code which may or may not >be safe. In particular, the code may load some modules that could >seg fault. In this case, deferToThread won't work and the python >process will bomb so I was wondering what Defer strategy I could >use to handle this situation: Deferreds don't do anything which will directly help in this situation. If your process receives a signal for which the action is to exit, then it will exit. If you want, you can change the action associated with certain signals (even SIGSEGV). What action you *do* cause to happen in such a case is independent of Deferreds. You could, conceivably, turn a segfault into an exception and then errback a Deferred. However, I wouldn't actually recommend this. I would recommend not using software which segfaults, fixing segfault bugs you find in software you are using, and having comprehensive unit tests so that if something is going to segfault, it does so during the course of development, not on whatever server or client machines your software is deployed to. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

4 3

[Twisted-Python] Web Proxy that does something different once a day
by Fred Trotter 21 Sep '07

21 Sep '07

Hello, I am new to python and I am potentially interested in writing a new protocol with Twisted. However I wanted to start with a simpler problem that I have been working on, so that I can understand better how twisted works. A particular client wants to enforce a home page for the first internet connection each day. They have a intranet portal that gives messages to employees and they want to be sure that each employee, no matter how they have their homepage set, will see the messages the first time they connect. After the first connection they want the proxy to simply forward requests to the internet without filtering. Since I need to understand this issue for my other project too, how do I get the python loop do one thing based on a "new day" for an IP address and then do something else after that action has been completed. Example code that addresses my problem would be great, but I would be happy to have links to examples that do this... Thanks!! -- Fred Trotter http://www.fredtrotter.com

3 3

[Twisted-Python] Twisted Server and Objective-C Client
by Stefan Arentz 21 Sep '07

21 Sep '07

I need to let Objective-C (OS X / Cocoa) clients speak to a Twisted server. The protocol needs to be a pretty simple request/response style protocol on a persistent TCP connection. Possible over SSL. Before I start inventing a wheel, does anyone know if there is something simple available already? Would the Perspective Broker be a good start? I haven't looked at the internals but maybe it is not too difficult to create a non-python client for it? Just thinking out loud :) S.

2 1

[Twisted-Python] Can a deferred catch a segmentation fault?
by Gerald John M. Manipon 20 Sep '07

20 Sep '07

Hi, I'm using deferToThread to run some python code which may or may not be safe. In particular, the code may load some modules that could seg fault. In this case, deferToThread won't work and the python process will bomb so I was wondering what Defer strategy I could use to handle this situation: from twisted.internet import threads, reactor def segFaultFunc(): import binning #this will seg fault return True def myCb(res): print "Got", res reactor.stop() d = threads.deferToThread(segFaultFunc) d.addCallback(myCb) reactor.run() Any help on this is greatly appreciated. Thanks, Gerald

2 2

[Twisted-Python] Who's using Gtk reactor?
by Jean-Paul Calderone 20 Sep '07

20 Sep '07

Greetings, As part of the work I've been doing to fix a bug in epoll reactor's handling of "unclean" tcp disconnection (the case which gives ConnectionLost instead of ConnectionDone), I've noticed that gtk reactor also doesn't handle this case correctly. Anyone who has looked at gtkreactor.py probably knows this (there's a big comment in _readAndWrite, so whoever wrote it in the first place realized what they were doing). Before I fixed epoll reactor, I added a test for this case. Gtk reactor won't be able to pass this test, since it doesn't provide the correct functionality. I'm not going to fix gtk reactor, since glib2 or gtk2 reactor is really what people should be using these days. However, if someone else is interested in fixing gtk reactor, the fix would be happily accepted (the hard part is done, since the unit test is written already ;). If no one is interested in fixing it, though, then I think this is a good time to start thinking about deprecating gtk reactor and eventually removing it. Anyone have any thoughts on this? Jean-Paul

2 2

[Twisted-Python] twisted.trial and test discovery
by Drew Smathers 19 Sep '07

19 Sep '07

Maybe a dead horse, but there was some prior discussion on nose vs. trial in Feb: http://twistedmatrix.com/pipermail/twisted-python/2007-February/014771.html I like trial a lot (both for Twisted and non-Twisted applications - heck I've even tested [gasp] Java applications with it). The only somewhat interesting feature of nose (vs. trial) I've found is test discovery. Is test discovery a planned feature for trial? -- \\\\\/\"/\\\\\\\\\\\ \\\\/ // //\/\\\\\\\ \\\/ \\// /\ \/\\\\ \\/ /\/ / /\/ /\ \\\ \/ / /\/ /\ /\\\ \\ / /\\\ /\\\ \\\\\/\ \/\\\\\/\\\\\/\\\\\\ d.p.s

2 3

[Twisted-Python] Control the number of parallel system calls
by Daniel de la Cuesta 19 Sep '07

19 Sep '07

6 6

Re: [Twisted-Python] cannot access svn repo
by Terry Jones 19 Sep '07

19 Sep '07

Hi Nicholas | [nmattiello@megatron ~]$ svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk This works fine for me too. | svn: Can't connect to host 'svn.twistedmatrix.com': Connection refused | [nmattiello@megatron ~]$ telnet svn.twistedmatrix.com 3690 | Trying 198.49.126.131... | telnet: connect to address 198.49.126.131: Connection refused | telnet: Unable to connect to remote host: Connection refused | | what could be wrong? Firewall? Terry

2 4

[Twisted-Python] cannot access svn repo
by Nicholas Mattiello 19 Sep '07

19 Sep '07

hello, I am trying to check out from the svn repository, but it keeps refusing me, even when i run the svn co command on the downloads page. svn co svn://svn.twistedmatrix.com/svn/Twisted/trunk svn: Can't connect to host 'svn.twistedmatrix.com': Connection refused any help would be appreciated, thanks.

3 4

[Twisted-Python] Transmit queue - how would I go at it?
by Nadav Aharony 16 Sep '07

16 Sep '07

Hi, I am working on an application where clients request files, and since each file is very large, the server ACKs the requests and then queues them to be served later, one at a time. I am relatively new to Twisted and was wondering what would be the best way to approach this. Here is what I came up with so far: 1) Use Deferred callbacks, and string together callbacks to a "file sender" function - once a file is sent, the function is called back with a pointer to the next file in line. If I do so - is there some automatic garbage collection of the callbacks that have already been handled? (there may eventually be hundreds of files to be sent) In this approach, would there be a natural way to check if a new request is actually a repeat of a previous request already queued? 2) Use some type of queue - where each new file request is queued, and some single function handles them one at at time (I guess with asynchronous callbacks to itself every time the transport layer is free to send another file and the queue is not empty). Is there some queue structure like this in Twisted? I saw something called a DefferedQueue but could not find many examples or documentation about it and about its usage. I mainly want to understand, from the "Twisted mindset" which would be the the right way to go? Also, say in the future I would want to implement priority queuing and handling of the requests rather than a FIFO - how would I do that in Twisted? Any thoughts and/or code examples would be greatly appreciated Thanks! Nadav

3 5

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 15 Sep '07

15 Sep '07

Bug summary ______________________ Summary for 2007-09-09 through 2007-09-16 Bugs opened: 6 Bugs closed: 5 Total open bugs: 1037 (+1) |== Type Changes |== Priority Changes |== Component Changes |Defect: +1 |Highest: -2 |Core: +0 |Normal: +3 |Mail: +1 |Names: -1 |Web2: +0 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2830] string exc_value special case in Failure.__init__ misses cases (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2830 [#2831] twisted.test.test_tcp.AddressTestCase.testBuildProtocol is buggy and overly complex (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2831 ===== Normal ===== [#2827] calling IConnector.disconnect() from a connectSSL does not disconnect if remote end is silent (opened by matthubb) defect core http://twistedmatrix.com/trac/ticket/2827 [#2828] xmlstream.IQ shouldn't create timeout deferreds for every IQ (opened by jack) defect words http://twistedmatrix.com/trac/ticket/2828 [#2829] twisted.web2.client.http, fixed (opened by synx) (CLOSED, invalid) defect web2 http://twistedmatrix.com/trac/ticket/2829 [#2832] twisted.mail.test.test_smtp.TimeoutTestCase.testSMTPClient fails intermittently due to timing issues (opened by exarkun) defect mail http://twistedmatrix.com/trac/ticket/2832 Closed Bugs ______________________ ===== Highest ===== [#1730] twisted.test.test_tcp.HalfClose3TestCase.testReadNotificationRaises fails intermittently (opened by radix, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1730 [#2831] twisted.test.test_tcp.AddressTestCase.testBuildProtocol is buggy and overly complex (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2831 [#2807] twisted.mail.relaymanager.MXCalculator._cbMX raises unhandled IndexError for response with no answers (opened by exarkun, closed by exarkun, fixed) defect names http://twistedmatrix.com/trac/ticket/2807 [#2830] string exc_value special case in Failure.__init__ misses cases (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2830 ===== Normal ===== [#2829] twisted.web2.client.http, fixed (opened by synx, closed by glyph, invalid) defect web2 http://twistedmatrix.com/trac/ticket/2829 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2832] twisted.mail.test.test_smtp.TimeoutTestCase.testSMTPClient fails intermittently due to timing issues (since 2007-09-12 10:55:56). Mean open ticket age: 517 days, 14:21:27.445920. Median: 415 days, 23:27:03.233770. Standard deviation: 401 days, 0:49:47.008297. Interquartile range: 502 days, 16:56:16. Mean time between ticket creation and ticket resolution: 120 days, 22:26:58.101251. Median: 14 days, 8:13:35. Standard deviation is 224 days, 16:16:14.995707. The interquartile range is 127 days, 2:42:46. Mean time spent in review: 64 days, 4:15:36.993554. Median: 4 days, 0:03:10. Standard deviation: 141 days, 14:18:03.262941. Interquartile range: 29 days, 2:08:35. Mean number of times a ticket is reviewed: 1.93882352941. Median: 1 Standard deviation: 1.56762046109. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 20 unique ticket reporters 4 unique ticket reviewers 5 unique ticket resolvers In the last 24 weeks, 82 unique ticket reporters 13 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 157 unique ticket reporters 18 unique ticket reviewers 26 unique ticket resolvers

1 0

[Twisted-Python] XML-RPC Service with Database Access
by Stefan Arentz 15 Sep '07

15 Sep '07

I'm interested in using Twisted for a simple XML-RPC service that needs to talk to either PostgreSQL or MySQL. (I would prefer PG). I've got the XML-RPC part under control, that is fairly simple with Twisted Web. But what would I use for the database part? What is generally the easy way to do database access with Twisted? Are there ORM solutions that are compatible with Twisted? It would be nice if I could use something like SQLObject instead of raw queries. I'm not looking for a complete solution here. Just trying to understand what is usually done in Twisted land. S.

3 2

[Twisted-Python] Cometd?
by Stefan Arentz 15 Sep '07

15 Sep '07

Has anyone done any work in the area of cometd? I'm interested in using twisted for a cometd load tester. If there is already code that I can build on that would be great. S.

2 2

Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

That indeed worked. It did occur to me for a fleeting moment when I looked at the man page; however, when I ran 'twistd --help-reactors' it spit out nothing so I was not sure of exactly what it wanted or if it worked at all. ----- Original Message ---- From: Christopher Armstrong <radix(a)twistedmatrix.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 12, 2007 5:15:53 PM Subject: Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory? On 9/12/07, Beau Hargis <beau(a)subobscur.us> wrote: > For one of the services I want to use pollreactor or epollreactor, so each of the Services has only 4 lines in it: > > from twisted.internet import pollreactor > pollreactor.install() > import Project.Processors.ProcessorX > theService = Project.Processors.ProcessorX.ProcXServiceMaker() Don't install a reactor in the plugin: in fact, don't import the reactor at all. The correct way to choose the reactor is by passing the "-r" or "--reactor" argument to 'twistd', like 'twistd -r poll myplugin' -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

1 0

Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

I am using the new twisted/plugin setup. I am not using mktap. So, in my project directory (which is included in the syspath) I have a dir 'twisted/plugins' into which I am going to put each of the services/applications. I do not want them running in the same process because I want to be able to stop one without affecting the others. Some of the services process a lot of data from 80-100 simultaneous connections so I need to be able to use multiple processors on the machine. The directory structure looks something like this: Project\ ...project.py ...Processors\ ......ProcessorX.py ......ProcessorY.py ...twisted\ ......plugins\ .........Service1.py .........Service2.py For one of the services I want to use pollreactor or epollreactor, so each of the Services has only 4 lines in it: from twisted.internet import pollreactor pollreactor.install() import Project.Processors.ProcessorX theService = Project.Processors.ProcessorX.ProcXServiceMaker() This all works great until I want to add a second service in the same manner. When I add the second service I get the error about the reactor already being installed. So 'twistd --help' will only list one of the services and scrolling up the screen one can see the error it spits out when it attempts to evaluate the second Service. If I start the first service, it finds it and then I comment out any reference to importing the reactor, then it will successfully evaluate the first AND second Service and allow me to start the second one. This is the error that it spits out: File "/usr/lib/python2.5/site-packages/twisted/internet/pollreactor.py", line 183, in install main.installReactor(p) File "/usr/lib/python2.5/site-packages/twisted/internet/main.py", line 24, in installReactor "reactor already installed" exceptions.AssertionError: reactor already installed And sometimes it will be a random failure (which is probably a cache interaction). It seems to be the combination of reactors. If I install the pollreactor in one and the epollreactor in the other then it works, but only one time. Then I check again and it fails. It seems to be loading up all the plugins in the same context. ----- Original Message ---- From: Jean-Paul Calderone <exarkun(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 12, 2007 2:26:29 PM Subject: Re: [Twisted-Python] Multiple Twisted Application Plugins in the same directory? On Wed, 12 Sep 2007 14:06:50 -0700 (PDT), Beau Hargis <beau(a)subobscur.us> wrote: >I started creating my services as twisted application plugins. When adding a second service as a plugin in the same directory as the first I get an error that the reactor was already installed. If I comment out the import of the reactor in one then I can start the other application. Neither the tutorial nor the source indicate what is going on except that it checks to see if a reactor is already installed at some point. What is the nuance here? Should I just go back to .tac files for these apps? Could you describe what you're trying in a little more detail? In particular, I'm not sure if you're using mktap to create a tap file with multiple services, if you're using the original plugin system (using tml files) or if you're using the new plugin system (twisted.plugin). Generally speaking, there's no problem running more than one service in a single process. A service is just an object which receives startup and shutdown notification. Anything beyond that is an application-specific thing. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1

[Twisted-Python] Multiple Twisted Application Plugins in the same directory?
by Beau Hargis 12 Sep '07

12 Sep '07

I started creating my services as twisted application plugins. When adding a second service as a plugin in the same directory as the first I get an error that the reactor was already installed. If I comment out the import of the reactor in one then I can start the other application. Neither the tutorial nor the source indicate what is going on except that it checks to see if a reactor is already installed at some point. What is the nuance here? Should I just go back to .tac files for these apps?

2 1

[Twisted-Python] Reconnecting PB Client
by Don Smith 12 Sep '07

12 Sep '07

Does anyone have some example code for building a reconnecting PB client? This is what I have now: from twisted.spread import pb from twisted.internet import reactor from twisted.python import util events = pb.PBClientFactory() reactor.connectTCP("localhost", 8789, events) queue = events.getRootObject()

3 2

[Twisted-Python] The Reactor
by Terry Jones 12 Sep '07

12 Sep '07

Word leaks that the Google Reader coders call their back-end "Reactor". http://blogoscoped.com/archive/2007-09-11-n21.html One wonders. Terry

1 0

[Twisted-Python] trial on specific directory
by Nathaniel Haggard 11 Sep '07

11 Sep '07

How do I tell trial to test the code in /mydir with the /mydir/test/test_class1 instead testing the code from /lib/site-packages like it does by default. Thanks, Nate

3 2

Re: [Twisted-Python] Wrapping a Perspective
by Andrew Francis 11 Sep '07

11 Sep '07

Hi Rasjid: > It might be possible to create a new twisted reactor > that integrates into the python com server event >loop. That would presumably be the correct way to >deal with this. But that would seem like far more work > that I have time to devote to this problem at this > time. Hopefully, I understood your problem correctly... If you are willing to use Stackless Python, I would recommend you use Christopher Armstrong's threadlesss.py blockOn function. This would allow Twisted calls to block. A few months ago, I posted a solution that would work, but it would be overkill. I am also curious why you can't use inline callbacks? If I can find the time, I will post an example. > So my question really is: > What are the problems / brawbacks with running the > twisted reactor in > a non-main thread, provided all calls to the reactor > are done through > twisted.internet.blockingCallFromThread? I have used threads with Twisted and Stackless and found I took a performance hit. Also I found it prudent to heed the advice and try not to mix threads with Twisted (or Stackless) Cheers, Andrew ____________________________________________________________________________________ Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7

4 8

[Twisted-Python] epoll reactor in Twisted 2.50 tarball
by Alvin Delagon 10 Sep '07

10 Sep '07

Hello guys, I have to install Twisted 2.50 on our Ubuntu Dapper servers. The problem is that I have to use the Twisted 2.50 tarball since Dapper apt repositories only have Twisted 2.40. The installation went smoothly and my twisted applications works except that when I try to run my scripts using the epoll reactor (twistd -ny ./myscript -r epoll) I get this error. Traceback (most recent call last): File "/usr/bin/twistd", line 21, in ? run() File "/usr/lib/python2.4/site-packages/twisted/scripts/twistd.py", line 27, in run app.run(runApp, ServerOptions) File "/usr/lib/python2.4/site-packages/twisted/application/app.py", line 374, in run config.parseOptions() File "/usr/lib/python2.4/site-packages/twisted/application/app.py", line 354, in parseOptions usage.Options.parseOptions(self, options) File "/usr/lib/python2.4/site-packages/twisted/python/usage.py", line 177, in parseOptions self.__dispatch[optMangled](optMangled, arg) File "/usr/lib/python2.4/site-packages/twisted/python/usage.py", line 333, in <lambda> fn = lambda name, value, m=method: m(value) File "/usr/lib/python2.4/site-packages/twisted/application/app.py", line 276, in opt_reactor installReactor(shortName) File "/usr/lib/python2.4/site-packages/twisted/application/reactors.py", line 80, in installReactor installer.install() File "/usr/lib/python2.4/site-packages/twisted/application/reactors.py", line 60, in install namedAny(self.moduleName).install() File "/usr/lib/python2.4/site-packages/twisted/python/reflect.py", line 357, in namedAny topLevelPackage = __import__(trialname) File "/usr/lib/python2.4/site-packages/twisted/internet/epollreactor.py", line 22, in ? from twisted.python import _epoll ImportError: cannot import name _epoll On my workstation (LinuxMint 3.0), the epoll reactor works fine. The difference is that I installed Twisted via apt-get. Thanks in advance for anyone who can point me to the right way. --- Alvin Delagon

2 2

[Twisted-Python] How to ... in Conch?
by Konrads Smelkovs 09 Sep '07

09 Sep '07

1) I want to invoke(exec) a command with arguments from a SSH Conch client. The example suggests that arguments should be passed along with command, so to invoke 'ls -l', one would pass common.NS('ls -l'). What to do about arguments that have spaces? Should I slash-escape them? 2) How to obtain the retcode from an exec'ed command? -- Konrads Smelkovs Applied IT sorcery.

2 4

[Twisted-Python] twistd doesn't work
by David Langford 09 Sep '07

09 Sep '07

Hey everyone I'm going through the finger tutorial -> http://twistedmatrix.com/projects/core/documentation/howto/tutorial/intro.h…. And at the end of the first section it talks about using the twistd command. Now it just plain doesn't work for me, telling me the command is not found. I'm using a macbook pro and i type the command into terminal. Can anyone tell me what i'm doing wrong. Because the next section doesn't work (i'm assuming its related to my lack of twistd-ness) I know this a n00bish question, and i've been searching for an answer, but everyone else seems to be able to get it working and no one mentions having problems. Then i thought maybee i have to download something extra for twisted, but a search of the site only returned tickets (bug tickets) instead of an actual search of the site. Any help would be greatly appreciated Lango

5 7

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 08 Sep '07

08 Sep '07

Bug summary ______________________ Summary for 2007-09-02 through 2007-09-09 Bugs opened: 19 Bugs closed: 7 Total open bugs: 1036 (+12) |== Type Changes |== Priority Changes |== Component Changes |Defect: +7 |Highest: +1 |Core: +7 |Enhancement: +7 |High: +0 |Ftp: -1 |Task: +1 |Normal: +14 |Trial: +3 |Vfs: +6 New / Reopened Bugs ______________________ ===== Highest ===== [#2811] eliminate shared mutable state from reactor implementations (opened by exarkun) (CLOSED, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2811 [#2821] create twistd plugin for vfs (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2821 [#1264] twisted.vfs.test.test_ftp is enough to drive a sane man mad (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/1264 ===== High ===== [#2822] deprecate twisted.python.reflect.filenameToModuleName so that it can be removed (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2822 ===== Normal ===== [#2812] cfreactor replacement using threadedselectreactor (opened by termie) enhancement core http://twistedmatrix.com/trac/ticket/2812 [#2813] logstr initialization should happen before calling makeConnection in tcp.BaseClient (opened by therve) enhancement core http://twistedmatrix.com/trac/ticket/2813 [#2814] bugfix in epollreactor.py: sometime event cann't be identified when connection is lost (opened by huyx) (CLOSED, duplicate) defect core http://twistedmatrix.com/trac/ticket/2814 [#2815] Update VFS backends to an async interface (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2815 [#2816] Port the vfs ftp adapter to new async backends (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2816 [#2817] Port the vfs sftp adapter to new async backends (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2817 [#2818] Port the vfs stream adapter to new async backends (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2818 [#2819] Delete old sync vfs code and cleanup (opened by cablehead) enhancement vfs http://twistedmatrix.com/trac/ticket/2819 [#2820] trial warning reporting uses print to stdout (opened by exarkun) defect trial http://twistedmatrix.com/trac/ticket/2820 [#2823] trial path/to/package// fails immediately with AssertionError (opened by exarkun) defect trial http://twistedmatrix.com/trac/ticket/2823 [#2824] spawnProcess and screen -D -m not getting along (opened by TTimo) defect core http://twistedmatrix.com/trac/ticket/2824 [#2825] reactors handle case where socket.fileno() returns -1 (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/2825 [#2826] twisted.internet.tcp.Server.repstr wrong when listening on port 0 (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/2826 [#2188] Trial catches exceptions during import (opened by jdahlin) defect trial http://twistedmatrix.com/trac/ticket/2188 [#2735] twisted.vfs.adapters.ftp._FileToConsumerAdapter doesn't fully implement IConsumer (opened by therve) defect vfs http://twistedmatrix.com/trac/ticket/2735 Closed Bugs ______________________ ===== Highest ===== [#1904] Passive port ranges are not settable in t.p.ftp (opened by krp, closed by therve, fixed) enhancement ftp http://twistedmatrix.com/trac/ticket/1904 [#2811] eliminate shared mutable state from reactor implementations (opened by exarkun, closed by exarkun, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2811 [#1264] twisted.vfs.test.test_ftp is enough to drive a sane man mad (opened by exarkun, closed by therve, ) defect core http://twistedmatrix.com/trac/ticket/1264 ===== High ===== [#1223] vfs methods should always return deferreds (opened by cablehead, closed by cablehead, duplicate) defect vfs http://twistedmatrix.com/trac/ticket/1223 ===== Normal ===== [#2188] Trial catches exceptions during import (opened by jdahlin, closed by exarkun, ) defect trial http://twistedmatrix.com/trac/ticket/2188 [#2814] bugfix in epollreactor.py: sometime event cann't be identified when connection is lost (opened by huyx, closed by therve, duplicate) defect core http://twistedmatrix.com/trac/ticket/2814 [#2735] twisted.vfs.adapters.ftp._FileToConsumerAdapter doesn't fully implement IConsumer (opened by therve, closed by therve, ) defect vfs http://twistedmatrix.com/trac/ticket/2735 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2826] twisted.internet.tcp.Server.repstr wrong when listening on port 0 (since 2007-09-08 18:59:55). Mean open ticket age: 511 days, 13:38:27.598094. Median: 408 days, 23:27:57.886705. Standard deviation: 400 days, 13:51:14.700782. Interquartile range: 502 days, 7:38:45. Mean time between ticket creation and ticket resolution: 120 days, 23:56:32.985168. Median: 14 days, 8:13:35. Standard deviation is 224 days, 18:24:36.530891. The interquartile range is 127 days, 8:11:35. Mean time spent in review: 64 days, 0:26:37.767588. Median: 4 days, 0:03:10. Standard deviation: 140 days, 6:05:08.850661. Interquartile range: 29 days, 11:02:35. Mean number of times a ticket is reviewed: 1.93571428571. Median: 1 Standard deviation: 1.57000476623. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 18 unique ticket reporters 6 unique ticket reviewers 6 unique ticket resolvers In the last 24 weeks, 81 unique ticket reporters 14 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 157 unique ticket reporters 19 unique ticket reviewers 27 unique ticket resolvers

1 0

Re: [Twisted-Python] Wrapping a Perspective
by Andrew Francis 07 Sep '07

07 Sep '07

Hi Rasjid: >I have read about Stackless, and it looks like it >would be really fun to play with. But I don't think my employer would be keen to go as 'fringe' as Stackless. Once you get the swing of it, Stackless Python is pretty fun... and powerful. Even more fun when one figures out how to properly integrate Stackless Python with Twisted. Anyhow I haven't played with the Perspective Broker before. So I read the Bruce Eckel "Grokking Twisted" article and wrote a simple example to illustrate how to make Twisted inter-operate with Stackless. The essense of the solution is the approach Armstrong takes in threadless.py. I provide a bit more syntactic sugaring: 1) Use a proxy that slightly modifies the method calls, mainly to hide the underlying deferred. 2) trigger a "TwistedException." example fragment: def execute(self, host, port): global flag factory = pb.PBClientFactory() log.msg("connecting") reactor.connectTCP(host, port, factory) log.msg("connected") try: perspective = self.processor.getRootObject(factory) print "=>", processor.calculate(perspective, 10, 20) flag = False except TwistedException: log.msg("A Twisted Exception was thrown") log.msg(sys.exc_info()) One of my long term goals is to standardise the approach and auto generate proxies and exceptions as to make Twisted safe for the average Stackless Python programmer. Cheers, Andrew P.S - reference http://www.artima.com/weblogs/viewpost.jsp?thread=156396 PBServer.py - is Bruce Eckel's server slightly modified. ____________________________________________________________________________________ Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&…

1 0

Re: [Twisted-Python] Properly handling Connection Refused
by Beau Hargis 05 Sep '07

05 Sep '07

That small script worked as it should. The traceback is exactly the same as in my application. I just wonder where the '[Uninitialized]' is coming from. How is it ending up like that? Is it getting garbage collected for some reason? I can't catch the state of the port fast enough to see if it's anything else in the stack. I am using the poll reactor. I will change it back to the default select reactor and see if I see a problem with it again and see what other tracing I can do. ----- Original Message ---- From: Jean-Paul Calderone <exarkun(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 5, 2007 12:54:31 PM Subject: Re: [Twisted-Python] Properly handling Connection Refused On Wed, 5 Sep 2007 12:37:16 -0700 (PDT), Beau Hargis <beau(a)subobscur.us> wrote: >Well, it subclasses 'protocol.ClientFactory'. I did not try 'ReconnectingClientFactory' yet as I wanted a little bit more control of the reconnecting process at some point. In any case, these functions are defined in the class as such: > > def clientConnectionFailed(self, connector, reason): > print 'Connection failed. Reason: ', reason > reactor.callLater(random.randint(10,91), connector.connect) > > def clientConnectionLost(self, connector, reason): > print 'Connection Lost. Reason: ', reason > reactor.callLater(random.randint(10,91), connector.connect) > >Works ok for every other problem except Connection Refused. When I first started testing reconnect, the seconds argument to callLater was constant like 3 or 4 seconds and it did the same thing. Even immediate reconnection caused the same problem with Connection Refused: 111. This looks as though it should work. If I run a similar test program, like this: from twisted.internet.protocol import ClientFactory, Protocol from twisted.internet import reactor class Reconnecter(ClientFactory): def clientConnectionFailed(self, connector, reason): print reason reactor.callLater(5, connector.connect) reactor.connectTCP('localhost', 12345, Reconnecter()) reactor.run() where nothing is bound to localhost:12345, then I see a connection refused message every five seconds for as long as I allow it to run. Can you see if this simple program exhibits the same behavior as your application? It would also be worth determining if your application is receiving the clientConnectionFailed notification but then failing to attempt another connection (ie, some problem with the callLater), if it is attempting the next connection but that is never happening (ie, some problem with the connector.connect), or if clientConnectionFailed isn't even being called for the connection which is failing (some problem inside the reactor with failed connection detection/notification). If you can, it would also be useful to know what the operating system thinks the state of the socket used for the connection attempt is. On Linux, you can get this with a tool like netstat. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1

Re: [Twisted-Python] Properly handling Connection Refused
by Beau Hargis 05 Sep '07

05 Sep '07

Well, it subclasses 'protocol.ClientFactory'. I did not try 'ReconnectingClientFactory' yet as I wanted a little bit more control of the reconnecting process at some point. In any case, these functions are defined in the class as such: def clientConnectionFailed(self, connector, reason): print 'Connection failed. Reason: ', reason reactor.callLater(random.randint(10,91), connector.connect) def clientConnectionLost(self, connector, reason): print 'Connection Lost. Reason: ', reason reactor.callLater(random.randint(10,91), connector.connect) Works ok for every other problem except Connection Refused. When I first started testing reconnect, the seconds argument to callLater was constant like 3 or 4 seconds and it did the same thing. Even immediate reconnection caused the same problem with Connection Refused: 111. ----- Original Message ---- From: Jean-Paul Calderone <exarkun(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Wednesday, September 5, 2007 12:29:15 PM Subject: Re: [Twisted-Python] Properly handling Connection Refused On Wed, 5 Sep 2007 12:24:35 -0700 (PDT), Beau Hargis <beau(a)subobscur.us> wrote: >I have this application built in twisted that is pretty robust save for one thing. The server reconnects when it falters or fails for any reason EXCEPT a refused connection, which happens when connecting to services on the Windows servers we have where particular services have not yet been started, which is normal. Every other failure is handled gracefully, including the database disappearing suddenly. I print out the reason in the log and I get this error: > >[Uninitialized] Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused. >[Uninitialized] ] >[Uninitialized] Stopping factory <SomeFactory instance at 0x88b77ac> > >And the whole thing stops processing and does not restart the connection. How did you implement reconnection? Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1

[Twisted-Python] Properly handling Connection Refused
by Beau Hargis 05 Sep '07

05 Sep '07

I have this application built in twisted that is pretty robust save for one thing. The server reconnects when it falters or fails for any reason EXCEPT a refused connection, which happens when connecting to services on the Windows servers we have where particular services have not yet been started, which is normal. Every other failure is handled gracefully, including the database disappearing suddenly. I print out the reason in the log and I get this error: [Uninitialized] Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused. [Uninitialized] ] [Uninitialized] Stopping factory <SomeFactory instance at 0x88b77ac> And the whole thing stops processing and does not restart the connection. I am running on Ubuntu 7.04 with the twisted packages therein: python-twisted 2.5.0-0ubuntu3 Is this a bug that others have seen? Should I use the tarball from twisted Matrix or use the gutsy packages?

2 1

[Twisted-Python] Re: Tell us how you're using Twisted
by Tomas Neme 05 Sep '07

05 Sep '07

We use twisted to make the networking platform for a number of MMO games that our client will be releasing sometime next year Tomas -- |_|0|_| |_|_|0| |0|0|0| (\__/) (='.'=)This is Bunny. Copy and paste bunny (")_(") to help him gain world domination.

1 0

[Twisted-Python] Win32 IOCPReactor?
by Trent Nelson 04 Sep '07

04 Sep '07

I'm new to Twisted. Perusing the documentation, I came across IOCPReactor. It's marked as 'extremely experimental', and it appears it only provides support for TCP events. It seems the last check in to anything IOCPReactor-related was back in 2005. Is it dead? Is anyone using it? Is anyone planning to pick development back up? Were there specific issues with IOCP that impeded development, or just a general lack of people with time, inclination, or Win32-fu? Assuming the IOCPReactor's not feasible for high load production deployment, can anyone comment on how the default select()-based reactor for Win32 fares under high load? Trent.

19 31

[Twisted-Python] Wrapping a Perspective Broker remote object with a blocking API
by Rasjid Wilcox 04 Sep '07

04 Sep '07

Hi everyone. I've done some reading, and it seems that there have been several attempts by new twisted users to get Perspective Broker to function more like Corba or DCOM. Despite having read http://twistedmatrix.com/pipermail/twisted-python/2007-April/thread.html#15… on why this does not work, I have had a go and have got something that seems to work well, at least for my test case. Server side it is pretty standard. Client side I run the twisted reactor in a separate (non-main) thread, and use threading.Event() objects placed into a 'pass through' callback to emulate blocking until the required deferred has its result. I have attached a simple server and client, with callback functionality illustrated. I have not really looked into the thread safety issues yet, but have been looking into reactor.callFromThread and whether it is required here. If the entire approach is inherently broken for some reason, I'd like to know so I can look at alternatives. Cheers, Rasjid.

3 8

[Twisted-Python] Media Resource Control Protocol (MRCP)
by Ivan Uemlianin 04 Sep '07

04 Sep '07

Dear All I am interested in writing an MRCP (Media Resource Control Protocol [1]) server. In particular I want to write an MRCP speech recognition resource server as part of my trefnydd [2] project. I am posting here to ask for advice about the network programming part. I shall also be writing an MRCP client (if only to test the server). All the code will be GPL'd. As far as I understand it, an overview of the networking would go like this: An MRCP session between client and server is initiated using the Session Initiation Protocol (SIP). Client and server negotiate using SIP messages containing SDP message bodies to define the participants and the resources required in the session. After session initiation, two channels are launched: a media channel using RTP to carry binary audio (i.e., speech) data, and a control channel using TCP (control channel messages are text-based and in MRCP format). It seems to me that both client and server software would have to cope with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the SIP/SDP and MRCP control channel messages). Here are my questions: - Does anyone know of a FOSS project to write an MRCP server? preferably in python. I know of mrcp4j [3]. - My first plan would be to make the MRCP client/server a subclass of an SIP client/server, as SIP is used to set up (and tear down) the session. Does that sound sensible? - Can twisted.protocols.sip be used to write a SIP server? Would it give me everything I need, or would I also need something like pjsip [4] or jain-sip [5] for the low-level stuff? - Have I missed anything? Any advice would be gratefully received. I'll report back with progress. Best wishes Ivan [1] MRCP: http://www.llaisdy.com/static/tech/mrcp/ [2] trefnydd: http://www.llaisdy.com/static/trefnydd/ [3] mrcp4j: "MRCP4J provides a Java API that encapsulates the MRCPv2 protocol and can be used to implement MRCPv2 clients and/or servers." http://mrcp4j.sourceforge.net/ [4] pjsip: http://www.pjsip.org/ [5] jain-sip: https://jain-sip.dev.java.net/

6 10

[Twisted-Python] Processing reload event
by Pavel Bastov 04 Sep '07

04 Sep '07

Hi all, how do I process the reload event (service <myservice> reload) in my Twisted application? TIA -- Good luck, Pavel Bastov xooChat Evangelist and Team Leader http://www.xoochat.com/

2 2

Re: [Twisted-Python] SMS Protocol UCP /EMI
by Pascal Bach 04 Sep '07

04 Sep '07

I think it is best if I publish my UCP/EMI implementation somewhere, I'm currently thinking about a common API for different SMSC communication API. I currently know about SMPP, UCP/EMI and CIMD from Nokia. Is there any other protocol, maybe with some specialties that should be considered in the Interface. Is there actually an interest in such protocols in twisted? Pascal

3 2

[Twisted-Python] Foolscap-0.1.6 released
by Brian Warner 02 Sep '07

02 Sep '07

I've released Foolscap-0.1.6, with a few minor bugfixes. The new release can be downloaded from PyPI or directly from the Foolscap home page: http://foolscap.lothar.com/releases/foolscap-0.1.6.tar.gz This release fixes the debian packaging problems that prevented 0.1.5 from being able to create .deb packages. It also fixes a problem with the ChoiceOf constraint, and fixes a couple of problems in the documentation. Foolscap versions 0.1.3 through 0.1.6 (inclusive) are fully wire-compatible with each other. Wire-compatibility-breaking changes are still anticipated in the future. Foolscap is a next-generation RPC/RMI/remote-object protocol, intended to eventually replace Perspective Broker. It offers remote object-capability facilities for Python programs using Twisted's event loop, based upon concepts and patterns from the E language (http://www.erights.org/). For more information and bug reporting, please visit the Foolscap home page at: http://foolscap.lothar.com/ cheers, -Brian

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 01 Sep '07

01 Sep '07

Bug summary ______________________ Summary for 2007-08-26 through 2007-09-02 Bugs opened: 7 Bugs closed: 3 Total open bugs: 1024 (+4) |== Type Changes |== Priority Changes |== Component Changes |Defect: +5 |Highest: +2 |Core: +4 |Enhancement: +0 |High: +2 |Names: +1 |Normal: +1 |Pydoctor: -1 |Web: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2808] AMP should raise MissingArgument (or other) if a callRemote is called with wrong arguments (opened by Lacrymology) defect core http://twistedmatrix.com/trac/ticket/2808 [#2808] AMP should raise MissingArgument (or other) if a callRemote is called with wrong arguments (opened by Lacrymology) defect core http://twistedmatrix.com/trac/ticket/2808 ===== High ===== [#2807] twisted.mail.relaymanager.MXCalculator._cbMX raises unhandled IndexError for response with no answers (opened by exarkun) defect names http://twistedmatrix.com/trac/ticket/2807 [#2809] 99.9% cpu load when using the epoll reactor (opened by mvw) defect core http://twistedmatrix.com/trac/ticket/2809 ===== Normal ===== [#2805] Cooperator adds a callback to iterated deferreds which does not pass on the original result (opened by rwall) defect core http://twistedmatrix.com/trac/ticket/2805 [#2806] Twisted web client should support HTTPS URL hostname verification (opened by radix) enhancement web http://twistedmatrix.com/trac/ticket/2806 [#2810] AMP command-handling and protocol-parsing should be on separate objects (opened by glyph) enhancement core http://twistedmatrix.com/trac/ticket/2810 Closed Bugs ______________________ ===== Highest ===== [#2808] AMP should raise MissingArgument (or other) if a callRemote is called with wrong arguments (opened by Lacrymology, closed by Lacrymology, ) defect core http://twistedmatrix.com/trac/ticket/2808 ===== Normal ===== [#1639] Full package and class hierarchy pages. (opened by jknight, closed by exarkun, fixed) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1639 [#2473] make twistd more usable (opened by itamarst, closed by glyph, invalid) enhancement core http://twistedmatrix.com/trac/ticket/2473 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2810] AMP command-handling and protocol-parsing should be on separate objects (since 2007-08-31 14:21:32). Mean open ticket age: 511 days, 15:26:23.405046. Median: 406 days, 14:01:43.587664. Standard deviation: 398 days, 11:41:57.222639. Interquartile range: 496 days, 3:03:16. Mean time between ticket creation and ticket resolution: 120 days, 15:02:59.593481. Median: 14 days, 8:13:35. Standard deviation is 224 days, 10:25:04.257292. The interquartile range is 127 days, 2:25:56. Mean time spent in review: 63 days, 14:54:37.734422. Median: 4 days, 4:50:20. Standard deviation: 138 days, 15:21:07.206937. Interquartile range: 30 days, 7:05:01. Mean number of times a ticket is reviewed: 1.91366906475. Median: 1 Standard deviation: 1.55081172087. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 19 unique ticket reporters 7 unique ticket reviewers 6 unique ticket resolvers In the last 24 weeks, 80 unique ticket reporters 14 unique ticket reviewers 16 unique ticket resolvers In the last 48 weeks, 155 unique ticket reporters 20 unique ticket reviewers 27 unique ticket resolvers

1 0

Re: [Twisted-Python] Possible to delay connectTCP?
by Beau Hargis 01 Sep '07

01 Sep '07

That is what I had thought. I had come up with a couple of possible options, some of them complex, but I wanted to make sure there wasn't anything hiding in twisted that was not obvious. Thanks. ----- Original Message ---- From: Christopher Armstrong <radix(a)twistedmatrix.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Friday, August 31, 2007 4:56:02 PM Subject: Re: [Twisted-Python] Possible to delay connectTCP? On 8/31/07, Beau Hargis <beau(a)subobscur.us> wrote: I have an application which connects to a machine which has 50-60 ports open on it to receive data (each for a different region). It acts like a switch. Problem is that when I make all those connection requests at the same time the server falls over trying to handle all the connections requests from my application. I cannot control or change the behaviour of the server I am connecting to, however, I am also unable to find a way to easily to add a delay between connection requests in twisted (without blocking the reactor), so that only 2 or 3 start up at a time. Is there an elegant and clean way to do such a thing? Twisted has worked beautifully for everything else, but I can't replace this one server for a while until I can duplicate the rest of its functionality, so I am dependant on it providing the data. The way to delay (that is, schedule) function calls until a later point in time is reactor.callLater(seconds, callable, *args, **kwargs). -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

1 0