Mailman 3 December 2008 - Twisted

[Twisted-Python] Pyjamas + web2py + twisted vs Django + Pinax
by Theron D. Huffman Dec. 19, 2008

Dec. 19, 2008

I'm about to start programming a game in Python. The game will be a web-based board game with a social aspect, something like http://www.chesspark.com. I want the game to run in both browsers and as a desktop client. Therefore I'm wondering what python-based technologies I need. I know that Javascript/AJAX is necessary for the browser part so I have decided on the following: 1. Pyjamas - (for the AJAX browser stuff) 2. Pyjamas-desktop - (for the desktop client) 3. Web2py (or djanjo, or … [View More]

3 2

[Twisted-Python] python properties in twisted don't work
by Gabriel Rossetti Dec. 18, 2008

Dec. 18, 2008

Hello everyone! I have a problem, if I try to use python property in a twisted program, it doesn't really work...the accessor works but as soon as I use the mutator, it no longer uses the property (and doesn't set the "real" variable. I tried an example without twisted, it works, and with the twisted example it doesn't... does anyone know what is going on? Thank you! Gabriel #------------------------------------------------------------------------------------------------------- # This … [View More]doesn't work, why???? #------------------------------------------------------------------------------------------------------- from twisted.internet.protocol import Protocol, ClientFactory from sys import stdout class Echo(Protocol): def __init__(self): self.__x = None def __getx(self): return self.__x def __setx(self, value): if(value != 0): self.__x = value def __delx(self): del self.__x x = property(__getx, __setx, __delx, "the doc") def connectionMade(self): t = self.x assert t is None self.x = 4 assert self.__x == 4 t = self.x assert t == 4 self.x = 0 assert self.__x == 4 t = self.x assert t == 4 def dataReceived(self, data): stdout.write(data) class EchoClientFactory(ClientFactory): def startedConnecting(self, connector): print 'Started to connect.' def buildProtocol(self, addr): print 'Connected.' return Echo() def clientConnectionLost(self, connector, reason): print 'Lost connection. Reason:', reason def clientConnectionFailed(self, connector, reason): print 'Connection failed. Reason:', reason if(__name__ == "__main__"): from twisted.internet import reactor reactor.connectTCP("localhost", 4444, EchoClientFactory()) reactor.run() #------------------------------------------------------------------------------------------------------- # This works, as expected #---------------------------------------------------------------------------------------------------- class C(object): def __init__(self): self._x = None def __getx(self): return self._x def __setx(self, value): if(value != 0): self._x = value def __delx(self): del self._x x = property(__getx, __setx, __delx, "The doc") if(__name__ == "__main__"): c = C() t = c.x assert t is None c.x = 4 assert c._x == 4 t = c.x assert t == 4 c.x = 0 assert c._x == 4 t = c.x assert t == 4 [View Less]

6 6

[Twisted-Python] SSH Server with commands filtering
by Tzury Bar Yochay Dec. 18, 2008

Dec. 18, 2008

Hi, I am willing to deploy on my server an SSH daemon which I can control what commands can be executed, and by who. After playing with jailkit [link] I realised that implementing my own SSH server will give me more power and scalability (especially regarding of users, keys and accounts management). Naturally I approached twisted knowing it will solve this problem as a smart kiddo solves a simple safari puzzle. Playing with samples found on the web and at Twisted docs, I realised that twisted … [View More]

2 4

[Twisted-Python] XmlStream and addObserver - root element's name?
by Marcin Gliński Dec. 18, 2008

Dec. 18, 2008

Hello! There's a problem I can't find an answer to. I'd like to design Xml based protocol. I've found XmlStream, read doc and found addObserver method. I wrote simple code: class XmlTest(xmlstream.XmlStream): # (...) def connectionMade(self): xmlstream.XmlStream.connectionMade(self) self.addObserver("/frame", self.frameHandler) # (...) # (...) def frameHandler(self, data): print "Wilma, I'm Home!" When I send XML with frame element as a root: … [View More]

3 4

[Twisted-Python] Syslog Listener UDP
by Marc Byrd Dec. 16, 2008

Dec. 16, 2008

Hi, I would like to build a syslog(udp port 514) listener in twisted. The goal is to make it compatible with syslog and syslog-ng, which may be configured on a "client" to send log entries (as they occur) via udp (fire and forget) to a remote server (my twisted server). Is there already something out there that would do this or that would be a good start? Thanks, Marc

4 4

[Twisted-Python] twisted eat the memory
by Muaaz Hussain Dec. 15, 2008

Dec. 15, 2008

I wrote web application on twisted and my application read from ulogd daemon and push the output to the client browser using AJAX .also I have another application displaying ethernets traffic using AJAX. but after some minutes the twisted process in the server eat the memory and the OOM-killer of the linux will kill the twisted process. are there any solution to make the twisted release the memory . note: my ram is 256mb -- Muaaz Osman Hussain IT Researcher

4 3

[Twisted-Python] Re: Twisted FTP
by Adriano Marques Dec. 15, 2008

Dec. 15, 2008

Hello Gergely, > I just started to learn twisted. I tried to look after a working FTP example > on the net for twisted, but unfortunately I could not really found any. This > one seemed to be the most completed, but it says "Unknown error code" when I > try to connect to it. I'm sorry, but I don't have the chance of looking it further right now, but I took a look at your code, and I say that instead of making FTPServerFactory inherit from FtpFactory, you opted to instantiate it … [View More]

1 0

[Twisted-Python] Twisted and MVC
by Christian Scharkus Dec. 15, 2008

Dec. 15, 2008

Hi folks. I want to build a MVC with the Model based on twisted with a custom protocol. At the moment all my attemps terrible failed. So far the client build a valid ssl-connection to the server and send the lines he is supposed to send but don't recognise any incomming lines from the server. So I decided to build the model from scratch again. The controller inits the model-class (implementation of the custom chat-protocol) to be able to call methods of the instance and send messages to the … [View More]

2 2

[Twisted-Python] Non-blocking http client?
by Erik Wickstrom Dec. 14, 2008

Dec. 14, 2008

Hi all, I have an application that is doing some web spidering. Right now I'm using urllib to retrieve the URLs, but it is painfully slow. I was wondering if it's feasible to swap out urllib with a twisted client that uses deferds so I can process urls in a more "parallel" fashion? I've done a bunch of Googleing, but I haven't come across anything that I can use as a drop in replacement. If you can point me in the right direction I'd really appreciate it! Thanks for your help! Erik

4 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Dec. 14, 2008

Dec. 14, 2008

1 0

[Twisted-Python] twisted.mail.relaymanager.MXCalculator.getMX in Win32 got problem
by biziap biziap Dec. 13, 2008

Dec. 13, 2008

The problem is that: the UDP listening ports increase rapidly and eventually come to the end: too many open file in selector. As many as the hosts you lookup, those UDP ports are still alive. It is very different from running this script in Linux (gentoo 2.6). In Linux, the listening UDP port keep only 1 instance no matter how many lookups . Try the script below: ------------------------------ from twisted.mail import relaymanager from twisted.internet import reactor,defer global MXC MXC = … [View More]

2 1

[Twisted-Python] PDF I thought may be of interest
by Michael Dec. 13, 2008

Dec. 13, 2008

Hi, I came across this livejournal entry today: * http://four.livejournal.com/934085.html Which has this link: http://s3.amazonaws.com/four.livejournal/20081209/threads-hotos-2003.pdf .. which is a paper by Rob von Behren, Jeremy Condit and Eric Brewer. I personally raised an eyebrow when I saw that because Eric Brewer was the CTO of Inktomi, who back the had a very highly scaleable network system called Traffic Server amongst other things (I worked at Inktomi between 2000 & 2002, … [View More]

4 3

[Twisted-Python] Re: PDF I thought may be of interest
by Andrew Francis Dec. 11, 2008

Dec. 11, 2008

Hi Michael and Colleagues: > .. which is a paper by Rob von Behren, Jeremy Condit and > Eric Brewer. As a part of writing "Adventures in Stackless Python/Twisted Integration," I have read "The Why Events are" paper as well as the Ousterhout "Why Threads Are A Bad Idea (for most purposes). The "Why Events" paper led me to the Adya et al, "Cooperative task management without manual stack management" and the original Lauer and Needham "On the duality of operating system structures." I … [View More]also visited the Schmitt papers on the Reactor pattern. >From what I recalled, I preferred the latter three papers because they provide a much better analytic framework for describing the issues. In particular, the Adya provides terminology. I have two basic sets of take-aways from the paper survey 1) "Events are great papers" state that threads are bad because of A,B, and C. "Threads are great papers" state that events are bad because of X,Y, and Z. As a programmer using these systems, perhaps I should pay special attention to issues A,B,C,X,Y and Z whenever applicable (and by the way papers thanks for telling me C and Y were issues since I didn't know). Perhaps out of all this, we can come up with appropriate guidelines for lockless programming? 2) The "Duality" paper argues that messaging passing and procedural systems can be described in terms of each other (bi-simulation?). I found this to be a very powerful way to look at Stackless Python and Twisted's threading models. A simple example. At the time I presented "Adventures," to avoid deadlock, I ran a reactor protocol instance in its own Stackless thread. This bugged me. It turned out that by replacing channels with deferreds (and doing a little buffering), I could replace the extraneous tasklet. My way of looking at it was, after all, protocol instances/deferreds are just another way of doing continuations... Use them, don't replace them.... > Looking slightly below the surface of things, it appears to > be advocating something more like stackless's tasklets, rather than > OS level threads (since it makes reference near the end to Erlang's >threading model). A quote from the paper: "To evaluate the ability of threads to support high concurrency, we designed and implemented a simple (5000 line) user-level cooperative threading package for Linux. Our thread package uses the coro coroutine library [15] for minimalist context switching, and it translates blocking I/O requests to asynchronous requests internally." > The paper appears to be 5 years old already, so the impact > of this paper appears minimal, but I thought it may be of interest to > others. (The reason it raised such a surprise for me is because it >would be as surprising to me as it would be to read of any of the core >twisted devs to suddenly come out in favour of a threading approach >rather than non-threading!) "The Why Events" is essentially the approach I took in regards to Stackless and Twisted. Right now, I am analyzing Stackless's scheduling model to see if improvements can be made. I feel I have much to learn... That said, I feel inline callbacks are a step in this direction. As Stackless/PyPy concepts get into mainstream Python, I can see Twisted following in this direction. Already one can see that with stuff like Christopher Armstrong's Corotwine. Again, I think the key to incorporate threads and keep to the spirit of Twisted, simplicity and safety, is to come up with some guidelines for lockless programming. Cheers, Andrew [View Less]

1 0

[Twisted-Python] Running commands / collecting datas via SSH on many machines: Twisted vs plain old scripts ?
by Alessio Pace Dec. 11, 2008

Dec. 11, 2008

Hi, I have to run commands and (later in time) collect logs from many machines, connecting through SSH. I was _genuinely_ wondering what an SSH client written in Twisted could give me more than executing plain old shell scripts in a for loop. >From what I saw, given the fact that my run/collect commands remain still shell commands (cp, tar, etc...), the only thing that may change is that using Twisted I could have a better utilization of resources rather than a simple shell for loop in … [View More]

2 2

[Twisted-Python] twisted reverse proxy for web filtering
by dave fowler Dec. 11, 2008

Dec. 11, 2008

I'm making a simple app that will run a local proxy and redirect me away from distracting websites. For example, if I go to facebook.com I'd like it to instead send me to google.com I'd like to use python and in my research I think a twisted reverse proxy is what I want. I've done the examples for setting up a proxy, including one that counts the words on the pages you visit, but i cannot find anything on fetching the destination and re-directing. I've been sifting through the many classes … [View More]

2 1

[Twisted-Python] lost tracebacks?
by Alec Flett Dec. 10, 2008

Dec. 10, 2008

So I am constantly "losing" my tracebacks when my defer's callbacks throw exceptions that aren't caught. When I get an uncaught exception (caught via twisted.log.addObserver()) I am basically storing the Failure instance and then later, when it's safe for me to do so, calling fail.raiseException() - but the Failure's 'tb' property is None. I traced this down to the code in Deferred._runCallbacks() which does a try: / except: with:: self.result = failure.Failure() Seems like … [View More]

3 4

[Twisted-Python] spawnProcess help needed
by Phil Christensen Dec. 10, 2008

Dec. 10, 2008

Hello All- I'm running into an issue with a project I'm working on that involves coordinating a large number of external processes to encode audio into different formats, and do other bookkeeping. I seem to be having a problem when I use spawnProcess to call the md5sum binary. The processes spawned seem to stick around forever, until I kill the server. This doesn't appear to happen with any other binary. I start with the following abstract class: class DefaultProcessProtocol(… [View More]protocol.ProcessProtocol): def __init__(self, preserveOut=True, preserveErr=True): self.deferred = defer.Deferred() self.debug = False self.preserveOut = preserveOut self.out = '' self.preserveErr = preserveErr self.err = '' def connectionMade(self): if(self.debug): print '%s connection made' % self.__class__.__name__ def outReceived(self, data): if(self.debug): print '%s stdout: %s' % (self.__class__.__name__, data) if(self.preserveOut): self.out += data def errReceived(self, data): if(self.debug): print '%s stderr: %s' % (self.__class__.__name__, data) if(self.preserveErr): self.err += data def processEnded(self, status): if(isinstance(status.value, error.ProcessDone)): self.deferred.callback(self) else: if(self.out): print self.out if(self.err): print self.err self.deferred.errback(status.value) Then I made a subclass as follows: class MD5Checksum(DefaultProcessProtocol): @staticmethod def run(target, md5_path='/usr/bin/md5sum'): hasher = MD5Checksum(preserveOut=True) hasher.debug = True reactor.spawnProcess(hasher, md5_path, [md5_path, target]) return hasher.deferred def get_checksum(self): filehash = self.out if(filehash.find('=') == -1): filehash = [output.strip() for output in filehash.split(' ')][0] else: filehash = [output.strip() for output in filehash.split('=')][1] return filehash With the plan of executing it inside an inlineCallbacks-decorated function, like this: proto = yield process.MD5Checksum.run(filename) checksum = proto.get_checksum() Everything is basically working, except for the fact that the md5sum processes never go away. It does seem that processEnded is being called, since that's what issues the callback on the deferred returned by run(), but the process sticks around, as a non-zombie, non-defunct process that still appears to be using small amounts of CPU time. Also, once the checksum has been retrieved, killing the server doesn't cause any errors, unlike when a process is legitimately terminated by killing the server, which displays an error (since right now i'm not actually catching ProcessTerminated scenarios): 2008-12-09 15:17:48-0500 [-] Unhandled error in Deferred: 2008-12-09 15:17:48-0500 [-] Unhandled Error Traceback (most recent call last): Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended by signal 2. Any help in this matter would be appreciated... -phil [View Less]

3 3

[Twisted-Python] wxpython and xmlrpc via twisted
by Ian Jones Dec. 10, 2008

Dec. 10, 2008

I'm trying to create a taskbar app (in wxpython) to monitor job progress on a render farm. I'd like the taskbar app to have an xmlrpc server built in so that other applications can inform the taskbar app additional jobs to monitor. I found this recipe online which is a great example of the functionality I want: http://code.activestate.com/recipes/298985/ However when I run the following from xmlrpclib import * s = ServerProxy('http://localhost:7080') # Get access to xmlrpc server s.title('… [View More]Hallo xmlrpc app') I don't get a response (though the title in the window changes - so I know it got it) I do get this though: Traceback (most recent call last): File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\protocols\basic.py", line 239, in dataReceived return self.rawDataReceived(data) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\http.py", line 1117, in rawDataReceived self.allContentReceived() File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\http.py", line 1108, in allContentReceived req.requestReceived(command, path, version) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\http.py", line 626, in requestReceived self.process() --- <exception caught here> --- File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\server.py", line 150, in process self.render(resrc) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\server.py", line 157, in render body = resrc.render(self) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\resource.py", line 189, in render return m(request) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\xmlrpc.py", line 122, in render_POST function = self._getFunction(functionPath) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\xmlrpc.py", line 170, in _getFunction handler = self.getSubHandler(prefix) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\xmlrpc.py", line 107, in getSubHandler return self.subHandlers.get(prefix, None) exceptions.AttributeError: 'MyXMLRPCApp' object has no attribute 'subHandlers' Unhandled error in Deferred: Traceback (most recent call last): File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\resource.py", line 189, in render return m(request) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\xmlrpc.py", line 126, in render_POST defer.maybeDeferred(function, *args).addErrback( File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\internet\defer.py", line 195, in addCallback callbackKeywords=kw) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\internet\defer.py", line 186, in addCallbacks self._runCallbacks() --- <exception caught here> --- File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\internet\defer.py", line 328, in _runCallbacks self.result = callback(self.result, *args, **kw) File "d:\toolkit\sww\win32\python\2.4\lib\site-packages\twisted-8.1.0-py2.4-win32.egg\twisted\web\xmlrpc.py", line 144, in _cbRender allow_none=self.allowNone) exceptions.AttributeError: 'MyXMLRPCApp' object has no attribute 'allowNone' I'm assuming this is just a good example that is out of date. However I'm not sure how to update it so that it works. Can someone help me out? [View Less]

2 4

[Twisted-Python] Deferred question
by Gabriel Rossetti Dec. 10, 2008

Dec. 10, 2008

Hello everyone, I have a question concerning defers, I have a service that needs to query IO until it answers. The IO can be disconnected, connected but off, connected and on; depending on the state I get different exceptions. I wrote something like : def __requestId(self, _): d = self.callPlugin(PLUGIN_NAME) d.addCallback(self.__receivedAns) d.addErrback(self.__requestId) The plugin makes the request in a thread and the deferred is returned and called when either … [View More]

2 3

[Twisted-Python] How to see the number of received msgs?
by Gabriel Rossetti Dec. 10, 2008

Dec. 10, 2008

Hello eveyone, is there a way to see how many msgs have been received and are waiting to be processed by a client? Thank you, Gabriel

3 4

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Dec. 7, 2008

Dec. 7, 2008

1 0

Re: [Twisted-Python] Re: A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones Dec. 6, 2008

Dec. 6, 2008

Hi Nicola! >>>>> "Nicola" == Nicola Larosa <nico(a)teknico.net> writes: Nicola> Terry Jones wrote: >> BTW, I spent an interesting morning getting my head around *exactly* how >> inlineCallbacks does its thing... No summary of my thoughts could do it >> justice. I wonder how many people on the planet have been down that >> rabbit hole. Nicola> Not me, man, not me. I just use the bloody thing, and curse it when Nicola> it shortens tracebacks.… [View More] I know exactly what you're talking about, and I hated that too. But I dug into that and found out where the problem was: I didn't know Python well enough :-) I bet you're doing something like this: @inlineCallbacks def f(): # Whatever try: # Your code here except Exception, e: # Do some cleaning up, maybe write to the log, maybe call another # function that uses inlineCallbacks raise e or just @inlineCallbacks def f(): # Whatever try: # Your code here except Exception: # Do some cleaning up, maybe write to the log, maybe call another # function that uses inlineCallbacks raise The problem here is that you're not using the 3-argument form of raise. So you're either dropping information (with the raise e) or pulling out the last exception (with the no-arg raise). Consider the latter case. Suppose your cleanup code calls another function that uses inlineCallbacks. That function will eventually raise StopIteration or raise via calling defer.returnValue. So by the time control returns to your code, your raise (which implicitly uses sys.exc_info()) raises the wrong exception! So you get e.g., a StopIteration exception coming at you out of your inlineCallbacks function and you curse the thing :-) This applies if your cleanup code calls any iterator, or if you call something that try/except catches a KeyError, etc. So just use @inlineCallbacks def f(): # Whatever try: # Your code here except Exception: info = sys.exc_info() # Do some cleaning up, maybe write to the log, maybe call another # function that uses inlineCallbacks raise info[0], info[1], info[2] and it just works. Try it :-) You cant do anything nice like raise *info because Python's parser doesn't like that (AFAIR). I nearly posted about this to the list, but I figured it might help if I had more of a clue about Python itself, seeing as what I was doing was nothing to do with Twisted. >> I hope people don't mind my peanut gallery commentary. We're all in this >> together, after all. A little gallows humor occasionally seems in order. Nicola> Again, certainly not me. Go right ahead! ;-) Don't encourage me... Terry [View Less]

1 0

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones Dec. 6, 2008

Dec. 6, 2008

>>>>> "glyph" == glyph <glyph(a)divmod.com> writes: glyph> On 02:33 am, terry(a)jon.es wrote: >> Am I missing something here? glyph> As a matter of fact, yes :). OK, thanks, I get it. That should have been obvious. BTW, I spent an interesting morning getting my head around *exactly* how inlineCallbacks does its thing... No summary of my thoughts could do it justice. I wonder how many people on the planet have been down that rabbit hole. Was the Twisted … [View More]

2 1

[Twisted-Python] one-shot reactor?
by Alec Flett Dec. 5, 2008

Dec. 5, 2008

Hey folks - So I'm trying what seems to be a fairly unusual use of Twisted, but I'm hoping that someone out there has tried the same thing as me and can offer some pointers. Bear with me as I explain how we're set up. The issues I'm having are at the end.. We're running Pylons as our appserver, but almost all of our internal requests from the appserver are actually retrieved over HTTP - i.e. database requests, LOBs, etc... we also have one or two other proprietary connections that … [View More]would benefit from asynchronous TCP access.. For years I've heard people raving about twisted, and I hate threads, so I thought I'd give it a shot. So here's how this is working, at least in my early prototypes: Pylons is a WSGI environment, which means it needs a real callstack for each request. Each request is more or less its own single-threaded environment, and the thread is only making new requests to internal services, not listening on any ports. So really Twisted is a client here, not a server. For each HTTP request, I'm running a "private" twisted reactor that simply runs until it runs out of reads/writes/delayedCalls. My theory is this: in a single-threaded environment that is not listening for new connections. So what I've done is make twisted.internet.reactor into a threadlocal object with Paste's StackedObjectProxy. So far I have this mostly working, but I've hit a few stumbling blocks along the way: 1) it would be nice if the standard twisted reactors had an API for running in a one-off client mode - something like a reactor.runUntilExhausted(). I did write a kind of scheduler that does this for me (more on that below) 2) it's vaguely annoying that reactors aren't restartable - it means I have to destroy any reactor that's left around from the last request. Not a huge deal, but I'd much rather just create a single reactor that lives the life of the thread, and be able to call .run() / .stop() over and over. 3) I've attached my scheduler below. I hook it up with reactor.callLater(0, stop_when_complete). The problem I'm running into with this approach is that many APIs like getPage() set a 30 second timeout, and then cancel the timeout later when it successfully retrieves the page. My scheduler picks up the fact that there is a 30-second timeout, but because HTTPClientProtocol cancels the timeout, my scheduler isn't aware of that, and has already scheduled itself for 30 seconds into the future, so it can't call reactor.stop(). So instead, I wake up at most every 0.1 seconds - but that kind of defeats the point of the reactor blocking on select()/poll() if I have to keep waking up! Maybe there's a better approach? What I kind of want is a hook into the reactor's runUntilCurrent() so I just get notified right before the select()/poll(). I'm considering just subclassing the reactor to hook into this.. ? def stop_when_complete(reactor, running=False): all_pending = (reactor.getReaders() + reactor.getWriters() + reactor.getDelayedCalls()) # depending on the platform, the waker is probably in there, and # shouldn't count as a pending event if reactor.waker in all_pending: all_pending.remove(reactor.waker) # ok, are we done? if so, tell the reactor to stop after its next # iteration. if not all_pending: reactor.stop() return # if we got here, we need to basically wait again to see if # there's anything left. We get the timeout for the next event, so # that we don't wake any more than we would have (this is how # ReactorBase.mainLoop works) timeout = reactor.timeout() if timeout is None: timeout = 0.0 print "Sleeping for no more than %s seconds" % min(timeout, 0.1) reactor.callLater(min(timeout, 0.1), stop_when_complete, reactor, running=True) [View Less]

1 0

[Twisted-Python] OSCAR buddy list
by Andrey Kravchuk Dec. 5, 2008

Dec. 5, 2008

Hi. I am trying to implement an ICQ service using twisted OSCAR protocol. Everything works fine so far, but i can't find out how to handle the buddy list there. I can't find anything on this topic neither in documentation, nor in this list or the source code. Please advice, or if this is not implemented yet, please tell me where to look so i could make a patch. The main problem is if i connect to ICQ service with twisted, the user appears offline for other users. In order to see his status … [View More]

1 0