Mailman 3 November 2009 - Twisted

[Twisted-Python] ProducerConsumerProxy problem
by Pietro Niccoli 04 Nov '09

04 Nov '09

Hi, I'm trying to write a simple echo server but with control flow. The program should send back all the data it receives from a client, but if one of the clients that are connected stop accepting the data that is sent back, my program should tell it to stop sending data too, for preventing to run out of memory. I've seen that twisted offers an iConsumer and iProducer classes, so my class should implement both of them. I know how to write the producer part, but I still having trouble in telling twisted that my class is a consumer for the transport. This is because usually in a program, whenever there is data available, the dataRecived function of the program is called, passing the data to it, so I don't know who I can pass as a producer for my echo class. I know that there's a class called BasicProducerConsumerProxy in pcp that seems to do my job, but I'm stuck in making it understand that "transport" is both a consumer and a producer. The code should be something like this: Echo = protocols.pcp.BasicProducerConsumerProxy(self.transport) #register transport as a consumer Echo.registerProducer(self.transport, True) #register transport as a producer But it doesn't work. Can you help me? An example will be greatly appreciated. Thanks. Pietro

1 0

[Twisted-Python] Unpedictable behaviour of transport.write call: Problem with return the control to reactor
by naman jain 04 Nov '09

04 Nov '09

Hi, I was trying this out: Server code: def dataReceived(self, data): if (args["cmd"] == "CHECK_IN"): foo = subprocess.Popen('python master.py',shell = True) *# this ACK notifies the client to close the connection* self.transport.write( self.create_message("CHECK_IN","ACK") ) I use the subprocess call , as I want to avoid threads here, and independently launch my master.py without blocking this code. I dont aim to have any interaction with the master.py once it is launched Now, I see unpredictable behaviour here.. I have been trying many times, but the subprocess call does not happen everytime ! The connection is closed by the client without the master getting launched in some cases. How does the transport.write behave in such cases.. Do you people see a structural problem here.. Or adding some small delay of 2 sec before transport.write might help? Regards Naman

2 1

[Twisted-Python] Connect to an IRC server, modules.
by Alberto Piu 04 Nov '09

04 Nov '09

Greetings, first of all, I apologize for bad english. I am new with Twisted, and although it seems very powerful, I'm having some problems in using it. I'm trying to create an IRC bot, which basically does nothing. My idea is to write a skeleton that's improved by modules, which do different things in an IRC channel. When some events happen some modules become active, so that they can send messages, or notices, or private messages, and all what an IRC bot can do; so, every module has to have complete access to the methods of the bot class, but in the same time I need to keep the two things separated, to improve modularity. I used an example, called LogBot example. I have a .py file (I'll call it "mainfile), in which the LogBot class is located, where the reactor is set up etc, then I have a dir called modules, where modules are. - mainfile.py - modules - modulea - modulea.py - moduleb - moduleb.py (I omitted __init__.py files). Now the question is: how I can make accessible all the methods of the LogBot class from EVERY .py inside the modules directory? I tried to do the following: In the mainfile, inside the LogBot class, I have def joined(self, channel): """This will get called when the bot joins the channel.""" self.logger.log("[I have joined %s]" % channel) This method is called when the bot joins the channel "channel". Know, if I want to use modulea to greet user "x" when I enter the channel, I would write: def joined(self, channel): """This will get called when the bot joins the channel.""" self.logger.log("[I have joined %s]" % channel) modulea.greet("x", "Hi, user x") where greet is a function inside the module modulea (imported somewhere in my mainfile). Anyway, modulea is defined as following: def greet(who, message): ## But here, how can the function know the method to use? The solution can be... ##In mainfile.py def joined(self, channel): """This will get called when the bot joins the channel.""" self.logger.log("[I have joined %s]" % channel) modulea.greet(self, "x", "Hi, user x") ##In module a: def greet(bot, who, message): bot.msg(who, message) But this does not work. I tried to explain as better as I could. If you can, please use examples and speak easy (can't understand very well). Hope you can help me! Thank you, in advance. --Alberto Piu

2 1

[Twisted-Python] twisted.internet.error.ProcessTerminated, exit code 1?
by Galina Kistanova 03 Nov '09

03 Nov '09

Hello all, I ran into issue with Twisted, could anybody help me? I have Twisted 8.2.0 on Ubuntu 9.04. I run tests for Buildbot 0.7.11p3 and one of the tests completed with the next error: [ERROR]: buildbot.test.test_locks.Locks.testLock3 Traceback (most recent call last): Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1. Cannot find any information what could cause this. What “exit code 1 “ means? I tried to stop MsAfee, but this did not help. Thanks Gal

1 0

Re: [Twisted-Python] The case of the missing Traceback
by Terry Jones 03 Nov '09

03 Nov '09

>>>>> "Dan" == Dan <tw.1(a)pyxos.com> writes: Dan> I was playing around with Twisted today and ran into a problem where Dan> Tracebacks aren't emitted on stderr after a deferred callback from a Dan> process called launched by reactor.spawnProcess. The simplified test Dan> case is below. I put "x = y" in the callback function to generate an Dan> exception, y doesn't exist. Uncomment "x = z" right before the Dan> callback and it produces the traceback on stderr just fine. Hi Dan The callback function raises an exception, which Twisted converts to failure and stores in the deferred. But seeing as there are no errback functions in the deferred *and you leave the reactor running*, the failure just sits there (just as it should). If you do a 'from twisted.python import log' and Change the addCallback line in test() to look like this d.addCallback(response).addErrback(log.err) You'll see the expected NameError just fine. Terry

2 1

[Twisted-Python] The case of the missing Traceback
by Dan 03 Nov '09

03 Nov '09

I was playing around with Twisted today and ran into a problem where Tracebacks aren't emitted on stderr after a deferred callback from a process called launched by reactor.spawnProcess. The simplified test case is below. I put "x = y" in the callback function to generate an exception, y doesn't exist. Uncomment "x = z" right before the callback and it produces the traceback on stderr just fine. System is Linux, Python 2.6 and Twisted 8.2. from twisted.internet import reactor, protocol, defer import sys def err(s): sys.stderr.write(s+'\n') sys.stderr.flush() def test(): err('test') cat = MyPP() reactor.spawnProcess(cat, "/bin/cat", ["cat", "-"], {}) d = cat.send("Hello world") d.addCallback(response) def response(resp): err('response called') x = y err(resp) class MyPP(protocol.ProcessProtocol): def __init__(self): self.defer = defer.Deferred() def outReceived(self, data): err("outReceived! with %d bytes" % len(data)) err(" %s" % data) self.defer.callback(data) def send(self, c): self.transport.write("%s\n" % c) #x = z return self.defer if(__name__=='__main__'): reactor.callWhenRunning(test) reactor.run()

1 0

[Twisted-Python] reverse web proxy & threads
by Artem Bokhan 02 Nov '09

02 Nov '09

Hello! I'm playing with reverse proxy from examples (Twisted-8.2.0/doc/web/examples/proxy.py). May somebody explain what part of code uses threads there? pstree -p | grep python -python(12786)-+-{python}(12795) |-{python}(13050) |-{python}(15547) `-{python}(20767)

2 1

[Twisted-Python] inlineCallbacks
by Pet 02 Nov '09

02 Nov '09

Hi, calling in client code self.cred.login() I can't print login result. What I'm doing wrong? login method returns deffered, but should yield result from callRemote method. Thanks for any help! Pet class Cred: @inlineCallbacks def login(self): proxy = Proxy(LOGIN_PROXY) l = {"user":LOGIN_USERNAME, "pass":LOGIN_PASSWD} loginResult = yield proxy.callRemote('login', l) print "RESULT", loginResult.result returnValue(loginResult)

7 23

Re: [Twisted-Python] Using a custom reactor in twisted trial for test cases?
by Andrew Francis 02 Nov '09

02 Nov '09

Hello Glyph: >These examples aren't the greatest, because they tend to assume that >you _always_ have Stackless code that's ready to run, and you want to >run at some number of "frames per second". In 2007, folks (especially myself) are still trying to figure out if Stackless can play with Twisted correctly. I know my stuff in the repository is old. > For simple examples this makes sense, but for a system architecture >this is a limiting approach. If your stackless application wants to >sit idle and wait for input, you're still going to wake up once every >1/30 second and checking to see if there's anything to do, burning CPU >cycles and battery life. An approach would be for the reactor tasklet to look at the runnable list to see if there are any entries. If there are no entries (i.e., tasklets are blocked and dependent on the reactor tasklet), then the reactor tasklet can stop the loopingCall. This is easy enough to do. However I suspect one starts to get into custom reactor land. >Also, if you want to run *faster* than the arbitrary timeout you've >selected (1/30 second can be a very long time, especially if you're >doing something pseudo-realtime, like audio playback) you're out of luck. I believe there are two scenarios here: 1) no blocked tasklets and only runnable ones. If the runnable tasklets make no IO requests, it should be possible to put the reactor tasklet to sleep. Let the system run as fast as it can. 2) There is a mix of runnable (scheduled) and blocked tasklets. This scenario requires a more sophisticated scheduler that is trying to compute an optimal loopingCall value. This scenario is tricky. >Better would be to have tasklets schedule *themselves* for when they >want to run. In the Stackless world, tasklets start running from the moment they are created/inserted into the runnable list. Stackless in non-preemptive mode is more about knowing when to make a tasklet yield to the scheduler. Cheers, Andrew

1 0

Re: [Twisted-Python] Using a custom reactor in twisted trial for test cases?
by Andrew Francis 02 Nov '09

02 Nov '09

Crispin: >OK. Having a closer look, its not that looping call doesn't work, its >that there is some unknown number of "reactor pumps" between starting >the test, and finishing it. What I need is a way for the reactor to be >pumping away while a particular test function of a testcase continues >working. As an example, here is a non-working test case. Notice the >comment "#what to do here to pump the reactor?". ... > while task.alive: > pass #what to do here to pump the reactor? This won't work because as long as the tasklet is alive (scheduled, paused, or blocked), you will be in a tight loop. And if that function is not running in the reactor, then the reactor will starve. If you are waiting to see if the tasklet finished, write a synchronizer. Cheers, Andrew

1 0