Mailman 3 April 2008 - Twisted

[Twisted-Python] win32 reactors
by James Mansion April 7, 2008

April 7, 2008

I'm just starting to evaluate Twisted, so apologies if these questions seem naive. It seems to me that the Twisted model for 'reactor' is actually not what you'd expect from the name (at least, if you have a background in other reactor implementations including ACE, basic select etc and Java NIO) since it a) handles queueing outgoing data for you and b) presents incoming data after completion of the IO So its a model that maps well to an AIO implementation so long as inputs from non-… [View More]

9 16

Re: [Twisted-Python] Stackless/Twisted integration again
by Andrew Francis April 6, 2008

April 6, 2008

Hi Simon and Colleages: >I note he recommends running a twisted server as a >tasklet. I do this so both the Twisted reactor and the Stackless scheduler can run without blocking each other.... The Twisted Reactor and the Stackless Schedule start their respective systems. Both the Stackless and Twisted run() method assume that the application has finished once the run() method returns. Here is simple example import stackless from twisted.internet import reactor def hello(): … [View More]print "hello mom" stackless.tasklet(hello)() reactor.run() stackless.run() This programme will hang. Why? The Twisted reactor is running in the main tasklet and returns only after it has been stopped (which is never in this example). Only after the reactor has finished, will the stackless scheduler start. Of course, this is not the behaviour we want. >factory = pb.PBClientFactory >reactor.connectTCP('127.0.0.1', 1234, Factory) ># Load other tasklets here >stackless.tasklet(reactor.iterate)(1) I have never used iterate(). I am having a hard time finding an example of iterate() in action. Perhaps I am wrong but I would assume iterate() is used in conjunction with something like reactor.interleave() so that something like a GUI can function properly. >From my understanding, your example would produce something because in the iteration followed by a one second delay, the client can actually complete a call. However this seems accidental. >factory = pb.PBClientFactory >reactor.connectTCP('127.0.0.1', 1234, Factory) ># Load other tasklets here >while 1: > stackless.tasklet(reactor.iterate)(1) > stackless.schedule() I don't understand iterate() well enough but I think this will accidentally work sometimes. I don't have time to run your programme but off hand, this is what I think is happening: 1) Reactor tasklet is created. Since other tasklets are ahead of it, it is not running. 2) stackless.schedule() kicks off the scheduling, yielding the main tasklet and allowing the next schedulable tasklet to run 3) Eventually the Twisted reactor runs. It does stuff. You get results. However I don't believe it will yield after that. >I want to make the reactor a tasklet so it is included >in the scheduler. Simon, stackless.tasklet(reactor.run)() will make the Twisted reactor into a tasklet. However you need a way to make it yield occasionally. That is why I use task.LoopingCall(stackless.schedule). One nice feature of this approach is that I do not have to alter the Reactor. I remember playing with PB a while ago. Here is the post: http://twistedmatrix.com/pipermail/twisted-python/2007-September/016000.html I am enclosing some sample code so you can play with it. The code may be a bit clunky because I was still learning. Heck I am still learning :-) Cheers, Andrew P.S - You may want to take out the 61 second sleep in the PBServer. And it was neat to actually meet Bruce Eckel at PyCon. ____________________________________________________________________________________ You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. http://tc.deals.yahoo.com/tc/blockbuster/text5.com [View Less]

1 0

[Twisted-Python] Deferreds and Tasklets
by Simon Pickles April 6, 2008

April 6, 2008

So I am still fiddling with interesting ways to combine stackless and twisted. Its fun, but I am probably naively hacking! (I always seem to want to use the deprecated code!- probably says a lot about me?) I'm using Perspective Broker to do remote calls to an app running Twisted-Stackless. PB is positively brilliant, btw. So say the a remote caller wants to get data from my app. It calls remote_GetData() in my PB app, which uses the actor model(each object has a channel.receive() running … [View More]in its own tasklet). remote_GetData() has to send a message thru a stackless.channel() to get the required data, then it must stackless.schedule() Now it waits for the result to arrive at its own channel.receive(), which then needs to be passed back as the return value of remote_GetData(). Is this possible? Perhaps like this: from twisted.spread import pb from twisted.internet import reactor import stackless dbRx = stackless.channel() pbRx = stackless.channel() class DataSource: def Listen(self): while 1: msg = dbrx.receive() # a channel will arrive here msg.send(42) stackless.schedule() def __init__(self): stackless.tasklet(self.Listen)() class pbBroker(pb.Referenceable): def remote_GetData(self): dbRx.send(pbRx) result = pbRx.receive() # return 42 to remote caller thru pb return result This isn't ideal since I don't want pbBroker to block during the receive. I'd like it to closer resemble the Listen loop of the first class, so other objects can talk to pbBroker while it is waiting. Maybe like this: class pbBroker(pb.Referenceable): def Listen(self): while 1: msg = self.rx.receive() # do stuff with msg from other local objects stackless.schedule() def __init__(self): self.rx = stackless.channel() stackless.tasklet(self.Listen)() def remote_GetData(self): ch = stackless.channel() dbRx.send(ch) # can block since its a dedicated channel result = ch.receive() # return 42 to remote caller thru pb return result I guess the same problem arise when remote_GetData() has to use a deferred too. from twisted.enterprise import adbapi dbPool = adbapi.ConnectionPool("MySQLdb", db="db", user="user", passwd="pw") class pbBroker(pb.Referenceable): def remote_GetDeferredData(self): d = dbPool.runInteraction(DoQuery, "SELECT * FROM users") # DoQuery just wraps a query with try/except d.addCallback(self.GetDeferredData_Success) def GetDeferredData_Success(self, result): # return to remote caller thru pb return result ## WRONG How can I return a value for a remote_* call when I only get that value thru another callback? Sorry its a long and confusing one! Simon -- Linux user #458601 - http://counter.li.org. -- Linux user #458601 - http://counter.li.org. [View Less]

2 1

[Twisted-Python] Stackless/Twisted integration again
by Simon Pickles April 6, 2008

April 6, 2008

I was interested in Andrews Paper on this subject. I note he recommends running a twisted server as a tasklet. Is there a problem doing this? factory = pb.PBClientFactory reactor.connectTCP('127.0.0.1', 1234, Factory) # Load other tasklets here stackless.tasklet(reactor.iterate)(1) or should I perhaps do this? factory = pb.PBClientFactory reactor.connectTCP('127.0.0.1', 1234, Factory) # Load other tasklets here while 1: stackless.tasklet(reactor.iterate)(1) stackless.schedule() … [View More]

2 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com April 6, 2008

April 6, 2008

1 0

[Twisted-Python] Re: Stackless Python/Twisted example
by Andrew Francis April 5, 2008

April 5, 2008

Hi Don and Colleagues: >Thanks much for the example. At a quick scan, it >looks like it'll take some digesting to get my head >around. I'll give it a closer look when I >get the chance. Don, you need to read the code along side the slides. I will probably be giving a revisted version of the talk in early May. I will annotate the example in the week to follow. However I need to change the code somewhat so it is a wee less confusing to a newbie and gets rid of a design flaw … [View More]

1 0

[Twisted-Python] Source specific multicasting (SSM) example?
by Rodney Lott April 5, 2008

April 5, 2008

Hi, there. I was wondering if there is any information or examples on how to use Twisted to perform a SSM operation. I am new to Twisted and to Python. In C, I believe one would use the setsocketopt() to specify the joining to a multicast group (G) to a specific source (S): setsockopt(sock,..., IP_ADD_SOURCE_MEMBERSHIP, (S, G, iface), ...) I believe that Python itself has analogous socket calls and could do this using a UDP socket. I read the Multicast UDP example on Twisted here: http:… [View More]

2 3

[Twisted-Python] Packaging a Twisted app as an Egg
by Matthew Glubb April 5, 2008

April 5, 2008

Hi All, Apologies if this has been covered before. I am looking to package a twisted application that relies on twistd as an egg but I am concerned as to how twistd will be aware of the my package. How do I make twistd aware of my application if it is packaged as an egg? Is this possible? Do I just create my own stub to twistd that takes care of ensuring that twisted/plugins/myapp.py is in Python's path? Thanks a lot, Matt Matthew Glubb Technical Partner email: matthew.glubb(… [View More]

2 2

[Twisted-Python] Other ways to integrate foreign event loops?
by Nathan April 4, 2008

April 4, 2008

Is there a way to do the opposite of task.coiterate? I'm using pyglet for my UI, which was working great until I upgraded from pyglet 1.0 to 1.1. Now for some reason, the pyglet performance tanked. Since I'm only using twisted to send AMP messages very rarely (on the order of one every few seconds), it seems like I could try to work around the problem by giving pyglet back control of the main loop and occasionally giving twisted some time to breath (occasionally like every 50ms or something).… [View More]

6 9

Stackless Python/Twisted example Re: [Twisted-Python] Advice sought on application evolution
by Andrew Francis April 3, 2008

April 3, 2008

Hi Don and Colleagues: >I'll be very interested to learn more about the >melding of the two paradigms. ... >One question: I had the impression that Stackless is >mostly an academic exercise, sort of a proof of >concept. Is this wrong? How stable, portable, etc. is Stackless at this point? Is it a reasonable choice >for a commercial application? Sorry for the wait - I have been very busy. I will probably update my blog on the weekend. Meanwhile, I have enclosed an … [View More]

2 1