Mailman 3 April 2009 - Twisted

RE: [Twisted-Python] RE: Question about Descriptors and SelectReactor
by andrewfr_ice＠yahoo.com 08 Apr '09

08 Apr '09

Hi Glyph and Folks: AM>I would be actually quite curious to know the rationale of choosing AM>select() over epoll() these days. epoll() scales like O(1) with the AM>number of file descriptors, it is very performant, stable, and has no AM>limitation on the overall number of fds on linux (except for your /proc AM>and ulimit -n settings). I'd use epoll reactor, unless you have some AM>very specialized requirement. G>Short answer: because select() is always available. Thanks. I do most development in Linux now. The main thing I wanted to avoid was altering file descriptor limits and bundling a new AMI and still not having the ability to test above a certain descriptor limit. After the semester is over, I want to conduct some new tests with Stackless Python and Twisted. I am also playing with new designs. I am interested in comparing Stackless/Twisted solutions to pure Twisted solutions (just to gauge of inefficiencies). I am particularly interested in how channel preferences (a Stackless scheduling mechanism) affect certain performance metrics depending on the number of (and I guess duration) of connections. I will probably start looking at Reactor code again. This is why in the donate time thread, I was willing to volunteer time there (if I can actually be of help) Cheers, Andrew

2 1

[Twisted-Python] Question about Descriptors and SelectReactor
by Andrew Francis 07 Apr '09

07 Apr '09

Hi: Perhaps this question has been asked before but is there a limit to the number of descriptors the SelectReactor can handle? Is this a function of the operating system? Cheers, Andrew

4 4

[Twisted-Python] New user - plugin question
by tim wylie 06 Apr '09

06 Apr '09

Hello, I'm a long time linux/python user who's just now starting to use twisted. I am looking for advice more than anything specific. We're working on a small game/etc server that has several small games that can be played between users. Maybe a card game, tic-tac-toe, etc. There are a lot of different plugin subclasses to inherit from-how do you know which is best? And with this kind of framework, is it better to have one interface class, say iGame, that all the games inherit from, or is it better to treat each game like a seperate plugin inheriting from different twisted plugin classes? There is a lot of good twisted documentation, but that is what also makes it difficult because each one does things differently depending on what they're implementing and it makes it difficult to decide which is the correct approach for what I want to do. So, if there are any good documents related closer to what I'm doing I'd appreciate the heads up on that as well. Well, thanks for your time. Tim _________________________________________________________________ Windows Live™ SkyDrive: Get 25 GB of free online storage. http://windowslive.com/online/skydrive?ocid=TXT_TAGLM_WL_skydrive_032009

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 05 Apr '09

05 Apr '09

1 0

[Twisted-Python] twisted and logging
by Fabrizio Mancini 04 Apr '09

04 Apr '09

Hi, i'm using the package twisted.python.log to log messages. Till now, it's all ok, but with the standard logging package from python i had setup a logging system that logged normal logs to one file, error logs to a separete files, and some special operations to one other file. Every logging instance was an instance of the RotatingFileHandler logging class. I saw that there is a rotating file handler like class in twisted, but how can i achieve the same behaviour above? (Three different logging instances to three different rotating files?) Thanks in advance Fabrizio

2 1

Re: [Twisted-Python] Documentation overviews
by Andrew Francis 03 Apr '09

03 Apr '09

Message: 1 Date: Thu, 2 Apr 2009 09:05:27 -0700 From: Cary Hull <cary.hull(a)gmail.com> Subject: [Twisted-Python] Documentation overviews To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Message-ID: <3fce0ae70904020905l286b78acvb17f4f0d572d9680(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi Cary: >One of the goals for the Pycon sprint was to develop user oriented >overviews for existing documentation. These would ideally guide users >through everything they would need to read in order to accomplish a >desired goal with Twisted. >I've formulated the following 'track' suggestions based on a conversation >in the sprint room a few nights ago. Feedback would be greatly >appreciated. Sadly I couldn't make it to PyCon 2009 this year. However I would add two things. 1) A Section on Control Flow. In my PyCon 2008 talk, I tried to explain what the control flow of a Twisted programme was. I consider this lack of information a deficiency in the Fettig book (a book that helped me). I needed to understand control flow to solve my problem. I believe how control flow understand how Twisted can provide a form of non-preemptive multitasking without threads. It also explains how to structure a Twisted programme. 2) The Class Hierarchy I believe a source of confusion is over how the classes are used. It would be good to have something like UML class diagrammes. Cheers, Andrew P.S - one of these days I have to fix the slides for "Adventures" and write the paper companion

1 0

[Twisted-Python] Question about application exit handling
by Jay Deiman 02 Apr '09

02 Apr '09

First, I'm pretty new to the Twisted framework and I've spent a lot of time poking through documentation and subsequently, I've managed to figure out a lot of things on my own. Here is one issue I'm having right now. I wrote an application to be run under twistd and it works great, except for one problem. When I try to exit twistd, it hangs. I know why this is, it's due to a twisted.python.threadpool.ThreadPool that I've instantiated and use in my code. Now, here is the actual question. Is there a callback/signal/notifier of some sort that is triggered when an application is closed? I know twistd handles the SIGINT, SIGTERM, etc., but is there some sort of notification that is sent to the current running application that I can use for clean up? Thanks, Jay -- Jay Deiman \033:wq!

1 1

[Twisted-Python] PBServerFactory - which client disconnected?
by Mike Stoddart 02 Apr '09

02 Apr '09

I'm experimenting with Twisted Spread for a registry server, among other things. I'm just curious how the onDisconnect function knows which client disconnected? I'm also curious why the PBServerFactory class doesn't have a disconnection function to complement clientConnectionMade? I don't need security - this is running on a closed, secure LAN and I don't need "Avatars". Any suggestions on how to improve this? from twisted.internet import protocol, reactor from twisted.python import log from twisted.spread import pb import sys log.startLogging(sys.stdout) class RegistryService(pb.Root): def __init__(self): self._clientCount = 0 self._clients = {} def remote_register(self, name, service): log.msg("Registering service " + name) log.msg(str(service)) # Welcome the client. service.callRemote("welcome") def onDisconnect(self): log.msg("onDisconnect") log.msg("Client count = " + str(len(self._clients))) def onConnect(self, broker): s = broker.transport.getPeer().host + ":" + str(broker.transport.getPeer().port) log.msg("onConnect - client at " + s) # Add the client. self._clients[s] = broker.transport log.msg("Client count = " + str(len(self._clients))) # Set a callback in case they disconnect. broker.notifyOnDisconnect(self.onDisconnect) class MyPBFactory(pb.PBServerFactory): def __init__(self, root): pb.PBServerFactory.__init__(self, root) def clientConnectionMade(self, broker): log.msg("clientConnectionMade") self.root.onConnect(broker) registry = RegistryService() f = MyPBFactory(RegistryService()) reactor.listenTCP(9999, f) reactor.run() Thanks Mike

1 1

Re: [Twisted-Python] Twistd and application framework questions
by Terry Jones 02 Apr '09

02 Apr '09

Hi Nadav, Itamar >>>>> "Itamar" == Itamar Shtull-Trauring <itamar(a)itamarst.org> writes: Itamar> On Wed, 2009-04-01 at 13:16 -0400, Nadav Aharony wrote: >> 1) The program uses multiple UDP/TCP clients and servers that are >> currently launched with reactor.listenTCP / UDP connectTCP/UDP etc. I've >> been using MultiService, according to the twisted documentation. Some >> of these services were are at a top level of my app so it was easy to >> turn that part into a .tac file and switch the reacor calls to >> internet.TCPServer etc. However, some of them are deep inside my code, >> and are instantiated on the fly. We (at Fluidinfo) have a similar situation, though I wouldn't say the services that start/stop later are deeply buried. They're just not started right away when startService is called on the MultiService via the plugin mechanism. >> What is the "right" way to attach them to my service parent? (the part >> with ".setServiceParent(<myMultiservice>") Should I now add a pointer to >> my MultiService that will be propagated down the code hierarchy to each >> of these calls and be accessible at the inner scopes? Is there a neater >> way to do it just with importing the right modules? (in the same way >> that in usual twisted scripts the loaded reactor is a global reactor) At the suggestion/insistence :-) of Esteve Fernandez, we do this by using the components machinery of Twisted. You write an adaptor that knows how to take the core of your service and adapt it to some other form of service interface. The adaptor can have access to the original Multiservice, it can provide startService and stopService methods that call into the special service that you're trying to launch, etc. It's nice because the mechanisms for creating a new service and getting it hooked up to the outside world, can be encapsulated in the adaptor and don't pollute the code that's trying to just provide the underlying service (your "business logic" I guess you'd call that :-)) BTW, we also have some service startup ordering constraints that we fixed by subclassing Multiservice. There's a slow-motion thread on this approach, so the jury's still out, but the code described here works fine for me: http://twistedmatrix.com/pipermail/twisted-python/2009-February/019225.html http://twistedmatrix.com/pipermail/twisted-python/2009-February/019249.html Itamar> 1. You don't need to attach to parents service, you can also call Itamar> start/stopService directly. Itamar> 2. You can still call "self.port = reactor.listenTCP(...)" directly Itamar> in a custom startService, and self.port.stopListening() in a custom Itamar> stopService. You don't have to use twisted.application.internet, it Itamar> doesn't add much beyond a little convenience. Itamar> The point of services is to encapsulate startup and shutdown logic, Itamar> not to make your life harder :) The main reason I like to add the new service to the original Multiservice is that its stopService gets called at the right time and I don't have to think about it. That's a nice convenience. Terry

1 0

[Twisted-Python] official packaging
by Esteve Fernandez 02 Apr '09

02 Apr '09

Hi all, I recently came across this issue [1], and Glyph commented about official packaging. We're using Ubuntu Hardy and created packages for Twisted 8.2.0, as Ubuntu LTS is our platform of choice. Those packages were accepted by their official maintainer (Matthias Klose, doko(a)debian.org) and uploaded to Ubuntu Jaunty and Debian unstable. Sadly, the official packages now use python-central (>=0.6.11), which isn't available in Hardy, so we'll have to stick to the packages in our PPA [2] In any case, we'll update them when Twisted 9.0 is released, but I think it would be better to upload them to a PPA repository officially approved by the Twisted team. Anyway, it would be great to have recent packages of Twisted for distributions that offer some kind of long term support contracts (Ubuntu, RHEL, etc.), but that tend to get a bit outdated. Cheers. 1 - http://twistedmatrix.com/trac/ticket/3102 2 - https://launchpad.net/~fluidinfo/+archive/ppa

3 2