Mailman 3 February 2011 - Twisted

[Twisted-Python] LDAP recommendation
by Brent McConnell 23 Feb '11

23 Feb '11

I'm working on a web application that uses Cyclone (Twisted port or Tornado) and wanted to know if anyone had any recommendations for integration with LDAP. I need to do basic authentication as well as add and modify users and groups (occasionally) from the app. I've looked around but haven't seen that much documentation on any LDAP client libraries that are "Twisted". Ldapter is really all I've found and it looks unmaintained. Any help would be appreciated. Brent McConnell

3 2

[Twisted-Python] Twisted Cred using network connections to check credentials
by Devin Jeanpierre 22 Feb '11

22 Feb '11

Hello, I want to authenticate a user in twisted.cred by attempting to log him in to an online service. If the authentication step of the connection fails, the user gets no avatar, and if it succeeds, the user gets an avatar, and that avatar either is or has a reference to that connection. How would I do this? I can think of two ugly ways: I can use a global dictionary mapping avatarIds to protocol instances, and do a lookup on that dict in the Realm to create the avatar. Otherwise, I can instantiate the connection as part of the credentials, and keep the reference to it around when I get the avatar. I'm going ahead with the former, but it's scary. Another possibility is that perhaps the credentials checker and the realm could be the same object, and then I could move the global dict into an attribute. Would that be a better approach? The background for this is that I have an implementation of the chat part of the Direct Connect (DC) protocol, and want to write an IRC<->DC bridge. I chose twisted.words.services on the recommendation of someone, which uses Cred for auth. DC has authentication as a given nick as part of the "handshake" during the start of the connection attempt. Devin Jeanpierre

2 2

[Twisted-Python] Error (and response) handling in protocols
by Jason Heeris 22 Feb '11

22 Feb '11

I'm still a little confused about exactly how to deal with errors and responses using protocols and factories. I think a lot of my misunderstanding stems from the fact that I'm using serial (and inter-process) communications, both of which Twisted treats rather differently from TCP based connections — and all of the documentation is TCP-centric. Anyway, say I have a serial device that I send a single-line command to, and expect a certain format of response. There are about a hundred possible commands, but they all follow the same pattern, so I have a CommandProtocol (some details omitted): class CommandProtocol(LineOnlyReceiver, TimeoutMixin): # For LineOnlyReceiver MAX_LENGTH = 20 def parseReply(self, line): if line[0:4] != self.command: raise CommandReplyError(self.command, self.line) # Parse rest of response return reply def lineReceived(self, line): self.setTimeout(None) try: reply = self.parseReply(line) except: # ...? else: self.factory.commandCompleted(reply) # ...maybe? finally: self.loseConnection() def connectionMade(self): self.sendLine(self.construct_message()) self.setTimeout(self.DEFAULT_TIMEOUT) So I can then write a factory with a buildProtocol method that takes an object with the command string, sets the appropriate attributes on the protocol, and returns that for use with the SerialPort transport. Peticolas' tutorial[1] shows the protocol calling a factory method when the correct data is received. I understand that part fine. But as far as I can tell, all the error handling relies on the ClientFactory's clientConnectionFailed method — and I have no idea when or how this is triggered. I *am* under the impression that it won't ever be triggered on a SerialTransport, though, which means it won't work for me. [1] https://github.com/jdavisp3/twisted-intro/blob/master/twisted-client-8/get-… What should my protocol do if there's an error? I notice that protocols like LineReceiver and NetstringReceiver just silently ignore such problems and let whatever handles connection failures sort it out; but again, this won't work for a serial transport. Even explicitly calling "loseConnection" won't trigger a subsequent "connectionLost" call. So how should I handle reply parsing errors, timeouts, LineReceiver-internal errors (eg. a too-long line) and whatever else might happen? One approach I considered was for my factory to pass my protocol a deferred. The factory would chain another deferred to this one which it could return to the caller. The protocol could then callback or errback *it's* deferred with a result or error. I know how to implement this, but I see that none of the protocols in Twisted (or any of the examples, or the tutorials) do anything like this — it's all just assembly and disassembly of the data and a call to a factory method. This makes me think that I'm doing it wrong. There's also the fact that protocol errors will be handled inconsistently: errors in *my* parsing methods will show up to callers of the factory methods, but errors in Twisted's parsing will pass silently. Another thing I could do is to have something like: def lineReceived(self, line): try: reply = self.parseReply(line) except: self.factory.commandFailed(failure.Failure()) # ...other stuff from above But again, this is inconsistent, and I've not seen anything else in Twisted doing it this way. So how I do I deal with protocol errors in a connectionless protocol? Cheers, Jason

5 15

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 20 Feb '11

20 Feb '11

1 0

[Twisted-Python] ANN: MV3D 0.50 released!
by Michael Handverger 19 Feb '11

19 Feb '11

I’m pleased to announce the v0.50 release of MV3D! With this new release, we are considering MV3D to be beta, which means future releases should not break backwards compatibility with existing data or API calls. The framework gains several major features this release such as a path-finding system for AI controlled characters (NPCs) and support for the Panda3D rendering engine. Besides new features, an incredible amount of bugs have been fixed. Many of these include performance and tool chain fixes which together make this the most stable release of MV3D to date. To add to the polish, in 0.50, a user manual was added as a reference to new users. MV3D is an open source virtual world simulation framework written in Python which uses Twisted for networking. It was designed with scalability in mind and aims to be able to distribute a world across as many servers as needed with dynamic load balancing. For more information on MV3D and this or future releases, please visit the website at http://www.mv3d.com. The full release notes for version 0.5 are available at this url: http://www.mv3d.com/trac/browser/tags/mv3d-0.50/release-notes.txt Also, feel free to stop by the IRC chat room, #MV3D on irc.freenode.net. Thanks, Mike

1 0

[Twisted-Python] A thread in isolation
by Andy.Henshaw＠gtri.gatech.edu 18 Feb '11

18 Feb '11

I apologize in advance - another question about Twisted and threading. I just seem to have trouble with this and I'm hoping my example is simple enough that a good explanation will be forthcoming. I'm trying to start and stop Twisted in its own thread, over and over. I've seen some notes that say this is a bad idea and that programs that do so should be restructured. But, it would really work better for this application if I could do it somehow. My controlling thread just wants to start Twisted, let it do its thing, and then let the thread stop. Later on, I'd like to do it again. In the example code below, the thread runs and stops fine in the first iteration. In the second iteration, it starts, seems to run okay, but it never stops. Is this an unreasonable request? from threading import Thread from twisted.internet import reactor from twisted.internet import protocol PORT = 9999 class AgentManager(Thread): def run(self): print '\nstarting agentmanager', self reactor.connectTCP('localhost', PORT, AgentClientFactory()) reactor.listenTCP(PORT, AgentServerFactory()) reactor.callLater(5, reactor.stop) reactor.run(installSignalHandlers=0) class Producer(protocol.Protocol): def connectionMade(self): print 'connection made' self.transport.write('Hello') def connectionLost(self, reason): print 'connection lost' class Consumer(protocol.Protocol): def dataReceived(self, data): print data class AgentServerFactory(protocol.ServerFactory): protocol = Producer class AgentClientFactory(protocol.ClientFactory): protocol = Consumer if __name__ == '__main__': for i in range(3): mgr = AgentManager() mgr.start() mgr.join()

4 4

[Twisted-Python] Looking for intern (and full time developers)
by Itamar Turner-Trauring 17 Feb '11

17 Feb '11

Hi everyone, My group at ITA Software, Availability and Inventory Control, is looking for an intern. Inventory Control in this context is a core part of an airline's reservation system, and we are currently hard at work developing for American Airlines: http://www.prnewswire.com/news-releases/american-airlines-selects-ita-softw… As part of our efforts, we're looking for an intern (for the summer, but could also be earlier). Large parts of our system are written in Python using Twisted, and so Twisted experience would be helpful. The current plan is to have the intern develop some of our XML processing infrastructure, but depending on experience or circumstances other projects may be chosen. You can apply here: http://www.itasoftware.com/careers/work-at-ita/showjob.html?uid=745590 We are also hiring for many full time positions, some of which may involve the use of Twisted, in particular the Operations group infrastructure. Other positions will have you coding in Common Lisp (learned on the job, typically), Java, C++ and more: http://www.itasoftware.com/careers/ ITA Software is located in lovely Cambridge, MA, home to MIT and Harvard. Google is currently attempting to acquire ITA, pending approval by the government. If you apply for a position at ITA, especially the intern position, please mention my name and email me to let me know. --Itamar

1 0

[Twisted-Python] What's the purpose of __?
by Attila Nagy 16 Feb '11

16 Feb '11

Hi, I'm trying to do an LMTP implementation based on smtp.py and came to the issue of class private variables with double underscores. Examples: http://twistedmatrix.com/trac/browser/trunk/twisted/mail/smtp.py#L746 http://twistedmatrix.com/trac/browser/trunk/twisted/mail/smtp.py#L815 and a lot of occurrences in this file. This makes me a problem, because I override some functions in a class, named LMTP, so these cannot work together. What's the rationale of using these, instead of static names? Thanks,

4 5

[Twisted-Python] transport.write(a + b) versus [transport.write(a); transport.write(b)]
by Carlos Valiente 16 Feb '11

16 Feb '11

>From twisted.protocols.basic.IntNStringReceiver.sendString(): def sendString(self, string): self.transport.write( struct.pack(self.structFormat, len(string)) + string) Would it make sense to do something like this instead: def sendString(self, string): self.transport.write( struct.pack(self.structFormat, len(string))) self.transport.write(string) in order to avoid the creation of the extra string object?

2 1

[Twisted-Python] sigchld
by Jason Rennie 15 Feb '11

15 Feb '11

I'm in the process of porting code from Twisted 8.1 to 10.1 which needed some custom signal handling code. Twisted 8.1's approach to signal handling was (as far as we could tell) too monolithic, so we chose "installSignalHandlers=False" and wrote our own signal handling code, installing reactor._handleSigchld so that spawnProcess would work. Of course, we have to change this now since #733 was fixed ( http://twistedmatrix.com/trac/ticket/733). IIUC, (and please correct if I don't) the SIGCHLD handler installation is performed by twisted.internet.posixbase.PosixReactorBase._handleSignals (the "if platformType == 'posix':" block). So, one option would be to run that code while still using "installSignalHandlers=False". I'm wondering if there's a better way. What if I were to let Twisted install signal handlers, but run my own signal handling code after the reactor starts (using callLater(0,...)) that installs "super" handlers which calls the Twisted handler along with any other handlers I want to run. I'd use signal.getsignal to get a reference to the Twisted handler(s). Might this work? Or, is there an obvious flaw? In particular, can I get the SIGCHLD handler using signal.getsignal, then call it via my "super" handler? I don't need to do any special handling of SIGCHLD, so I could just leave that signal handler alone. All thoughts appreciated. Thanks, Jason -- Jason Rennie Research Scientist, ITA Software 617-714-2645 http://www.itasoftware.com/

3 6

Twisted February 2011