Mailman 3 January 2008 - Twisted

[Twisted-Python] Teach Me Twisted?
by Steve Holden 26 Jan '08

26 Jan '08

Hi, Twisters: I've been lurking on the list a while, and wondered how much of a Twisted presence there is going to be at PyCon in March. The reason I ask is because of a conversation that I had with Glyph recently. We both agreed that it would be a good thing to have more learning materials available for Twisted. One possible solution would be to run an open space session at PyCon in which experienced users explain to a novice how to get started in the Twisted environment. Of course the idea is that I should be the novice (I really don't know that much about Twisted) and that you should be the people explaining to me and (hopefully) leading me through to the Nirvana of twisted enlightenment. Naturally this idea won't work unless experienced Twisted users get behind it. I would like your suggestions for preliminary reading, though I don't want around arrive at the conference knowing too much. Is this a good idea? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

8 13

Re: [Twisted-Python] Soon to be not-a-newbie?
by Tristan Seligmann 26 Jan '08

26 Jan '08

* Simon Pickles <sipickles(a)hotmail.com> [2008-01-26 09:47:08 +0000]: > Tristan Seligmann wrote: >> it certainly leads to subtle traps in most cases, while providing >> little real benefit (despite the perceived benefit). >> > Does this mean if I don't know what inlinecallbacks are, I shouldn't worry? Briefly, inlineCallbacks is a "trick" for writing a series of Deferred callback chains as a single generator. If you're not using it, you don't really need to care about it, unless you end up reading some code that uses it; but as far as I know, there is no code in Twisted itself that uses it. -- mithrandi, i Ainil en-Balandor, a faer Ambar

2 1

[Twisted-Python] Another inlinecallback question
by Andrew Francis 26 Jan '08

26 Jan '08

Hi Folks: Another inclinecallback question. This time I am using PyAMF. Like xmlrpc, operations are method calls. So I have the following: def echo(self, x): . . return result now from my PyAMF.operation, I want to do the following, @defer.inlineCallbacks def echo(self, x): result = yield someCallResultingInADeferred() return result the problem is that echo is now a generator, I can't write - return result - returns in generators can't take arguments. Once again, I am at a bit of loss about how to structure the code. Any suggestions are welcomed. Cheers, Andrew ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping

2 1

Thanks! Re: [Twisted-Python] Question about RequestHandlers
by Andrew Francis 26 Jan '08

26 Jan '08

Hello Jean-Paul: AF>What I am unclear about is how to properly structure AF>the programme to get the appropriate behaviour. I am AF>assuming this is possible. If so, what am I missing? >The part where you got confused is the part that >inlineCallbacks, I think. Yes. >You defined a gotPage callback which uses >defer.returnValue. This is totally bogus. If you >want a callback to return a value, then you just >return it. defer.returnValue lets you specify the >result of a Deferred returned by a call to a function >decorated with inlineCallbacks. >Your gotPage function has not been so decorated, so >there's your bug I suspect. No, I found the example for inlinecallback a wee confusing. When I see the line "makeSomeRequestResultingInDeferred()", I automatically thought I needed to create a deferred and in the process, add a callback function. I also misinterpreted the proviso about returnValue. Perhaps it is just me, but maybe it would be better to use a more concrete example using a twisted call. That said, I have done what you suggested: class MyRequestHandler(http.Request): @defer.inlineCallbacks def process(self): try: result = yield client.getPage("http://www.google.com") except Exception, err: log.err(err, "process getPage call failed") else: self.setHeader('Content-Type', 'text/html') self.write(result) self.finish() and this works beautifully. Thanks! inlinecallbacks look really cool..... Cheers, Andrew ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping """ inlineExample.py Based on the example on page 41 of Abe Fettig's "Twisted Networking Programming Essentials," this example uses an inclinecallback to allow the HTTP requestHandler to make a twisted call and wait for the result. """ from twisted.web import http, client from twisted.internet import defer class MyRequestHandler(http.Request): @defer.inlineCallbacks def process(self): try: result = yield client.getPage("http://localhost") except Exception, err: log.err(err, "process getPage call failed") else: self.setHeader('Content-Type', 'text/html') self.write(result) self.finish() class MyHttp(http.HTTPChannel): requestFactory = MyRequestHandler class MyHttpFactory(http.HTTPFactory): protocol = MyHttp if __name__ == "__main__": from twisted.internet import reactor reactor.listenTCP(8000, MyHttpFactory()) print "server running" reactor.run()

1 0

[Twisted-Python] LDAP server based on python
by xavier dutoit 25 Jan '08

25 Jan '08

Hello, My goal is to be able to access contacts that are into a (open source) CRM from any LDAP compatible client (mostly email clients that can access an LDAP server as a shared contact database). This is a fairly simple LDAP server, mostly implementing search on emails and get contacts back (+a very simple ACL). I read a post from Tommi Virtanen a (few) years ago http://twistedmatrix.com/pipermail/twisted-python/2005-May/010484.html And found more information on : http://www.inoi.fi/open/trac/ldaptor I've svn checkouted ldaptor, and did find ldaptor/protocols/ldap/ldapserver.py However, I'm kind of lost now and didn't find any examples (or discussion), and was wondering about the status of the ldap server. If this is still active, could you tell me where to find more info and hopefully some examples ? I saw this thread: http://twistedmatrix.com/pipermail/twisted-python/2008-January/016534.html about "ldaptor and bind" If I read properly, the ldap server is something for testing, but not production ready (eg no security/acl implemented by default). Am I right ? Did Ottavio Campana manage to have something working and did he publish the result of his modifications ? In short, is there a simple example of a server with a sql backend, that implement searches ? thanks in advance, Xavier

2 1

[Twisted-Python] Question about RequestHandlers and inlinecallbacks
by Andrew Francis 25 Jan '08

25 Jan '08

Hello Folks: I am starting to learn inlinecallbacks. I am trying to do the following: from within a RequestHandler, I get a web page with the client.getPage() method. I subsequently return the web page to the requesting web browser. The problem - The web browser hangs. However the requestHandler.writes are echoed to standard output (or error). The problem is that in process(self), a deferred is created but process() ends finishes without outputting anything to the web browser. Eventually, the associated callback is triggered but the request has already been serviced. What I really want is process() to wait until the client.getPage() is finished. What I am unclear about is how to properly structure the programme to get the appropriate behaviour. I am assuming this is possible. If so, what am I missing? I have included some sample code. Cheers, Andrew ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping """ inlineExample.py A simple example to see if I can use inlinecallbacks from an RequestHandler """ from twisted.web import http, client from twisted.internet import defer def gotPage(result): print result return defer.returnValue(result) class MyRequestHandler(http.Request): @defer.inlineCallbacks def __process__(self): aResult = yield client.getPage("http://localhost").addCallback(gotPage) self.setHeader('Content-Type', 'text/html') self.write(aResult) self.finish() def process(self): """ I want process to wait until client.getPage is finished. """ self.__process__() class MyHttp(http.HTTPChannel): requestFactory = MyRequestHandler class MyHttpFactory(http.HTTPFactory): protocol = MyHttp if __name__ == "__main__": from twisted.internet import reactor reactor.listenTCP(8000, MyHttpFactory()) print "server running" reactor.run()

2 1

[Twisted-Python] Newbie question
by Simon Pickles 25 Jan '08

25 Jan '08

Hi, I thought I'd better try and understand twisted a bit more rather than just using it! Can anyone tell me why this code is erroneous? ##################### from twisted.internet import reactor, defer print "test" def printResult(s): print "Data: %s", str(s) def printEnd(): print "End" def printError(error): print error d = defer.Deferred() reactor.callLater(4, d.callback) d.addCallback(printEnd) d.addErrback(printError) reactor.run() ################################### I get the error: TypeError: callback() takes exactly two arguments (1 given) in twisted/internet/base.py call.func(*call.args, **call.kw) Thanks Simon -- Linux user #458601 - http://counter.li.org.

3 3

[Twisted-Python] Senior Python developer position
by Tim Stebbing 24 Jan '08

24 Jan '08

Senior Python Developer [AU] [snip out jobsite preamble].. G'day all, we're in need of another senior pythonista able to start asap at our Melbourne offices. We run an online health and fitness service which is growing into the global market, and the application is being redeveloped to scale globally. The role will primarily involve implementing a distributed framework; global authentication mechanisms, business logic, RPC between nodes etc. There is plenty of opportunity for free-thought, as a senior developer there is an expectation you will be able to work autonomously on components and meet deadlines, be a creative voice in the team, as well as assist the more junior developers. This is a full time position, 70-90k neg based on exp. Technical requirements: - Strong OO design principals - 4+ years commercial experience - Demonstrable ability to code in Python - Experience with Open Source Relational Databases - Knowledge of web technologies such as HTML, XML, Javascript, XML-RPC - Strong unit testing skills Professional requirements: - Excellent written and verbal communication - Good expectation setting - Prioritisation and time management skills - Ownership of projects and relevant outcomes Desirable experience: - Knowledge of Zope3 application server - Knowledge of Twisted - Performance/Scalability knowledge - Familiarity with Linux

1 0

Re: [Twisted-Python] Teach Me Twisted?
by Andrew Francis 24 Jan '08

24 Jan '08

Hi Steve: >I've been lurking on the list a while, and wondered >how much of a Twisted presence there is going to be at >PyCon in March. Good question. I am giving a talk called "Adventures in Stackless Python/Twisted Integration." The talk has been labelled "advanced." However for me, a large part of the "adventure" is slowly learning Twisted and Stackless concepts. So I really want to capture newbie misconceptions, the pitfalls, the "you shouldn't do that" advice, and the cool things you can do once you get the swing of things. And finally, where do we go from here? One of things I will do is use diagrams to show control flow in Twisted in Stackless. I feel the Abe Fettig book and the Stackless literature would be greatly improved if it has more illustrations of this sort. >One possible solution would be to run an open space >session at PyCon in which experienced users explain to >a novice how to get started in the Twisted >environment. Well no one from Twisted has contacted me (on the other hand, I have talked a bit with Christian since he has a Stackless and PyPy talk). However I would love to help novices get on board. Cheers, Andrew ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs

2 1

[Twisted-Python] How can I make twisted and dbus work together?
by hwan2265＠mail.usyd.edu.au 22 Jan '08

22 Jan '08

Hello everyone, I am trying to write a imap mail fetcher base on twisted and dbus. I have written a simple fetcher without dbus functionality and it's working fine. However I got a problem while adding dbus. http://twistedmatrix.com/trac/ticket/1352 is the only article about how can twisted and dbus work together that I could get from google and I just followed its suggestion and added from twisted.internet import glib2reactor glib2reactor.install() to my code. As what I read from twisted API http://twistedmatrix.com/documents/current/api/twisted.internet.glib2reacto… "In order to use this support, simply do the following: | from twisted.internet import glib2reactor | glib2reactor.install() Then use twisted.internet APIs as usual. " I thought everything will be fine. But when I was running my code, I got a error: Traceback (most recent call last): File "dbustwisted.py", line 5, in <module> glib2reactor.install() File "/usr/lib/python2.5/site-packages/twisted/internet/glib2reactor.py", line 33, in install return gtk2reactor.install(False) File "/usr/lib/python2.5/site-packages/twisted/internet/gtk2reactor.py", line 270, in install installReactor(reactor) File "/usr/lib/python2.5/site-packages/twisted/internet/main.py", line 24, in installReactor "reactor already installed" AssertionError: reactor already installed I don't know what's wrong with it. This is my code, I have not add the dbus part yet and just test the glib2reactor. from twisted.mail import imap4 from twisted.internet import protocol, defer import email, os, time MAILPATH = "Mails" IMAPSERVER = "xxxxxxxxxxx" USERNAME = "xxxxx" PASSWORD = "xxxxx" class IMAPDownloadProtocol(imap4.IMAP4Client): def serverGreeting(self, capabilities): login = self.login(self.factory.username, self.factory.password) login.addCallback(self.__loggedIn) login.chainDeferred(self.factory.deferred) # any event that fires login will also fire factory.deferred def __loggedIn(self, result): return self.select(self.factory.mailbox).addCallback(self.__selectedMailbox) def __selectedMailbox(self, result): allMessages = imap4.MessageSet(1, None) # a set of messages from message number 1 to the end of the mailbox return self.fetchUID(allMessages, True).addCallback(self.__gotUIDs) # fetch a list of UID def __gotUIDs(self, uidResults): self.messageUIDs = [result['UID'] for result in uidResults.values( )] self.messageCount = len(self.messageUIDs) print "INFO: %i messages in %s." % (self.messageCount, self.factory.mailbox) return self.fetchNextMessage( ) # start to download the message one by one def fetchNextMessage(self): if self.messageUIDs: nextUID = self.messageUIDs.pop(0) messageListToAnalysis = imap4.MessageSet(nextUID) # creates a MessageSet matching only that message's UID return self.fetchAll(messageListToAnalysis, True).addCallback(self.__gotIndex) # fetchAll - fetching the UID, flags, size, date, envelope # UID keyword set to true to tell the server that the MessageSet is referring to messages by UID, not sequence number else: return self.logout( ).addCallback(lambda _: self.transport.loseConnection( )) def __gotIndex(self, indexResults): index = indexResults.values() index_details = index[0] UID = index_details['UID'] flags = index_details['FLAGS'] size = index_details['RFC822.SIZE'] date = index_details['INTERNALDATE'] envelope = index_details['ENVELOPE'] title = envelope[1] sender = envelope[3] temp = [] # replace None in the envelope as "Empty", because None is a key word in python for element in sender[0]: if element != None: temp += [element] if element == None: temp += ["Empty"] sender = temp if flags == []: flags = ['\\Unseen'] print "INFO: Downloading message %i of %i" % (self.messageCount-len(self.messageUIDs), self.messageCount) messageListToFetch = imap4.MessageSet(UID) return self.fetchMessage(messageListToFetch, True).addCallback(self.__gotMessage, UID) def __gotMessage(self, fetchResults, UID): messageData = fetchResults.values( )[0]['RFC822'] self.factory.handleMessage(messageData, UID) return self.fetchNextMessage( ) # fetch next message def connectionLost(self, reason): if not self.factory.deferred.called: # connection was lost unexpectedly! self.factory.deferred.errback(reason) class IMAPDownloadFactory(protocol.ClientFactory): protocol = IMAPDownloadProtocol def __init__(self, username, password, mailbox, path): self.username = username self.password = password self.mailbox = mailbox self.path = path self.deferred = defer.Deferred( ) def handleMessage(self, messageData, UID): mailFile = os.path.join(path, UID) output = file(mailFile, 'w+b') parsedMessage = email.message_from_string(messageData) output.write(parsedMessage.as_string(unixfrom=True)) def startedConnecting(self, connector): print "INFO: Connection start" def clientConnectionFailed(self, connection, reason): self.deferred.errback(reason) def clientConnectionLost(self, connection, reason): print "INFO: Connection stop" #connection.connect() if __name__ == "__main__": from twisted.internet import glib2reactor glib2reactor.install() ################################################ # I added the 2 lines here and made the program crash ################################################ from twisted.internet import reactor import sys, getpass server = IMAPSERVER user = USERNAME mailbox = "Inbox" path = MAILPATH password = PASSWORD factory = IMAPDownloadFactory(user, password, mailbox, path) reactor.connectTCP(server, 143, factory) reactor.run( ) I am new to twisted, dbus even python, any helps will be appreciated. Regards Huisan ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

5 6