Mailman 3 July 2003 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Woven: access to submodels
by Mary 09 Aug '03

09 Aug '03

I'm trying to replicate the functionality of http://eyes.puzzling.org/ (currently powered by a bunch of Python CGI scripts running on an Apache webserver) in Woven. eyes.puzzling.org is a kind of a minimalist blog, with pages that list recent entries, the authors of the entries, and so on. At the moment, I'm trying to replicate the authors functionality (2 different sets of pages: one which lists all the authors and links to their individual page; and one for each author, which lists their individual articles. The /authors/ page (listing all the authors and linking to their individual page) is generated by objects of the AuthorsPage subclass. For each author, I would like to generate a /author/AUTHORNAME/ page which lists their individual entries. I am trying to create these pages from AuthorsPage's getDynamicChild method. The idea is to pass the already constructed submodel -- a dictionary representing the individual author -- to the IndividualPage object I will construct to represent the individual author's page, rather than have IndividualPage make its own database query for that information. At present, as you can see below, I'm using getSubmodel to retrieve the (dictionary) submodel associated with a particular author. However, this returns a twisted.web.woven.model.DeferredWrapper instance, rather than the dictionary itself. Should I make IndividualPage add callbacks to the DeferredWrapper, or am I missing something crucial about retrieving submodels or is this model more fundamentally incorrect? For example, should I be passing submodels around like that at all? If so, is there a higher level call I should make to access the submodel? The two classes following are the AuthorsPage and IndividualPage classes described above. Thanks, -Mary class IndividualPage(page.Page): # getTemplate munges two template files together template = getTemplate("/home/mary/cvs/Projects/Eye/author-page.html") def initialize(self, *args, **kwargs): self.amodel = kwargs['amodel'] def wmfactory_author(self, request): return self.amodel['author'] class AuthorsPage(page.Page): template = getTemplate("/home/mary/cvs/Projects/Eye/authors-page.html") def initialize(self, *args, **kwargs): self.dbpool = kwargs['pool'] self.authors = None def wmfactory_authors(self, request): query = self.dbpool.runQuery("SELECT DISTINCT posts.name, link, password FROM authors, posts WHERE authors.name = posts.name ORDER BY name;") query.addCallback(self.makeAuthors) return query def makeAuthors(self, rows): ''' Construct self.authors, return them as a list @return a list of author dictionaries ''' authors = {} for row in rows: (name, link, password) = row authors[name] = { 'name' : name, 'link' : link, 'password' : password } return authors def wvfactory_alink(self, request, node, model): return AuthorLink(model) def getDynamicChild(self, name, request): return IndividualPage(amodel=self.getSubmodel('authors')[name]))

4 10

[Twisted-Python] Re: [Zope3-dev] twisted zope
by Gary Poster 06 Aug '03

06 Aug '03

Shane Hathaway wrote: > Gary Poster wrote: >> ... > May I suggest a different approach? > ... But of course. ;-) FWIW, this is exactly what I meant by "a lot more work" (except that you know more about it than I :-). Yes, I'd love to see this happen if you're up for it, Butch. > Straighten out the licensing first. Oh yeah, forgot about that. I've seen several discussions of the licensing on the Twisted list... Maybe Phil Eby's implementation is the only way out of that one. Gary

6 9

[Twisted-Python] Remote PB calls in ApplicationService.stopService
by Alex Levy 02 Aug '03

02 Aug '03

I had two programs running; one was a twistd -y instance, the other was a simple script that received objects over Perspective Broker and printed them out. The -y'able application consists of one Service: class MyService(app.ApplicationService): def somePBRemoteCall(self, event): d = pb.getObjectAt(self.host, self.port, self.timeout) d.addCallback(self.foundCatcher, event) def foundCatcher(self, obj, event): d = obj.callRemote('catch', event) def stopService(self): somePBRemoteCall(i_am_dying_event) app.ApplicationService.stopService(self) This isn't the code verbatim, but this is basically what it's doing. I want the receiving side to be informed when the sending side is about to stop. On the sending side, this results in 'Connection failed'. On the receiving side, the following traceback appears. Could somebody more familiar with the internals of Perspective Broker explain what's going wrong, or how I might change my approach? ---snip--- Traceback (most recent call last): File "sandbox/test-catcher.py", line 15, in ? reactor.run() File "/usr/lib/python2.2/site-packages/twisted/internet/default.py", line 122, in run self.mainLoop() File "/usr/lib/python2.2/site-packages/twisted/internet/default.py", line 133, in mainLoop self.doIteration(t) File "/usr/lib/python2.2/site-packages/twisted/internet/default.py", line 472, in doSelect _logrun(selectable, _drdw, selectable, method, dict) --- <exception caught here> --- File "/usr/lib/python2.2/site-packages/twisted/python/log.py", line 65, in callWithLogger callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.2/site-packages/twisted/python/log.py", line 52, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.2/site-packages/twisted/python/context.py", line 32, in callWithContext return func(*args,**kw) File "/usr/lib/python2.2/site-packages/twisted/internet/default.py", line 495, in _doReadOrWrite selectable.connectionLost(f) File "/usr/lib/python2.2/site-packages/twisted/internet/tcp.py", line 253, in connectionLost protocol.connectionLost(reason) File "/usr/lib/python2.2/site-packages/twisted/spread/pb.py", line 563, in connectionLost for lobj in self.remotelyCachedObjects.values(): exceptions.AttributeError: Broker instance has no attribute 'remotelyCachedObjects' ---snip--- -- Alex Levy WWW: http://mesozoic.geecs.org "Never let your sense of morals prevent you from doing what is right." -- Salvor Hardin, Isaac Asimov's _Foundation_

4 3

[Twisted-Python] StepReactor?
by Nicola Larosa 02 Aug '03

02 Aug '03

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 What about a StepReactor? It would be great to be able to stop time, have a means to look at the event queue and to say "DispatchNextEvent" by hand. Or even better, "PushTimeForward(x)" and see what happens. In short, a simulator. This assuming, of course, that all other parts of the system are pausable, be they internal or external. - -- "It seems like Zope has done a dance with Twisted but still hasn't asked it out on a date." Paul Everitt Nicola Larosa - nico(a)tekNico.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQE/KLzLXv0hgDImBm4RApH3AJ0SqGfDLxmdI7GjrWBBzGOlbI3V3QCfdUyB bQkgGavTvpi1WnWpI6aO/Gg= =Jsn5 -----END PGP SIGNATURE-----

3 4

[Twisted-Python] Callbacks, Looping, and Variable Binding
by Justin Johnson 01 Aug '03

01 Aug '03

When I run the following code, everytime the callbacks to "onSuccess" get called, "item" is bound to the last element in "list". Obviously what I want is to have "item" bound to the value it had when I added the callback. How does one handle looping constructs like this in Twisted? def someFunc(): deferreds = [] def onSuccess(results): d = item.doSomethingElseThatReturnsDeferred() return d d = doSomethingThatReturnsDeferred() for item in list: d.addCallbacks(onSuccess, log.err) deferreds.append(d) return defer.DefferedList(deferreds)

2 5

[Twisted-Python] Considering Twisted for OfflineIMAP
by John Goerzen 01 Aug '03

01 Aug '03

Hello, I am the author of OfflineIMAP, a multi-threaded multi-account bi-directional synchronization tool (http://quux.org/devel/offlineimap) written in Python. I am running up against several limitations of the current way I'm doing things. They include: 1. Lack of thread groups in Python, meaning that if there was a network error synchronizing account A, I'd also have to kill off the entire application and the synchronizing threads for account B -- I couldn't just kill of account A. 2. Various bugs in Python's threading implementation 3. imaplib.py being a stinking festering heap of crap leading to unmaintainable code in everything that touches it My first plan was to just rewrite imaplib, which wouldn't be too hard. But someone on the OfflineIMAP list pointed me to Twisted, which I observe already contains an IMAP library that looks nice, plus a different paradigm that may spare me from the whole thread groups question altogether. My questions, then, are these: 1. How does Twisted interact with things like threads and connection pooling? I currently let the user control the maximum number of connections that are open to a given server, and whenever the code needs to access an IMAP server, it grabs a connection from the appropriate pool and uses it (using threading primitives to block if no connection is available.) Can Twisted handle multiple connections to multiple servers doing multiple things at once? Can it do connection pools? If it needs threading to do these things, does it play nice with it? 3. I notice the "Deferred" object and the benefits it provides when a server response is expected to take some time. I assume that some sort of internal threading is taking place here? What if the server *request* takes some time -- say, uploading a 2MB e-mail over a dialup link. Are we still OK with handling that in a non-blocking fashion? 4. The howto "book" alludes to pending improvements on the mail infrastructure. Anything I should be aware of here? 5. Is there any sort of unified mail API in Twisted (like JavaMail) that would present me with a single API to both IMAP and Maildir repositories, or is that something I need to do on my own? (I've already done it, so it's no big deal to do that again) 6. OfflineIMAP supports several different user interfaces (two written with Tkinter, 1 using Curses, and three plain console ones.) I would like to reimplement the Tkinter ones with wxPython, and am glad to see that Twisted works well with this. One concern, though, is that the users can supply a list of UIs to try in the config file: for instance, Tk, Curses, Noninteractive. The system will try each one in turn until it reaches one that works. (For instance, Tk may not work if the user is not in X, and Curses may not work if stdout is not a tty). It looks like this may be problematic with Twisted since the UI selection always seems to be known in advance with the import commands related to the reactor. Any thoughts here? Thanks! -- John Goerzen

6 7

[Twisted-Python] Win32 Service
by Mike Owens 01 Aug '03

01 Aug '03

I am currently running twisted on an XP machine. What is the recommended way of running twistd as a service? Or should I just create a custom web script and run that as a service?

2 1

[Twisted-Python] Twisted and Rendezvous
by screwtape＠froup.com 01 Aug '03

01 Aug '03

A few weeks ago, someone on the list mentioned Rendezvous support for Twisted. Curious, I picked up the Rendezvous specification documents, and got to hacking. What I've got so far is almost half a multicast DNS responder. The multicast DNS specification covers two main areas - asking other hosts for DNS records and serving out DNS records yourself. My code at the moment sends out queries and caches the responses in more or less the proper fashion, but does not respond to requests. The upshot of all this is that you can look things up over multicast DNS, but you can't advertise your own services. The code was written on Linux and tested against Mac OS X 10.2.6 (it'll never run on OS X or any other operating system that includes a Rendezvous service), and seems to Do The Right Thing. You can find a nice tarball here: http://members.optusnet.com.au/thristian/mDNS-0.5.tar.gz It contains the main module (mDNS.py) and a simple API demonstration (mDNS-test.py). There's lots of comments and docstrings, so it shouldn't be too hard to figure things out. If you're interested in Rendezvous support for Python, download it and have a play, then tell me what you think. -- ___________ ___________________________________ | Screwtape | http://livejournal.com/~thristian |______ _____ ___ __ _ _ _ _ | | "At 640x480 the world is peaceful 'cos noone can see well enough to aim" |

2 2

[Twisted-Python] the right way of unit testing protocols
by Jeethu Rao 01 Aug '03

01 Aug '03

What is the right way of unit testing protocols in twisted ? Currently, I'm just subclassing my protocol with unittest.TestCase And directly calling the dataRecieved with a string and am checking If the overridden callbacks (like dataRecieved) are called. The tests pass, but back when I was writing threaded servers, I would actually setup a server on localhost and would connect to The server from another thread, for unit testing. That gives me a lot more confidence. I know that I can setup two reactors, for the client and the server and then try to make them talk, But I'm sure there's a shortcut. I just could'nt figure out the test suites from the Twisted sources. Jeethu Rao

4 6