Mailman 3 March 2007 - Twisted

[Twisted-Python] using stdio with Twisted
by Neurophyre 20 Mar '07

20 Mar '07

Hello, I'm building an old-school BBS client using Twisted. It needs to take user input, parse it, send messages over the wire and get messages back and display them.. the usual. I'm just now getting to writing the UI and have run into a couple problems. First, whether using protocol.Protocol or protocols.basic.LineReceiver (in raw mode) with internet.stdio.StandardIO, I have the same problem: the dataReceived (or rawDataReceived in the case of LineReceiver in raw mode) functions are only called after one types some text and hits return. I need to process input on a keystroke level, since there will be prompts at which a single keystroke will do things. Second, I was looking at twisted.conch.insults.insults.TerminalProtocol after some Googling -- does anyone know if I can just drop this in in place of protocols.basic.LineReceiver (or whatever) and have it work properly, or does it require some special invocation and/or grab stdio by itself? Thanks!

2 4

[Twisted-Python] Inheriting t.a.service.Service and super
by Sébastien LELONG 20 Mar '07

20 Mar '07

Hi, I've been using Twisted for several months, and have frequently encountered the following problem. In my Twisted apps, I declare services (t.a.service.Service), which then I need to subclass. The problem is I can't call super() on such subclass, since the class Service in twisted does not inherit the built-in 'object' type ("TypeError: super() argument 1 must be type, not classobj"). For now, I can use other form of superclass calling, but that can be error prone. Is there any particular reason why class Service isn't a subclass of object ? It's kind of surprising since Service is designed to be subclassed. Thanks in advance for your help/replies. Cheers, Seb -- Sébastien LELONG sebastien.lelong[at]sirloon.net http://www.sirloon.net

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 17 Mar '07

17 Mar '07

Bug summary ______________________ Summary for 2007-03-11 through 2007-03-18 Bugs opened: 12 Bugs closed: 11 Total open bugs: 879 (+1) |== Type Changes |== Priority Changes |== Component Changes |Defect: +0 |Highest: -4 |Conch: +1 |Enhancement: +2 |High: +3 |Core: -4 |Normal: +2 |Lore: +3 |Lowest: +1 |Mail: +1 |Trial: +0 |Web: +1 |Web2: +1 |Words: -1 New / Reopened Bugs ______________________ ===== Highest ===== [#2521] SMTP authentication tracebacks when receiving an empty line (opened by washort) defect mail http://twistedmatrix.com/trac/ticket/2521 [#2523] portal can't check for inherited interfaces (opened by therve) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2523 [#2430] web.xmlrpc returns response with "text/html" content-type when xmlrpc method does not exist. (opened by Remi) (CLOSED, fixed) defect web http://twistedmatrix.com/trac/ticket/2430 ===== High ===== [#2525] Improve Lore's automated test coverage to UQDS standards (opened by glyph) enhancement lore http://twistedmatrix.com/trac/ticket/2525 [#2526] Integrate Lore patches which do index generation for ThinkCSPy (opened by glyph) enhancement lore http://twistedmatrix.com/trac/ticket/2526 [#2527] Lore Index output for LaTeX (opened by glyph) enhancement lore http://twistedmatrix.com/trac/ticket/2527 ===== Normal ===== [#2517] Xish XPath event observers not tested (opened by exarkun) defect words http://twistedmatrix.com/trac/ticket/2517 [#2518] t.w.xmlrpc.QueryProtocol does not close the connection after handling the request (opened by karlb) defect web http://twistedmatrix.com/trac/ticket/2518 [#2519] Couple of conch bug-fixes (opened by johnnypops) defect conch http://twistedmatrix.com/trac/ticket/2519 [#2520] twisted.mail.pop3client.POP3Client does not allow plain text login (opened by yasusii) (CLOSED, fixed) defect mail http://twistedmatrix.com/trac/ticket/2520 [#2522] HTTPAuthResource: 'ValueError: too many values to unpack' when IRealm.requestAvatarId returns a logout callable (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/2522 ===== Lowest ===== [#2524] Support for triggering trial's waiting for Deferreds from within doctests (opened by strank) enhancement trial http://twistedmatrix.com/trac/ticket/2524 Closed Bugs ______________________ ===== Highest ===== [#1652] xish' EventDispatcher grinds to a halt when using many different one-time observers (opened by ralphm, closed by ralphm, fixed) defect words http://twistedmatrix.com/trac/ticket/1652 [#2476] local variable 'cbl' referenced before assignment (opened by jack, closed by ralphm, fixed) defect words http://twistedmatrix.com/trac/ticket/2476 [#2419] t.i.fdesc improvements (opened by itamarst, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2419 [#2005] trial twisted/mail/ traces back, omitting trailing / does not. (opened by jerub, closed by therve, fixed) defect trial http://twistedmatrix.com/trac/ticket/2005 [#2430] web.xmlrpc returns response with "text/html" content-type when xmlrpc method does not exist. (opened by Remi, closed by therve, fixed) defect web http://twistedmatrix.com/trac/ticket/2430 [#2430] web.xmlrpc returns response with "text/html" content-type when xmlrpc method does not exist. (opened by Remi, closed by therve, fixed) defect web http://twistedmatrix.com/trac/ticket/2430 [#2523] portal can't check for inherited interfaces (opened by therve, closed by therve, fixed) defect core http://twistedmatrix.com/trac/ticket/2523 [#2463] add tests for logging of unhandled deferred errors (opened by antoine, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2463 ===== Normal ===== [#2515] "python setup.py build" fails for Twisted (opened by edwards, closed by therve, fixed) defect core http://twistedmatrix.com/trac/ticket/2515 [#2520] twisted.mail.pop3client.POP3Client does not allow plain text login (opened by yasusii, closed by yasusii, fixed) defect mail http://twistedmatrix.com/trac/ticket/2520 [#1739] (patch) Removing system event triggers while firing does the wrong thing. (opened by antony, closed by glyph, duplicate) defect core http://twistedmatrix.com/trac/ticket/1739

1 0

[Twisted-Python] Game using twisted: Castle-Combat
by Karl Bartel 17 Mar '07

17 Mar '07

Hi! I've written a game which could be added to http://twistedmatrix.com/ trac/wiki/ProjectsUsingTwisted, if it qualifies as successful. Name: Castle-Combat URL: http://linux-games.com/castle-combat Desc: a multiplayer game designed after the arcade game "Rampart" Thanks, Karl

2 1

[Twisted-Python] Foolscap-0.1.0 released
by Brian Warner 16 Mar '07

16 Mar '07

I made a new release of Foolscap last night, available in the usual place at http://twistedmatrix.com/trac/wiki/FoolsCap . This release makes a few usability improvements for Foolscap-based servers, and fixes a bug that made it impossible to use a Tub right after creating it for the first time (if that system's clock was faster than the system being connected to). The Foolscap-0.1.0 release tarball is signed by my GPG public key (0x1514A7BD) and available from: http://twistedmatrix.com/~warner/Foolscap/foolscap-0.1.0.tar.gz http://twistedmatrix.com/~warner/Foolscap/foolscap-0.1.0.tar.gz.sig There are no protocol changes between 0.1.0 and the previous 0.0.7 release, just bugfixes and API additions. There are still protocol changes in the works, though. Why call this version 0.1.0? Because this release is actually kind of useful for something. What is Foolscap? From the README: This is a ground-up rewrite of Perspective Broker, which itself is Twisted's native RPC/RMI protocol (Remote Procedure Call / Remote Method Invocation). If you have control of both ends of the wire, and are thus not constrained to use some other protocol like HTTP/XMLRPC/CORBA/etc, you might consider using Foolscap. Fundamentally, Foolscap allows you to make a python object in one process available to code in other processes, which means you can invoke its methods remotely. This includes a data serialization layer to convey the object graphs for the arguments and the eventual response, and an object reference system to keep track of which objects you are connecting to. It uses a capability-based security model, such that once you create a non-public object, it is only accessible to clients to whom you've given the (unguessable) PB-URL. You can of course publish world-visible objects that have well-known PB-URLs. The complete release notes for 0.1.0 are attached below. share and enjoy, -Brian * Release 0.1.0 (15 Mar 2007) ** usability improvements *** Tubs now have a certFile= argument A certFile= argument has been added to the Tub constructor to allow the Tub to manage its own certificates. This argument provides a filename where the Tub should read or write its certificate. If the file exists, the Tub will read the certificate data from there. If not, the Tub will generate a new certificate and write it to the file. The idea is that you can point certFile= at a persistent location on disk, perhaps in the application's configuration or preferences subdirectory, and then not need to distinguish between the first time the Tub has been created and later invocations. This allows the Tub's identity (derived from the certificate) to remain stable from one invocation to the next. The related problem of how to make (unguessable) object names persistent from one program run to the next is still outstanding, but I expect to implement something similar in the future (some sort of file to which object names are written and read later). certFile= is meant to be used somewhat like this: where = os.path.expanduser("~/.myapp.cert") t = Tub(certFile=where) t.registerReference(obj) # ... *** All eventual-sends are retired on each reactor tick, not just one. Applications which make extensive use of the eventual-send operations (in foolscap.eventual) will probably run more smoothly now. In previous releases, the _SimpleCallQueue class would only execute a single eventual-send call per tick, then take care of all pending IO (and any pending timers) before servicing the next eventual-send. This could probably lead to starvation, as those eventual-sends might generate more work (and cause more network IO), which could cause the event queue to grow without bound. The new approach finishes as much eventual-send work as possible before accepting any IO. Any new eventual-sends which are queued during the current tick will be put off until the next tick, but everything which was queued before the current tick will be retired in the current tick. ** bug fixes *** Tub certificates can now be used the moment they are created In previous releases, Tubs were only willing to accept SSL certificates that created before the moment of checking. If two systems A and B had unsynchronized clocks, and a Foolscap-using application on A was run for the first time to connect to B (thus creating a new SSL certificate), system B might reject the certificate because it looks like it comes from the future. This problem is endemic in systems which attempt to use the passage of time as a form of revocation. For now at least, to resolve the practical problem of certificates generated on demand and used by systems with unsynchronized clocks, Foolscap does not use certificate lifetimes, and will ignore timestamps on the certificates it examines.

1 0

[Twisted-Python] Learning about IPushProducer
by Rutt, Benjamin 16 Mar '07

16 Mar '07

When running the following code (my 2nd twisted program!), it works as I had hoped - it doesn't starve any clients that want to receive data back, even with a simultaneously active really long streaming server-to-client communication (i.e. one piggy client asking for millions of bytes). i.e. another client can get in and ask for just a few bytes while a large payload is being delivered to a different client. Which is great! Here's a sample interaction from the client side: $ telnet localhost 8007 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 1 x 2 xx 3 xxx 10 xxxxxxxxxx 99999 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [...lots of x's...] xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx bye Connection closed by foreign host. $ So I have 2 questions on my code: 1) am I doing anything wrong in setting up the plumbing? 2) does pauseProducing() get called by another thread whilst resumeProducing() is running? (I believe it must, otherwise my resumeProducing() would only be entered once). If so I should have an appropriate mutex around the read/write of self.pause, no? Here is the code, and output from the server is at the end. Thanks -- Benjamin #!/usr/bin/env python import os, os.path, sys, re, commands, pickle, tempfile, getopt, datetime import socket, string, random, time, traceback, shutil, popen2 from zope.interface import implements from twisted.internet import protocol, defer, interfaces, error, reactor from twisted.internet.protocol import Protocol, Factory from twisted.protocols.basic import LineReceiver class NonStarvingXGiver: implements(interfaces.IPushProducer) def __init__(self, howmany, consumer): self.howmany = howmany self.sent_already = 0 self.paused = False self.consumer = consumer def beginSendingXs(self): self.deferred = deferred = defer.Deferred() self.consumer.registerProducer(self, False) return deferred def pauseProducing(self): print 'pauseProducing: invoked' self.paused = True def resumeProducing(self): print 'resumeProducing: invoked' self.paused = False maxchunksz = 1024 while not self.paused and self.howmany > self.sent_already: chunksz = min(maxchunksz, self.howmany - self.sent_already) self.consumer.write('x' * chunksz) self.sent_already += chunksz if self.howmany == self.sent_already: self.consumer.write('\n') self.consumer.unregisterProducer() print 'resumeProducing: exiting for the last time' def stopProducing(self): print 'stopProducing: invoked' self.consumer.unregisterProducer() class xgiver(LineReceiver): def lineReceived(self, howmany): print 'got line [%s] from client [%s]' % (howmany, self.transport.getPeer()) if howmany == 'bye': print 'goodbye to', self.transport.getPeer() self.transport.loseConnection() return try: howmany = int(howmany) s = NonStarvingXGiver(howmany, self.transport) s.beginSendingXs() except Exception, ex: self.transport.write("invalid input " + howmany + "\n") # Next lines are magic: factory = Factory() factory.protocol = xgiver # 8007 is the port you want to run under. Choose something >1024 reactor.listenTCP(8007, factory) reactor.run() ------------------------------------------------------------------- Server output: $ ./xgiver.py got line [1] from client [IPv4Address(TCP, '127.0.0.1', 51007)] resumeProducing: invoked resumeProducing: exiting for the last time got line [2] from client [IPv4Address(TCP, '127.0.0.1', 51007)] resumeProducing: invoked resumeProducing: exiting for the last time got line [3] from client [IPv4Address(TCP, '127.0.0.1', 51007)] resumeProducing: invoked resumeProducing: exiting for the last time got line [10] from client [IPv4Address(TCP, '127.0.0.1', 51007)] resumeProducing: invoked resumeProducing: exiting for the last time got line [99999] from client [IPv4Address(TCP, '127.0.0.1', 51007)] resumeProducing: invoked pauseProducing: invoked resumeProducing: invoked resumeProducing: exiting for the last time got line [bye] from client [IPv4Address(TCP, '127.0.0.1', 51007)] goodbye to IPv4Address(TCP, '127.0.0.1', 51007)

2 7

[Twisted-Python] problem running ldaptor-webui
by Michael Torrie 16 Mar '07

16 Mar '07

I am using python 2.4 on FC6, and Twisted 2.5.0. I have installed the latest stable versions from source of Nevow, Webut, and ldaptor (0.0.43). I tried to run the ldaptor-webui program, but encountered the following traceback: Traceback (most recent call last): File "/usr/bin/ldaptor-webui", line 4, in ? from ldaptor.apps.webui import main File "/usr/lib/python2.4/site-packages/ldaptor/apps/webui/main.py", line 5, in ? from ldaptor.config import LDAPConfig File "/usr/lib/python2.4/site-packages/ldaptor/config.py", line 4, in ? from ldaptor import interfaces File "/usr/lib/python2.4/site-packages/ldaptor/interfaces.py", line 3, in ? class ILDAPEntry(components.Interface): AttributeError: 'module' object has no attribute 'Interface' Apparently this stems from something related to Zope. The imported module in question, <module 'twisted.python.components' from '/usr/lib/python2.4/site-packages/twisted/python/components.pyc'>, indeed has no attribute Interface. However it does import "interface", module <module 'zope.interface.interface' from '/usr/lib/python2.4/site-packages/zope/interface/interface.pyc'>. Is ldaptor-webui in need of updating to run with something new that either Twisted or Zope changed? thanks. Michael

3 4

[Twisted-Python] running a TurboGears from Twisted
by David Konerding 15 Mar '07

15 Mar '07

Hi, I want to run a turbogears site from inside Twisted via WSGI. This isn't documented, and I'm struggling to get it to work. This is what I have so far: from twisted.web2 import wsgi from twisted.application import service, strports from twisted.web2 import server, channel from twisted.python import util from turbogears import update_config, start_server from twisted.web2.wsgi import WSGIResource as WSGI from cherrypy._cpwsgi import wsgiApp import cherrypy cherrypy.lowercase_api = True # This part gets run when you run this file via: "twistd -noy demo.py" if __name__ == '__builtin__': update_config(configfile="dev.cfg",modulename="quickie.config") test = WSGI(wsgiApp) # Create the site and application objects site = server.Site(test) application = service.Application("demo") # Serve it via standard HTTP on port 8080 s = strports.service('tcp:8080', channel.HTTPFactory(site)) s.setServiceParent(application) I'm not having much luck with this, and I thought I'd check on the list if anybody already had a recipe for this. Thanks, Dave

1 0

[Twisted-Python] newbe twisted xmpp load
by wrobyn＠cox.net 15 Mar '07

15 Mar '07

Hello, I am attempting to write an xmpp load testing thingy in python. I am hitting big walls with regard to creating concurrent connections above 250.. I need really 1-n concurrent connections, the upper limit would be around 500,000 but I would be happy at this point breaching the 250 ceiling. Would twisted be able to handle this? Can someone post a simple loop or routine to log in 1-n "people" (i am using the increment in a counter as username) into a jabber server? Thanks for the help renee

4 11

[Twisted-Python] Migrate Asyncore to Twisted
by Sean Roark 15 Mar '07

15 Mar '07

I would love to migrate my asyncore based applications to Twisted but the learning curve seems quite steep. Anybody have a good tutorial? Migrating asyncore.dispatcher objects to twisted protocols examples would be awesome. -- Sean Roark

6 11