Mailman 3 September 2006 - Twisted

[Twisted-Python] New OS X buildbot courtesy of Apple Computer, Inc.
by glyph＠divmod.com 23 Sep '06

23 Sep '06

Today a box arrived in the Twisted Matrix Labs global headquarters (AKA the Divmod offices) containing a tricked-out Xserve G5, sent gratis by Apple Computer to help the Twisted team better support the Macintosh platform. Chris Armstrong and JP Calderone have already set up the buildslaves, and you can see them featured on our Buildbot page as "osx-2.4" and "poll-osx-2.4". http://twistedmatrix.com/buildbot/ This new machine is serving its purpose admirably. The new build configuration, poll on osx, has already revealed some bugs that need fixing, less than 20 minutes after the machine was set up :). There are several people who deserve some kudos here: First of all, Wilfredo Sanchez, for both getting Apple to use Twisted on a high-profile project in the first place, and driving the process which culminated in the delivery of this beautiful machine. Thanks to Apple Computer for dedicating these resources to our project. I trust it will prove a worthwhile investment for them. And finally, thanks to Stephen Thorne, who has been tirelessly keeping our tests running on a personal machine, and helping to keep the OS X platform supported until now. Stephen, you can finally turn it off :-). Pictures are forthcoming, if you know how Flickr works and such.

5 4

[Twisted-Python] PB over SSH
by Aron Bierbaum 22 Sep '06

22 Sep '06

Is there currently any way to us PB over a SSH connection? I noticed that there was a post on this list back in Feb 2004, but I didn't see anyone mention it after that. If anyone has any sample code that does this I would greatly appreciate it. -Aron

3 2

[Twisted-Python] Re: [Twisted-commits] r18220 - Make tests clean up after themselves
by Jean-Paul Calderone 22 Sep '06

22 Sep '06

On Thu, 21 Sep 2006 21:32:47 -0600, Jonathan Lange <jml(a)wolfwood.twistedmatrix.com> wrote: >Author: jml >Date: Thu Sep 21 21:32:47 2006 >New Revision: 18220 > >Modified: > trunk/twisted/web/test/test_distrib.py > trunk/twisted/web2/test/test_static.py > trunk/twisted/words/test/test_service.py >Log: >Make tests clean up after themselves > > * Fixes #1883 > * Author: jml > * Reviewer: MFen > >Although not apparent to the naked eye, these tests previous did not >correctly clean up after themselves. Instead, they relied on a couple >of spins of the reactor to do their cleaning for them. This has been >corrected. > > > >Modified: trunk/twisted/web/test/test_distrib.py >============================================================================== >--- trunk/twisted/web/test/test_distrib.py (original) >+++ trunk/twisted/web/test/test_distrib.py Thu Sep 21 21:32:47 2006 >@@ -15,16 +15,26 @@ > self.logFile.close() > del self.logFile > >+ >+class PBServerFactory(pb.PBServerFactory): Docstring? >+ def buildProtocol(self, addr): >+ self.proto = pb.PBServerFactory.buildProtocol(self, addr) >+ return self.proto >+ >+ > class DistribTest(unittest.TestCase): > port1 = None > port2 = None > sub = None > > def tearDown(self): Docstring? >+ dl = [defer.Deferred(), defer.Deferred()] >+ self.f1.proto.notifyOnDisconnect(lambda: dl[0].callback(None)) > if self.sub is not None: >+ self.sub.publisher.broker.notifyOnDisconnect( >+ lambda: dl[1].callback(None)) > self.sub.publisher.broker.transport.loseConnection() > http._logDateTimeStop() >- dl = [] > if self.port1 is not None: > dl.append(self.port1.stopListening()) > if self.port2 is not None: >@@ -36,8 +46,8 @@ > r1 = resource.Resource() > r1.putChild("there", static.Data("root", "text/plain")) > site1 = server.Site(r1) >- f1 = pb.PBServerFactory(distrib.ResourcePublisher(site1)) >- self.port1 = reactor.listenTCP(0, f1) >+ self.f1 = PBServerFactory(distrib.ResourcePublisher(site1)) >+ self.port1 = reactor.listenTCP(0, self.f1) > self.sub = distrib.ResourceSubscription("127.0.0.1", > self.port1.getHost().port) > r2 = resource.Resource() > >Modified: trunk/twisted/web2/test/test_static.py >============================================================================== >--- trunk/twisted/web2/test/test_static.py (original) >+++ trunk/twisted/web2/test/test_static.py Thu Sep 21 21:32:47 2006 >@@ -11,7 +11,6 @@ > self.text = "Hello, World\n" > self.data = static.Data(self.text, "text/plain") > >- > def test_dataState(self): > """ > Test the internal state of the Data object >@@ -42,7 +41,6 @@ > http_headers.MimeType("text", "plain")) > def checkStream(data): > self.assertEquals(str(data), self.text) >- > return stream.readStream(iweb.IResponse(self.data.render(None)).stream, > checkStream) > >@@ -58,7 +56,8 @@ > maxBytes=16) > self.root.addSlash = True > >- def uploadFile(self, fieldname, filename, mimetype, content, resrc=None, host='foo', path='/'): >+ def uploadFile(self, fieldname, filename, mimetype, content, resrc=None, >+ host='foo', path='/'): > if not resrc: > resrc = self.root > >@@ -102,32 +101,33 @@ > return d > > def test_enforcesMaxBytes(self): Docstring? >- return self.assertInResponse(self.uploadFile('FileNameOne', 'myfilename', >- 'text/html', 'X'*32), >- (200, {}, 'exceeds maximum length')) >+ return self.assertInResponse( >+ self.uploadFile('FileNameOne', 'myfilename', 'text/html', 'X'*32), >+ (200, {}, 'exceeds maximum length')) > > def test_enforcesMimeType(self): Docstring? >- return self.assertInResponse(self.uploadFile('FileNameOne', 'myfilename', >- 'application/x-python', 'X'), >- (200, {}, 'type not allowed')) >+ return self.assertInResponse( >+ self.uploadFile('FileNameOne', 'myfilename', >+ 'application/x-python', 'X'), >+ (200, {}, 'type not allowed')) > > def test_invalidField(self): Docstring? >- return self.assertInResponse(self.uploadFile('NotARealField', 'myfilename', >- 'text/html', 'X'), >- (200, {}, 'not a valid field')) >+ return self.assertInResponse( Docstring? >+ self.uploadFile('NotARealField', 'myfilename', 'text/html', 'X'), >+ (200, {}, 'not a valid field')) > > def test_reportFileSave(self): Docstring? >- return self.assertInResponse(self.uploadFile('FileNameOne', 'myfilename', >- 'text/plain', >- 'X'), >- (200, {}, 'Saved file')) >+ return self.assertInResponse( >+ self.uploadFile('FileNameOne', 'myfilename', 'text/plain', 'X'), >+ (200, {}, 'Saved file')) > > def test_compareFileContents(self): Docstring? > def gotFname(fname): > contents = file(fname, 'r').read() > self.assertEquals(contents, 'Test contents') >- >- return self.uploadFile('FileNameOne', 'myfilename', 'text/plain', >- 'Test contents').addCallback( >- self.fileNameFromResponse >- ).addCallback(gotFname) >+ >+ d = self.uploadFile('FileNameOne', 'myfilename', 'text/plain', >+ 'Test contents') >+ d.addCallback(self.fileNameFromResponse) >+ d.addCallback(gotFname) >+ return d > >Modified: trunk/twisted/words/test/test_service.py >============================================================================== >--- trunk/twisted/words/test/test_service.py (original) >+++ trunk/twisted/words/test/test_service.py Thu Sep 21 21:32:47 2006 >@@ -875,21 +875,26 @@ > self.portal = portal.Portal( > self.realm, [self.checker]) > self.serverFactory = pb.PBServerFactory(self.portal) >+ self.serverFactory.protocol = self._protocolFactory > self.serverFactory.unsafeTracebacks = True > self.clientFactory = pb.PBClientFactory() > self.clientFactory.unsafeTracebacks = True >- > self.serverPort = reactor.listenTCP(0, self.serverFactory) > self.clientConn = reactor.connectTCP( > '127.0.0.1', > self.serverPort.getHost().port, > self.clientFactory) > >+ def _protocolFactory(self, *args, **kw): Docstring? >+ self._serverProtocol = pb.Broker(0) >+ return self._serverProtocol > > def tearDown(self): Docstring? >+ d3 = Deferred() >+ self._serverProtocol.notifyOnDisconnect(lambda: d3.callback(None)) > return DeferredList([ > maybeDeferred(self.serverPort.stopListening), >- maybeDeferred(self.clientConn.disconnect)]) >+ maybeDeferred(self.clientConn.disconnect), d3]) > > def _loggedInAvatar(self, name, password, mind): > creds = credentials.UsernamePassword(name, password) > >_______________________________________________ >Twisted-commits mailing list >Twisted-commits(a)twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-commits >

3 5

[Twisted-Python] Perspective Broker vs XMLRPC for this...
by Yoheeb 21 Sep '06

21 Sep '06

Hi all, I have never used twisted, but really would like to. I am in the middle of rough design of an xmlrpc OR OTHER remote method invocation type system. I am debating the use of xmlrpc vs the use of perspective broker. (for a quick prototype I will likely use the python base xmlrpc lib, but just for proof of concept) I need to support both java and python. (and really c/c++ in some fashion would be nice also) from both "local" and remote sources. I really need something that is secure as well, although I can work around security with either via pre-post process encrytption techniques as needed. I will need the ability to expose/hide methods based on the user privledges. I envision a single "services" like tool that populates a list of things you can run based on user rights. For this I need usernames/pwds to somehow be secure (md5, rot13, whatever, just something) I will have connections over vpn, local code calls, and eventually over local serial and usb ports. (assuming everthing looks like a simple IP socket stream) Finally, I have somehting else running a simple xmlrpc server on the system, and may want to propogate calls to that with some magic wrapper function. (to be specific, I am running a copy of eXist, the xmldb) Breathe.... Ok, so with all those rough requirements in mind, will PB be worthwhile? There is no preference from the client applications point of view. Right now I am doing a proof of concept, and have a ton of issues that could be neatly solved by a nice RMI system that is secure. Thanks in advance. Yoheeb

2 1

[Twisted-Python] how to manage multiple services using application
by William Hanwoody 19 Sep '06

19 Sep '06

Hi All: I want to manage multiple services in one twisted.application， multiple services contain t.app.internet.TCPClient services and t.app.internet.TCPServer services. After client service(a.app.internet.TCPClient) connected and get results, Do I need to stop client service? if needed, how to do it? another question is how to use Deferred in twisted.application framework? Thanks -- _______________________________________________________________ William Hanwoody hanwoody(a)gmail.com A senior Linux engineer, like system integrate My blog - hanwoody.livejournal.com _______________________________________________________________ Q: What does it say on the bottom of Coke cans in North Dakota? A: Open other end. _______________________________________________________________

1 0

[Twisted-Python] Foolscap (aka newpb) 0.0.2 released
by Brian Warner 18 Sep '06

18 Sep '06

I'm pleased to announce the release of Foolscap-0.0.2 . (What is 'foolscap', you ask? Good question. It is a size of paper, or the hat worn by a court jester, or a distributed object-capability protocol for Twisted. Only the first two definitions show up in wikipedia, however.) newpb has been moved out of the Twisted SVN subdirectory into a new top-level package where it can be developed independently of the Twisted release cycle. (see ticket #1999 for details). In the process it has been renamed to "Foolscap". (this name has some good properties, but "rolls pleasantly off the tongue" is not one of them, and I'm not completely settled on it yet, so don't be surprised if the name changes again in the next couple of months). Foolscap is the next-generation version of Twisted's native RPC protocol, the venerable Perspective Broker. It lets you make certain methods of certain objects on one machine invokable from other machines. Foolscap (aka newpb) has been in development on and off for, um, years, and is starting to be ready for brave adventurers to give it a spin. Incompatible protocol and API changes are likely to come, so keep in mind that it is not yet suitable for production deployment. The 0.0.2 release does not make any significant behavioral or protocol changes relative to the earlier 0.0.1 release, but it does change the package name and rearranges the preferred entry point. In 0.0.1 you needed to import a number of second-level modules to use newpb (i.e. 'from twisted.pb import pb; tub = pb.PBService()'). In 0.0.2 the foolscap/__init__.py file contains all the important entry points, so you just do 'from foolscap import PBService'. The newpb (aka foolscap) wiki page contains lots of information about what sorts of new features foolscap has relative to PB, links to view the source and protocol specifications, and of course the download tarball: http://twistedmatrix.com/trac/wiki/FoolsCap The release itself is available in my home directory on pyramid: http://twistedmatrix.com/~warner/Foolscap/ The source code lives in Twisted SVN in the sandbox: svn://svn.twistedmatrix.com/svn/Twisted/sandbox/warner/Foolscap/trunk For a quick taste of what it looks like to make objects remotely available via Foolscap, take a look at the simple client+server examples in the docs: http://twistedmatrix.com/trac/browser/sandbox/warner/Foolscap/trunk/doc/lis… http://twistedmatrix.com/trac/browser/sandbox/warner/Foolscap/trunk/doc/lis… Please grab the release and give it a whirl. The more people who use it, the better ideas we'll get about where it needs to go and what development still needs to be done. The docs have a number of files describing the protocol work that remains to be finished, but usability and practical deployment issues are the biggest unknowns right now, and the only way to figure those out is to actually use it for something. share and enjoy, -Brian

1 0

[Twisted-Python] manhole and exceptions
by Manlio Perillo 17 Sep '06

17 Sep '06

Hi. I have started to play with manhole (on Windows XP, using putty as the client). After doing a: >>> raise SystemExit() from the terminal, the prompt ('>>>') does not appear. Moreover every time I hit [RETURN] the same exception is re-raised in the program. I have to hit Ctrl-C to obtain the prompt again. Thanks and regards Manlio Perillo

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 17 Sep '06

17 Sep '06

Bug summary ______________________ Summary for 2006-09-10 through 2006-09-17 Bugs opened: 32 Bugs closed: 20 Total open bugs: 730 (+12) |== Type Changes |== Priority Changes |== Component Changes |Defect: +2 |Highest: +6 |Conch: -2 |Enhancement: +11 |High: +1 |Core: +5 |Task: +3 |Normal: +8 |Mail: +1 |Low: -1 |Pb: +2 |Lowest: +2 |Trial: +3 |Web2: +2 |Web2.Dav: +0 |Website: +2 |Words: +3 New / Reopened Bugs ______________________ ===== Highest ===== [#2074] Access to twisted request object in WSGI applications (opened by martinpaljak) enhancement web2 http://twistedmatrix.com/trac/ticket/2074 [#2076] twisted from svn will traceback when run on platform without working xml.dom.minidom (opened by jerub) (CLOSED, duplicate) defect core http://twistedmatrix.com/trac/ticket/2076 [#2079] Deprecate twisted.python.reflect.funcinfo (opened by radix) task core http://twistedmatrix.com/trac/ticket/2079 [#2083] Add test for statusForException() (opened by wsanchez) enhancement web2.dav http://twistedmatrix.com/trac/ticket/2083 [#2086] t.words.basesupport attributeerror when "already connecting" (opened by moonfallen) defect words http://twistedmatrix.com/trac/ticket/2086 [#2089] twisted.words.protocol.irc.IRC.whois references undefined variable RPL_WHOISOPER, (opened by jerub) defect words http://twistedmatrix.com/trac/ticket/2089 [#2091] Make reactor unclean warnings reported errors (opened by jml) enhancement trial http://twistedmatrix.com/trac/ticket/2091 [#2054] ProperlyCloseFilesTestCase.testProperlyCloseFiles written in an aggressively stupid manner (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/2054 ===== High ===== [#2084] Review public API of web2 (opened by glyph) enhancement web2 http://twistedmatrix.com/trac/ticket/2084 [#2090] Deprecate reactor methods in tests (opened by jml) task trial http://twistedmatrix.com/trac/ticket/2090 ===== Normal ===== [#2070] cr+lf replacement in pop3.py/transformChunk() (opened by kaimorich) (CLOSED, duplicate) enhancement mail http://twistedmatrix.com/trac/ticket/2070 [#2071] cr+lf replacement in pop3.py/transformChunk() (opened by kaimorich) (CLOSED, wontfix) defect mail http://twistedmatrix.com/trac/ticket/2071 [#2072] Document newline convention required by twisted.mail.pop3.IMailbox.getMessage (opened by exarkun) enhancement mail http://twistedmatrix.com/trac/ticket/2072 [#2073] Access to twisted request object in WSGI applications (opened by martinpaljak) (CLOSED, duplicate) enhancement web2 http://twistedmatrix.com/trac/ticket/2073 [#2075] set SCRIPT_NAME (properly?) in CGI environment (opened by martinpaljak) defect web2 http://twistedmatrix.com/trac/ticket/2075 [#2077] Add a report for "Tickets Pending Review" (opened by jml) enhancement website http://twistedmatrix.com/trac/ticket/2077 [#2078] Twisted shouldn't start processes until the reactor is running (opened by radix) enhancement core http://twistedmatrix.com/trac/ticket/2078 [#2080] (Select) reactor doesn't wake up as a result of callFromThread (opened by mwk) (CLOSED, worksforme) enhancement core http://twistedmatrix.com/trac/ticket/2080 [#2081] Linux filesystem extended attributes require a "user." prefix (opened by wsanchez) (CLOSED, fixed) enhancement web2.dav http://twistedmatrix.com/trac/ticket/2081 [#2082] web2.dav.fileop should avoid using the os module (opened by wsanchez) defect web2.dav http://twistedmatrix.com/trac/ticket/2082 [#2085] Declare twisted.web2 stable and supported, de-emphasize twisted.web (opened by glyph) enhancement web2 http://twistedmatrix.com/trac/ticket/2085 [#2087] buffer size doubling for iocpreactor (opened by ghazel) enhancement core http://twistedmatrix.com/trac/ticket/2087 [#2088] twisted.words.im.basechat.ChatUI.contactChangedNick is completely broken. (opened by jerub) enhancement words http://twistedmatrix.com/trac/ticket/2088 [#2092] Remove reactor.iterate calls from do_cleanPending (opened by jml) enhancement trial http://twistedmatrix.com/trac/ticket/2092 [#2093] URLPath should parse netloc more thoroughly. (opened by dreid) enhancement core http://twistedmatrix.com/trac/ticket/2093 [#2094] URLPath should provide an API for manipulating query arguments (opened by dreid) enhancement core http://twistedmatrix.com/trac/ticket/2094 [#2095] Add WebDAV quota support (opened by wsanchez) enhancement web2.dav http://twistedmatrix.com/trac/ticket/2095 [#2097] Web Site Maintenance Info (opened by oubiwann) task website http://twistedmatrix.com/trac/ticket/2097 [#133 ] Allow registration of adapters for PB serialization (opened by itamarst) enhancement pb http://twistedmatrix.com/trac/ticket/133 [#1502] call selectreactor._preenDescriptors for WSAENOTSOCK (opened by titty) defect core http://twistedmatrix.com/trac/ticket/1502 ===== Lowest ===== [#2096] Support binding extensions to WebDAV (opened by wsanchez) enhancement web2.dav http://twistedmatrix.com/trac/ticket/2096 [#1997] perhaps wakeUp could be slightly simpler (opened by wingo) defect core http://twistedmatrix.com/trac/ticket/1997 Closed Bugs ______________________ ===== Highest ===== [#2054] ProperlyCloseFilesTestCase.testProperlyCloseFiles written in an aggressively stupid manner (opened by exarkun, closed by exarkun, ) defect core http://twistedmatrix.com/trac/ticket/2054 [#2076] twisted from svn will traceback when run on platform without working xml.dom.minidom (opened by jerub, closed by jerub, duplicate) defect core http://twistedmatrix.com/trac/ticket/2076 [#1850] web2 documentation should recommend putChild instead of setattr (opened by marienz, closed by dreid, fixed) enhancement web2 http://twistedmatrix.com/trac/ticket/1850 ===== High ===== [#2033] Removing non-existent property results in error. (opened by rfmohr(a)osc.edu, closed by wsanchez, fixed) defect web2.dav http://twistedmatrix.com/trac/ticket/2033 ===== Normal ===== [#2070] cr+lf replacement in pop3.py/transformChunk() (opened by kaimorich, closed by kaimorich, duplicate) enhancement mail http://twistedmatrix.com/trac/ticket/2070 [#2071] cr+lf replacement in pop3.py/transformChunk() (opened by kaimorich, closed by exarkun, wontfix) defect mail http://twistedmatrix.com/trac/ticket/2071 [#2073] Access to twisted request object in WSGI applications (opened by martinpaljak, closed by martinpaljak, duplicate) enhancement web2 http://twistedmatrix.com/trac/ticket/2073 [#1848] Python 2.5b1 svn (opened by scmikes, closed by radix, invalid) enhancement core http://twistedmatrix.com/trac/ticket/1848 [#1854] Suse 10, Python 2.5 32 bit Twisted SVN (opened by scmikes, closed by radix, invalid) enhancement core http://twistedmatrix.com/trac/ticket/1854 [#1572] twistd starts services synchronously: it should start them using the startup system event. (opened by exarkun, closed by radix, wontfix) defect core http://twistedmatrix.com/trac/ticket/1572 [#2067] string formatting bug in "twisted.python.reflect.funcinfo" (opened by james.townson, closed by radix, wontfix) defect core http://twistedmatrix.com/trac/ticket/2067 [#1501] Make twistd a library (opened by radix, closed by radix, wontfix) enhancement conch http://twistedmatrix.com/trac/ticket/1501 [#2080] (Select) reactor doesn't wake up as a result of callFromThread (opened by mwk, closed by mwk, worksforme) enhancement core http://twistedmatrix.com/trac/ticket/2080 [#1825] web2.dav COPY and MOVE methods fail if root directory of DAV resource is unwritable (opened by troy(a)osc.edu, closed by wsanchez, fixed) defect web2.dav http://twistedmatrix.com/trac/ticket/1825 [#2081] Linux filesystem extended attributes require a "user." prefix (opened by wsanchez, closed by wsanchez, fixed) enhancement web2.dav http://twistedmatrix.com/trac/ticket/2081 [#1807] Exception handling improvements in web2/dav (opened by troy(a)osc.edu, closed by wsanchez, fixed) enhancement web2.dav http://twistedmatrix.com/trac/ticket/1807 [#1838] DAV MKCOL method with body causes uncaught exception (opened by troy(a)osc.edu, closed by wsanchez, fixed) defect web2.dav http://twistedmatrix.com/trac/ticket/1838 [#1752] t.w2's header tests that use runRoundtripTest and requiredGeneratedElements are invalid. (opened by dreid, closed by dreid, wontfix) defect web2 http://twistedmatrix.com/trac/ticket/1752 ===== Low ===== [#1203] web2.stream.IStream.read() needs to take a length argument (opened by PenguinOfDoom, closed by dreid, invalid) enhancement conch http://twistedmatrix.com/trac/ticket/1203 ===== Lowest ===== [#1997] perhaps wakeUp could be slightly simpler (opened by wingo, closed by glyph, ) defect core http://twistedmatrix.com/trac/ticket/1997

1 0

[Twisted-Python] Re: 2 questions: adbapi.ConnectionPool ; Defer.DeferredSemaphore , threads.deferToThread
by Ed Suominen 15 Sep '06

15 Sep '06

> specifically, i'm trying to have a scheduled job queue seed 1-50 > 'tasks' at once, each one using a seperate DB connection and running > in a seperate thread. > once they all complete, the job reschedules itself to run in the future. I suggest you take a look at the asynchronous task queue on which my sAsync package is based. I've been lobbying to get it included into the twisted code base, see http://twistedmatrix.com/trac/ticket/1768. The latest SVN of the module is at http://foss.eepatents.com/sAsync/browser/trunk/sasync/taskqueue.py and the API document is at http://foss.eepatents.com/api/sasync/sasync.taskqueue.html. Unlike the rest of sAsync, which is GPL'd, the taskqueue module and its unit test module are distributed under a variant of the MIT license in order to be suitable for inclusion in the Twisted codebase. Best regards, Ed

1 0

[Twisted-Python] Rotating logfiles (SIGUSR1 doesn't work)
by Christian Schneider 15 Sep '06

15 Sep '06

Hi, we've written a server running as a twisted application. Logging is done via twisted.python.log.msg. The log file is growing rather big and only rotated on server startup, which hopefully is a very rare occurrence, indeed. Sending a SIGUSR1 to twistd unfortunately doesn't do anything. I've seen that the class LogFile has a rotate method but I don't see the connection between twisted.python.log and that class if there is any at all. Any hint would be highly appreciated chris

3 2