Mailman 3 April 2006 - Twisted

[Twisted-Python] twisted ftp client
by John Pote 20 Apr '06

20 Apr '06

Hi, I'm new to twisted and interested in using the ftp client module. But apart from the api reference I stumbled across did not find any docs on the ftp module. Are there any? It would be helpful if there was a doc outlining the use of the module, order in which the various methods should be called/not called, example use of the module etc. Twisted looks highly appropriate for my project but without appropriate documentation I'm reluctant to use it. Many thanks, John Pote

2 1

[Twisted-Python] Documentation download
by Luke Dunstan 20 Apr '06

20 Apr '06

Hi, I installed Twisted_NoDocs-2.1.0.win32-py2.4.exe. The installer seems to have put a couple of broken shortcuts on my start menu pointing to a directory TwistedDocs that doesn't exist. For example: C:\Python24\lib\site-packages\TwistedDocs\howto\index.xhtml The real question though is: where can I download the HTML documentation for this release? Luke

3 4

[Twisted-Python] ReverseProxy with additional headers
by Pawel Lewicki 20 Apr '06

20 Apr '06

Hello, I try to make a simple reverse proxy service that would add authentication info for the Plone site. Is it a good direction or not? I'm not sure if I modify outgoing or incoming headers. Pawel ---------------------- from twisted.internet import reactor from twisted.web import proxy, server class MyReverseProxyResource(proxy.ReverseProxyResource): def render(self, request): request.setHeader('__ac_name','username') request.setHeader('__ac_password','password') return proxy.ReverseProxyResource.render(self, request) def getChild(self, path, request): return MyReverseProxyResource(self.host, self.port, self.path+'/'+path) site = server.Site(MyReverseProxyResource('localhost', 80, '/')) reactor.listenTCP(8080, site) reactor.run()

1 0

[Twisted-Python] error "limits.h missing" while compiling zope interface in Sumo on mac os x 10.4.6 Py 2.4
by Nitin Borwankar on Twisted-Python list 20 Apr '06

20 Apr '06

Hello Twisted gurus, Apologies if this is a FAQ - couldn't find answers in the obvious places. ( list archives, google, ...). I've been trying to build the TwistedSumo-2006-02-12 package - especially the ZopeInterface package within, and am getting the following errors about limits.h not being found. This is on Mac OS X 10.4.6 on a PPC powerbook G4 with gcc 4.0, I tried swicthing to 3.3 via gcc_select and got a different error about gcc command line switches. An 'ls' on /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/ indicates that indeed there's no limits.h there. Separately downloading earlier stable ZopeInterface version and building gave exactly the same error. Python 2.4 is the default python installed via the MacPython 2.4 dmg. An older Python 2.3 also exists on the system but 2.4 is ahead in the PATH. Any help getting past this appreciated. Nitin Borwankar gcc 4.0 errors ----------------- nitin-pbook:~/dl/TwistedSumo-2006-02-12/ZopeInterface-3.1.0c1 nitin$ sudo python setup.py install running install running build running build_py running build_ext building 'zope.interface._zope_interface_coptimizations' extension gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -IDependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c Dependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface/_zope_interface_coptimizations.c -o build/temp.macosx-10.4-fat-2.4/Dependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface/_zope_interface_coptimizations.o In file included from Dependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface/_zope_interface_coptimizations.c:15: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:18:20: error: limits.h: No such file or directory /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:21:2: error: #error "Something's broken. UCHAR_MAX should be defined in limits.h." [...] many more errors which I guess stem from limits.h not being found gcc 3.3 errors ----------------- nitin$ sudo gcc_select 3.3 Password: Default compiler has been set to: gcc version 3.3 20030304 (Apple Computer, Inc. build 1809) nitin-pbook:~/dl/TwistedSumo-2006-02-12/ZopeInterface-3.1.0c1 nitin$ sudo python setup.py install running install running build running build_py running build_ext building 'zope.interface._zope_interface_coptimizations' extension gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -IDependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c Dependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface/_zope_interface_coptimizations.c -o build/temp.macosx-10.4-fat-2.4/Dependencies/zope.interface-ZopeInterface-3.1.0c1/zope.interface/_zope_interface_coptimizations.o gcc: cannot specify -o with -c or -S and multiple compilations error: command 'gcc' failed with exit status 1 -- Nitin Borwankar nitin(a)borwankar.com

2 1

[Twisted-Python] win32 now an officially supported platform
by glyph＠divmod.com 19 Apr '06

19 Apr '06

As of r16628, 'trial twisted' on a stock win32 twisted installation will actually succeed and give you a PASSED at the end of it. I went to a lot of trouble to get things into this state, so I would appreciate it if everyone could keep it this way. More accurately: I will personally revert any changes that cause real failures on win32. I say "real" failures because (as the buildbot history immediately following this commit amply shows) the win32 buildbot itself seems kind of flaky; some of this may have to do with the slave itself running a released version of Twisted, I'm not sure. Any other buildbot enforcers / reviewers should be aware that it fails intermittently and give it a few chances to build to make sure the failure is legitimate, especially if the failure is a timeout or a buildbot disconnect. Finally, I'd like to thank everyone who provided me with help, reviews, commentary, and test fixes to get to this state, especially jerub, exarkun, foom, and jml. Now, let's get a new release out so we can upgrade that buildbot :).

2 1

[Twisted-Python] deferred TCP writes
by Doug Fort, Consulting Programmer 19 Apr '06

19 Apr '06

Hi, I have a server that just continously pushes blocks of data to clients after they connect. I'd like to defer transport.write(), and then send another block of data when it completes. All the samples and documentation I hae seen show synchronous writes. Is there an async. version? -- Doug Fort, Consulting Programmer http://www.dougfort.com

5 5

Re: [Twisted-Python] .tap and twistd and persistence
by glyph＠divmod.com 19 Apr '06

19 Apr '06

On Mon, 17 Apr 2006 12:34:56 -0400, Matt Feifarek <matt.feifarek+twisted(a)gmail.com> wrote: >So, if the docs tell us that this way is the way to use twisted, and they're >wrong, where do we go to learn how to use twisted? Apparently you've come to the right place. :-) >The docs are quite clear about using application objects and making your >code into a twisted 'plugin' rather than simply poking at the reactor (as in >some of the quick-n-dirty tutorials). I have to apologize for that; .tap plugins, in particular, are my fault. Sometimes, a system will be written, documented, people will use it for a while, and it will turn out that the supposed benefits of putting up with the pain of a plugin interface or a layer of indirection are actually not worthwhile. There is chaos for a while as everyone tries to figure out what the best way to solve the problem that the interface originally wanted to solve is. I wish Twisted didn't have this problem, but it's hardly unique. The difference is, we admit we have the problem, let our users know we are reconsidering our solution, and attempt to discover the correct way :-). (For a counterexample, see the web stuff in J2EE. The XML addicts who designed it refused to admit they were wrong or that the user community was having trouble with the mountains of configuration required, and now the "right" way to use J2EE for Web seems to be ditching it in favor of Ruby on Rails...) Right now the appropriate thing to do is use a .tac file. Some of what the documentation says is in fact correct: you still shouldn't poke at the reactor directly: just create the Application object (as documented in those howtos) in a .tac configuration file rather than an axiom plugin. One of the major enhancements we need to make (this will probably be seen in a Twisted 3.0 release) is to make a publicly-supported version of twisted.scripts.twistd.ServerOptions. If you are uncomfortable reading grotty platform-specific code and getting into spirited arguments on this mailing list about broken compatibility from release to release, that is not yet the class for you. Eventually we will provide a nice way to make a twistd-like command-line program (and hopefully also a similar GUI-friendly equivalent) that adds additional arguments and startup logic of your choosing. However, some projects written by Twisted core team members are already abusing the private ServerOptions interface to provide this same effect. One such project is something that I wrote, Divmod Axiom ( http://divmod.org/trac/wiki/DivmodAxiom ). Axiom abuses this internal interface to implement a rather nice mechanism that actually accomplishes what .tap plugins set out to do. Axiom is an object database with native support for the Twisted programming model. The feature I am talking about specifically here though, is a nifty command-line tool called 'axiomatic'. Axiomatic provides a plugin architecture, like tap plugins, which perform manipulations on a database, and a "start" sub-command, which allows you to do things like this: % axiomatic -d mymantissa.axiom mantissa Enter Divmod™ Mantissa™ password for 'admin@localhost': Confirm Divmod™ Mantissa™ password for 'admin@localhost': % axiomatic -d mymantissa.axiom start -n The first command would be roughly equivalent to a hypothetical mktap plugin: % mktap -a mymantissa.tap mantissa % twistd -nf mymantissa.tap The difference is that Axiom is a real object database, based around SQLite, with support for managed transactions, schema evolution, persistent timed events, and more. You can therefore put a reasonable amount of real data into it in addition to your configuration. >I can certainly write my own persistence code, but I'm a little worried that >more of the "best practices" presented by the docs are wrong. Should I still >use twistd? Should I use the application object? Etc. Yes, use twistd. Yes, use the application object. You *can* even write mktap plugins; they will work as described by the documentation. They're not buggy. (Okay, they're not *very* buggy.) It's just that the advertised benefits are not really so great so most people don't use them any more. The ones that come included with Twisted are handy toys. A separate issue is that if you have large, complex configurations, storing them in pickle (or any automatic persistence mechanism) is dangerous. A chance upgrade of any Twisted module which your configuration happens to contain instances from, or the standard library, or any other library which Twisted depends on, or any of your code... well, basically if any Python code on your computer changes at all, you could end up with a configuration that won't load. This isn't really a Twisted-specific issue, but it is a lesson that many Twisted Matrix Labs developers have learned by way of .tap files. There is a well-written paper on the dangers of automatic persistence here: http://www.erights.org/data/serial/jhu-paper/upgrade.html If you're intrigued by Axiom after reading this, there's good news and bad news. The good news: Axiom can provide all the advantages that tap plugins were supposed to, along with a relatively scalable and powerful general persistence mechanism. The dangers of upgrading are far reduced from .tap files, because (A) axiom includes very good schema migration support and automatic upgrades between released versions, (B) all attributes are explicit, so there are no "hidden surprises" when you upgrade, and (C) only Item objects in the database need to be upgraded, not every random thing you invoke in your code. The bad news: Axiom is heavily in development, nowhere near as stable as Twisted. The 0.4 release is ancient, in terms of available features, and documentation is mainly being written for the main line of development. Unfortunately there aren't necessarily upgrades beween every random revision of SVN trunk. In practice there have been relatively few breakages. However, it sounds like your needs are relatively lightweight and that you could happily use the 0.4 release of Axiom to deploy a real application. When we do get around to producing a new release, you shouldn't have to worry too much about backwards compatibility.

2 2

[Twisted-Python] I saw something about someone writing a Reactor that used libevent ...
by jarrod roberson 18 Apr '06

18 Apr '06

but now I can't find the information about it anywhere, I am dreaming this? Twisted with libevent would rock, not that Twisted isn't awesome right now :-)

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 18 Apr '06

18 Apr '06

Bug summary ______________________ Summary for 2006-04-09 through 2006-04-16 Bugs opened: 18 Bugs closed: 9 Total open bugs: 589 (+9) |== Type Changes |== Priority Changes |== Component Changes |Defect: +5 |Highest: -1 |Conch: +1 |Enhancement: +5 |Normal: +10 |Core: +1 |Task: -1 |Low: +0 |Mail: +1 |Pydoctor: +6 |Release Management: +1 |Trial: +2 |Web2: -1 |Website: -1 |Words: -1 New / Reopened Bugs ______________________ ===== Highest ===== [#1628] TwistedEmacs should be in emacs/ in Twisted (opened by jml) (CLOSED, fixed) enhancement core http://twistedmatrix.com/trac/ticket/1628 [#1634] Add ralphm to the matrix (opened by ralphm) defect release management http://twistedmatrix.com/trac/ticket/1634 [#1637] [PATCH] twisted.mail examples use deprecated imports (opened by jerub) defect mail http://twistedmatrix.com/trac/ticket/1637 ===== Normal ===== [#1629] Unjellyable.unjellyFor does not return self (opened by ml.flex) defect core http://twistedmatrix.com/trac/ticket/1629 [#1630] trial test hangs if using the twisted enterprise connection pool (opened by harshaw) defect trial http://twistedmatrix.com/trac/ticket/1630 [#1631] Make tables showing listings of members of packages, classes, etc, sortable (opened by moof) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1631 [#1632] "browse Twisted conch tickets" link wrong (opened by slamb) (CLOSED, fixed) defect website http://twistedmatrix.com/trac/ticket/1632 [#1633] Undocumented dependency on "Crypto" (opened by slamb) defect conch http://twistedmatrix.com/trac/ticket/1633 [#1635] Completely replace t.p.c.Interface with z.i.Interface (opened by ralphm) (CLOSED, duplicate) enhancement core http://twistedmatrix.com/trac/ticket/1635 [#1636] Completely replace t.p.c.Interface with z.i.Interface (opened by ralphm) enhancement core http://twistedmatrix.com/trac/ticket/1636 [#1638] provide a way for test collector to find custom test suites (opened by acapnotic) enhancement trial http://twistedmatrix.com/trac/ticket/1638 [#1639] Full package and class hierarchy pages. (opened by jknight) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1639 [#1640] Child process environment cannot be fully overridden on Windows (opened by bwh) defect core http://twistedmatrix.com/trac/ticket/1640 [#1641] sort subclasses in pydoctor's class hierarchy pages (opened by mwh) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1641 [#1642] pydoctor's nevowhtml needs tests (opened by mwh) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1642 [#1643] package listings in pydoctor should be alpha-sorted (opened by mwh) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1643 [#1645] force the rest of win32-select tests to pass (opened by glyph) defect core http://twistedmatrix.com/trac/ticket/1645 ===== Low ===== [#1644] pydoctor's nevowhtml is probably easy to speed up (opened by mwh) enhancement pydoctor http://twistedmatrix.com/trac/ticket/1644 Closed Bugs ______________________ ===== Highest ===== [#1560] Death to util.wait() (opened by jml, closed by jml, fixed) task core http://twistedmatrix.com/trac/ticket/1560 [#1628] TwistedEmacs should be in emacs/ in Twisted (opened by jml, closed by jml, fixed) enhancement core http://twistedmatrix.com/trac/ticket/1628 [#1612] Words (jabber code) still uses deprecated components API (opened by itamarst, closed by ralphm, fixed) defect words http://twistedmatrix.com/trac/ticket/1612 [#1615] stdio tests should be run for all reactors (opened by exarkun, closed by exarkun, fixed) enhancement core http://twistedmatrix.com/trac/ticket/1615 ===== Normal ===== [#1632] "browse Twisted conch tickets" link wrong (opened by slamb, closed by exarkun, fixed) defect website http://twistedmatrix.com/trac/ticket/1632 [#1635] Completely replace t.p.c.Interface with z.i.Interface (opened by ralphm, closed by ralphm, duplicate) enhancement core http://twistedmatrix.com/trac/ticket/1635 [#1596] Leftover use of childNotFound() in t.w2.twcgi.CGIDirectory (opened by grimmtooth, closed by dreid, fixed) defect web2 http://twistedmatrix.com/trac/ticket/1596 [#1320] Show current version of projects in sidebar, projects page (opened by itamarst, closed by exarkun, wontfix) enhancement website http://twistedmatrix.com/trac/ticket/1320 ===== Low ===== [#1487] Make LoopingCall deal with deferred returning callables (opened by ralphm, closed by ralphm, fixed) enhancement core http://twistedmatrix.com/trac/ticket/1487

5 9

[Twisted-Python] .tap and twistd and persistence
by Matt Feifarek 18 Apr '06

18 Apr '06

I've been using twisted for a couple of years, and am re-writing my application yet-again to make it even more twisty, so I'm trying to use the persistence mechanisms that come from services and applications, and I have hit a snag. I've conformed my application to the details of the tutorial in the docs (and I see the demonstrated method of making a 'plugin' is deprecated, and I get that message after registering my app and running mktap). First question: The deprecation warning tells me to look in twisted.plugin... but I don't see much in the way of docs in there. Where can I learn what I should do? Second question: Even though I'm using a fresh Twisted 2.2, installed from the tarball, I don't have any manpages... not for mktap or twistd, so the docs tell me to look in the manpage, but I can't (I'm on Ubuntu Breezy, FWIW). I see the man files themselves in my twisted source distribution, but they don't appear to be installed when I install twisted. Third question: I'm able to make a .tap file, run it, and send it ^C to shut it down. I get the [app]-shutdown.tap file, but when I try to run that file with 'twistd -f wrangler-shutdown.tap' I get a problem with the re-building of my objects. There's an attribute error, from one of the classes that's a part of my application. Do I have to write my application in some way to get it to persist and unpersist properly? There's nothing in my app that isn't pickle-able... it's just instances and dictionaries, nothing like open files or database connections. Thanks!

6 12