Mailman 3 August 2002 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Twisted superservers and setuid/setgid subservers
by Andrew Bennetts 31 Aug '02

31 Aug '02

Here's some ideas I've been thinking about with aid of my housemate (who is CC'd). A feature of some servers is the ability to run chroot'd, so say you want to be a WebDAV server for a site, when a user logs in, it forks, chroots to the users home directory, and sets its uid/gid to that of the user, thereby restricting the potential for security holes to damage the system. (The chroot is an optional paranoid extra if you are dropping uid/gid) Consider twisted.web.distrib (I might have bits of this slightly wrong; please correct me if so). It currently does something like this, although manually: each user has to start their own twistd, which only runs twisted.web, and talks to the superserver with PB via a pipe. This is good, but not as general as it could be. It's limited to twisted.web, and users have to manually ensure that twistd is running. I think it'd be nice to generalise this to *any* service that can run within the context of different users, e.g. for FTP, WebDAV, whatever. So there'd be some automated way to tell a Twisted application "Ok, now serve this as a particular user by passing it to a user-owned twistd". What I'm envisaging is something like this: * The superserver receives a WidgetService request from user Bob * The superserver connects to the user's WidgetService twistd, starting it if necessary (more on this in a moment) * The user's WidgetService then does all the hard work :) If the user doesn't have a currently running twistd, then you could do, say: * The superserver creates a pipe * The superserver (which is running as root (or something effectively root)) forks a child * The child does a reactor.crash() to stop it interfering with the parent's normal handling of events * The child closes all file descriptors in the reactor, except for the pipe. It should probably start logging to a different log file as well. Also, things like DB-backed twisted.cred authenticators probably should be shutdown... hmm, this part is messy :( * The child then starts a fresh Application, which runs the user's WidgetService and starts processing for the superserver Now that I think about it, it is probably cleaner to simply spawn a fresh process rather than futzing around with fork, so just: * The superserver spawns a user twistd as a particular uid/gid, and talks to it over a PB pipe. Anyway... The nice thing about this model is that there is only 1 twistd running per user (and you could get the user twistds to automatically shutdown if inactive for, say, 10 minutes to free resources), which means your process load is still mostly independent of the number of connections. It'd be *really* cool if a superserver listening on multiple protocols, e.g. "WidgetService" and HTTP, could have only a *single* twistd per user, regardless of number of protocols the superserver is dealing with. It'd be really nice to be able to chroot as well, but I suspect python doesn't work cleanly with chroot, due to file descriptors being left open to modules... So yeah, this is a long-winded way of saying "twisted.web.distrib is cool, but should be general and more automatic", because I think this would probably be useful. In vaguely related news, I've nearly got a simple Twisted inetd replacement written... expect a checkin sometime this weekend. -Andrew.

1 0

[Twisted-Python] Twisted 0.99.1rc4
by Christopher Armstrong 31 Aug '02

31 Aug '02

Another release candidate out; this one will probably turn into 0.99.1 this weekend. Grab it at http://twistedmatrix.com/products/download Moshez finally got me around to doing signed md5sums: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 6c0934460fe24ffa75247cb6a324f2ee Twisted-0.99.1rc4.tar.bz2 e063effe8789bc3d5741c75a229d7a4a Twisted-0.99.1rc4.tar.gz 5d9b44a2509b0eee3e9299527f4995f6 Twisted-0.99.1rc4.zip 0f6f036318844098e5bff153094737e8 TwistedDocs-0.99.1rc4.tar.bz2 3107769bbac3de34ad66ad611738e902 TwistedDocs-0.99.1rc4.tar.gz 2e66ad4f3dbbc8b19316c5187bba028c TwistedDocs-0.99.1rc4.zip f0a4fe811c738b51f6ec1fff0408f0a7 Twisted_NoDocs-0.99.1rc4.tar.bz2 41d84e73fc1890eb010d087bd12e0764 Twisted_NoDocs-0.99.1rc4.tar.gz a90b68e1755b92089dce6941fb3facea Twisted_NoDocs-0.99.1rc4.zip -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE9cEmjNp279O1/OG4RAklwAJ4tXUR0a9N6BnMPz7gsSuMFldQmLwCghFpW +/BjfnMRsICK/Vq0MllWMJw= =qsAe -----END PGP SIGNATURE----- I've also updated the downloads page to have a link to these signed md5sums. -- Chris Armstrong << radix(a)twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/

2 1

[Twisted-Python] Dud bug tracker link on tm.com
by Andrew Bennetts 30 Aug '02

30 Aug '02

>From anywhere other than the front page (e.g. the uber-nifty success stories page, http://twistedmatrix.com/services/success), the "Bug Tracker" link points to: http://twistedmatrix.com/http://sourceforge.net/tracker/?func=browse&group_… -Andrew.

4 4

[Twisted-Python] Twisted 0.99.1rc3
by Chris Armstrong 29 Aug '02

29 Aug '02

Hey all, I made 99.1rc3 available on the website (thanks to moshez and exarkun for doing deb/win32 packages). Please download and test. As always, this will become 0.99.1 once it's deemed to be Bug Free[TM]. -- Chris Armstrong << radix(a)twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/

1 0

[Twisted-Python] rfc: pb filetransfers
by Paul Boehm 29 Aug '02

29 Aug '02

My approach to doing pb filetransfers is attached, Suggestions welcome! this code should illustrate the basic idea: # A Session is a chain of Jobs that are carried out in order, # one after another. # The server builds a custom Session from Mixins, to provide # only the jobs he wants to accept. ServerSession is the base # class (pb.Viewable). # FileSSM is the ServerSessionMixin for Filetransfers. class FileSession(session.ServerSession, session.FileSSM): pass # Then that class is offered to clients via callRemote. class FilePerspective(pb.Perspective): def perspective_getsession(self): return FileSession() # That's all, the server now provides file transfer functionality # to clients. ---- # Client builds a Custom Session too. # FileCSM is the ClientSessionMixin for Filetransfers, # CallableCSM provides session.call(func, args) # which is used to call a function after some other # job has completed (e.g. a file download) class MySession(session.ClientSession, session.FileCSM, session.CallableCSM): pass # Then a session is requested from the server, and the received # deferred is wrapped by MySession. After that we can queue jobs # on the session. session = MySession(perspective.callRemote("getsession")) session.sendfile("/etc/hosts", "remotecopy") session.getfile("/etc/passwd", "localcopy") session.getfile("/etc/hosts", "localcopy") session.call(woot, "Done with Session.") # Go! reactor.run()

1 0

[Twisted-Python] http://www.twistedmatrix.com/documents/howto/time
by Boyd Roberts 29 Aug '02

29 Aug '02

Have you thought of getting the callLater() method to return an object with a cancel() method? Eg: from twisted.internet import reactor def f(): print "I'll never run." call = reactor.callLater(5, f) call.cancel() Other methods could be added and both would reduce reactor's namespace. Modification of the arguments or re-scheduling the time could be such methods to add.

3 4

[Twisted-Python] Submitted for your approval
by Jp Calderone 28 Aug '02

28 Aug '02

Because I use rebuild a _lot_ in one of my applications, I finally broke down and write this addition to scripts/twistd.py. I'm not sure if its appropriate for inclusion in the repository (and if it is, it certainly needs some cleanup first). diff -a -c -t -r1.1 twistd.py *** twisted/scripts/twistd.py 20 Aug 2002 02:31:59 -0000 1.1 --- twisted/scripts/twistd.py 28 Aug 2002 23:32:23 -0000 *************** *** 309,311 **** --- 309,342 ---- if usepid: os.unlink(config.opts['pidfile']) log.msg("Server Shut Down.") + + + def daemonize(logf = 'twistd.log'): + from twisted.python import log + if logf == '-': + print 'daemons cannot log to stdout' + else: + logPath = os.path.abspath(logf or 'twistd.log') + logFile = logfile.LogFile(os.path.basename(logPath), os.path.dirname(logPath)) + + # rotate logs on SIGUSR1 + if os.name == "posix": + import signal + def rotateLog(signal, frame, logFile=logFile): + logFile.rotate() + signal.signal(signal.SIGUSR1, rotateLog) + + oldstdin = sys.stdin + oldstdout = sys.stdout + oldstderr = sys.stderr + log.startLogging(logFile) + sys.stdout.flush() + + # Turn into a daemon. + if os.fork(): # launch child and... + os._exit(0) # kill off parent + os.setsid() + os.umask(077) + oldstdin.close() + oldstdout.close() + oldstderr.close() An undaemonize() would be _really_ handy, too... Hmmm :) Jp -- There are 10 kinds of people: those who understand binary and those who do not. -- 7:00pm up 99 days, 19:51, 5 users, load average: 0.03, 0.07, 0.02

1 0

[Twisted-Python] Generalised data/metadata storage/retrieval
by Jp Calderone 28 Aug '02

28 Aug '02

Interface description attached. Written in response to glyph's comments that Twisted.news was waaay too NNTPy. Suggestions and other comments appreciated. -- Seduced, shaggy Samson snored. She scissored short. Sorely shorn, Soon shackled slave, Samson sighed, Silently scheming, Sightlessly seeking Some savage, spectacular suicide. -- Stanislaw Lem, "Cyberiad" -- 4:00pm up 99 days, 16:51, 5 users, load average: 0.11, 0.13, 0.09

1 0

[Twisted-Python] M-x customize-group whitespace
by Kevin Turner 28 Aug '02

28 Aug '02

Silly point that only a bunch of emacs-thumping Python wonks would make. Should/may I put a bit in the Whitespace section of the CodingStandard about emacs' whitespace-cleanup mode? Point being that it doesn't let you leave trailing whitespace on the end of (possibly empty) lines or trailing lines at the end of file. Being consistent about this would cut down on spurious file-modified flags from CVS. See C-h f whitespace-describe if unfamiliar. The only evil thing about it is that if you don't disable whitespace-check-indent-whitespace, it'll, um, use tabs. -Yours, - +Yours, + acap -

2 1