hi there, folks:
I'd really like to release 0.7.0 but I would like it to be at least a
little bit tested before I do so. Could those of you with CVS trees check
everything out and see if it performs as advertised? Deeper bugs than
that will have to wait for the next release, but I'd at least like to know
if it works for someone other than me.
______ __ __ _____ _ _
| ____ | \_/ |_____] |_____|
|_____| |_____ | | | |
@ t w i s t e d m a t r i x . c o m
Here's some ideas I've been thinking about with aid of my housemate (who is
A feature of some servers is the ability to run chroot'd, so say you want to
be a WebDAV server for a site, when a user logs in, it forks, chroots to the
users home directory, and sets its uid/gid to that of the user, thereby
restricting the potential for security holes to damage the system. (The
chroot is an optional paranoid extra if you are dropping uid/gid)
Consider twisted.web.distrib (I might have bits of this slightly wrong;
please correct me if so). It currently does something like this, although
manually: each user has to start their own twistd, which only runs
twisted.web, and talks to the superserver with PB via a pipe. This is good,
but not as general as it could be. It's limited to twisted.web, and users
have to manually ensure that twistd is running.
I think it'd be nice to generalise this to *any* service that can run within
the context of different users, e.g. for FTP, WebDAV, whatever. So there'd
be some automated way to tell a Twisted application "Ok, now serve this as a
particular user by passing it to a user-owned twistd".
What I'm envisaging is something like this:
* The superserver receives a WidgetService request from user Bob
* The superserver connects to the user's WidgetService twistd, starting it
if necessary (more on this in a moment)
* The user's WidgetService then does all the hard work :)
If the user doesn't have a currently running twistd, then you could do, say:
* The superserver creates a pipe
* The superserver (which is running as root (or something effectively
root)) forks a child
* The child does a reactor.crash() to stop it interfering with the
parent's normal handling of events
* The child closes all file descriptors in the reactor, except for the
pipe. It should probably start logging to a different log file as well.
Also, things like DB-backed twisted.cred authenticators probably should
be shutdown... hmm, this part is messy :(
* The child then starts a fresh Application, which runs the user's
WidgetService and starts processing for the superserver
Now that I think about it, it is probably cleaner to simply spawn a fresh
process rather than futzing around with fork, so just:
* The superserver spawns a user twistd as a particular uid/gid, and talks
to it over a PB pipe.
The nice thing about this model is that there is only 1 twistd running per
user (and you could get the user twistds to automatically shutdown if
inactive for, say, 10 minutes to free resources), which means your process
load is still mostly independent of the number of connections. It'd be
*really* cool if a superserver listening on multiple protocols, e.g.
"WidgetService" and HTTP, could have only a *single* twistd per user,
regardless of number of protocols the superserver is dealing with.
It'd be really nice to be able to chroot as well, but I suspect python
doesn't work cleanly with chroot, due to file descriptors being left open to
So yeah, this is a long-winded way of saying "twisted.web.distrib is cool,
but should be general and more automatic", because I think this would
probably be useful.
In vaguely related news, I've nearly got a simple Twisted inetd replacement
written... expect a checkin sometime this weekend.
Another release candidate out; this one will probably turn into 0.99.1
this weekend. Grab it at http://twistedmatrix.com/products/download
Moshez finally got me around to doing signed md5sums:
-----BEGIN PGP SIGNED MESSAGE-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
-----END PGP SIGNATURE-----
I've also updated the downloads page to have a link to these signed md5sums.
<< radix(a)twistedmatrix.com >>
Hey all, I made 99.1rc3 available on the website (thanks to moshez and
exarkun for doing deb/win32 packages). Please download and test. As always,
this will become 0.99.1 once it's deemed to be Bug Free[TM].
<< radix(a)twistedmatrix.com >>
My approach to doing pb filetransfers is attached,
this code should illustrate the basic idea:
# A Session is a chain of Jobs that are carried out in order,
# one after another.
# The server builds a custom Session from Mixins, to provide
# only the jobs he wants to accept. ServerSession is the base
# class (pb.Viewable).
# FileSSM is the ServerSessionMixin for Filetransfers.
class FileSession(session.ServerSession, session.FileSSM):
# Then that class is offered to clients via callRemote.
# That's all, the server now provides file transfer functionality
# to clients.
# Client builds a Custom Session too.
# FileCSM is the ClientSessionMixin for Filetransfers,
# CallableCSM provides session.call(func, args)
# which is used to call a function after some other
# job has completed (e.g. a file download)
class MySession(session.ClientSession, session.FileCSM, session.CallableCSM):
# Then a session is requested from the server, and the received
# deferred is wrapped by MySession. After that we can queue jobs
# on the session.
session = MySession(perspective.callRemote("getsession"))
session.call(woot, "Done with Session.")
Have you thought of getting the callLater() method to return an
object with a cancel() method?
from twisted.internet import reactor
print "I'll never run."
call = reactor.callLater(5, f)
Other methods could be added and both would reduce
Modification of the arguments or re-scheduling the
time could be such methods to add.
Because I use rebuild a _lot_ in one of my applications, I finally broke
down and write this addition to scripts/twistd.py. I'm not sure if its
appropriate for inclusion in the repository (and if it is, it certainly
needs some cleanup first).
diff -a -c -t -r1.1 twistd.py
*** twisted/scripts/twistd.py 20 Aug 2002 02:31:59 -0000 1.1
--- twisted/scripts/twistd.py 28 Aug 2002 23:32:23 -0000
*** 309,311 ****
--- 309,342 ----
log.msg("Server Shut Down.")
+ def daemonize(logf = 'twistd.log'):
+ from twisted.python import log
+ if logf == '-':
+ print 'daemons cannot log to stdout'
+ logPath = os.path.abspath(logf or 'twistd.log')
+ logFile = logfile.LogFile(os.path.basename(logPath), os.path.dirname(logPath))
+ # rotate logs on SIGUSR1
+ if os.name == "posix":
+ import signal
+ def rotateLog(signal, frame, logFile=logFile):
+ signal.signal(signal.SIGUSR1, rotateLog)
+ oldstdin = sys.stdin
+ oldstdout = sys.stdout
+ oldstderr = sys.stderr
+ # Turn into a daemon.
+ if os.fork(): # launch child and...
+ os._exit(0) # kill off parent
An undaemonize() would be _really_ handy, too... Hmmm :)
There are 10 kinds of people: those who understand binary and those who do
7:00pm up 99 days, 19:51, 5 users, load average: 0.03, 0.07, 0.02
Silly point that only a bunch of emacs-thumping Python wonks would
make. Should/may I put a bit in the Whitespace section of the
CodingStandard about emacs' whitespace-cleanup mode? Point being that
it doesn't let you leave trailing whitespace on the end of (possibly
empty) lines or trailing lines at the end of file. Being consistent
about this would cut down on spurious file-modified flags from CVS.
See C-h f whitespace-describe if unfamiliar.
The only evil thing about it is that if you don't disable
whitespace-check-indent-whitespace, it'll, um, use tabs.