[Twisted-Python] supporting start/stop/restart behavior
First, before I get started, I'd like to say that I'm a new fan of twistd, it is quite nice tool, and I think everyone should use it. However, that being said, what it misses, IMHO, is a simple interface to stop and restart. Also, it lacks a nice way to be embedded within a python script so that the __main__ == __name__ hack runs the file properly. Anyway, what I'd like... is to take my arbitrary "test.py" file, let's say an application as simple as... from twisted.internet import app from twisted.web.server import Site from twisted.web.static import File application = app.Application('test') application.listenTCP(8080, Site(File('.'))) And add these three lines of code to make it runnable... if '__main__' == __name__: from twisted.scripts.twistd import runDirect runDirect(save=1) In this way, the following commands 'just work': python test.py # runs the app directly (no daemon) python test.py start # runs the app as a deamon python test.py stop # stops the app as a deamon python test.py restart # restarts the app as a deamon And, even twistd options can be included... python test.py --quiet start python test.py --quiet What this involves is two things: 1. Refactoring runApp to remove a hunk of code which runs os.kill on a pid found in the given pidfile. This is moved to another function, killApp, which takes two arguments, config and signal. From the runApp the code then calls killApp(config, signal=0) 2. Adding a new function, runDirect which does several things: a. takes 'start', 'stop', and 'restart' as the last argument on the command line, if none of these are found, then the application is assumed to be run interactively (nodaemon); and b. it adds --python <filename> where filename is the name of the current python file (argv[0]), further, it gives a direct option in the function arguments to not save the tap file (saving a tap file is not always useful, and as an argument it is hard to miss); and c. if stop or restart are chosen, then this kills the current process using the given pidfile (which defaults to twistd.pid in the current directory) through killApp(signal=SIGTERM) d. unless stop is chosen, the app is then started using the configuration options as parsed (and modified via the --python option) This function could be broken into the start/stop/restart behavior from the --python option; but I don't have this requirement and people in the IRC list didn't seem to think that either of these two functions are useful. Anyway, I'm posting to this list beacuse I think the way in which twistd is used isn't obvious... i.e. the primary way you'd use it with a python source file is an option buried among many others. I raised this change on the IRC list, and the primary argument against the above was that it is out-of-scope; in other words, one could write a shell script to do the same. I think this is a bad argument beacuse _all_ of twistd could be done in a shell script. For example, if you ask D. J. Bernstein, he'd say that none of this deamon / logging behavior needs to be in an application, and this is why he has his 'daemontools'. So, the argument shouldn't be _can_ this be done externally, of course it can. The question is does it make sence to standardize on particular ways of using twisted so that everyone doesn't have their own approach, to logging, etc. I feel that this comes all the way down to Twisted having a simple way to start/stop/restart a server without having to rely upon external, non-Twisted scripts. At the very least, I'd like the killApp refactor to be accepted, as I'd rather not have to duplicate killApp code within a private/sandbox copy of runDirect. Best, Clark
--- twistd.py.orig Wed Mar 19 16:21:46 2003 +++ twistd.py Wed Mar 19 20:19:15 2003 @@ -211,6 +211,31 @@ import pdb pdb.set_trace() +def killApp(config, signal = 0): + if os.path.exists(config['pidfile']): + try: + pid = int(open(config['pidfile']).read()) + except ValueError: + sys.exit('Pidfile %s contains non numeric value' % config['pidfile']) + + try: + os.kill(pid, signal) + except OSError, why: + if why[0] == errno.ESRCH: + # The pid doesnt exists. + if not config['quiet']: + print 'Removing stale pidfile %s' % config['pidfile'] + os.remove(config['pidfile']) + else: + sys.exit('Can\'t check status of PID %s from pidfile %s: %s' % (pid, config['pidfile'], why[1])) + else: + if not(signal): + sys.exit("""\ +Another twistd server is running, PID %s\n +This could either be a previously started instance of your application or a +different application entirely. To start a new one, either run it in some other +directory, or use my --pidfile and --logfile parameters to avoid clashes. +""" % pid) def runApp(config): global initRun @@ -248,29 +273,7 @@ # This will fix up accidental function definitions in evaluation spaces # and the like. initRun = 0 - if os.path.exists(config['pidfile']): - try: - pid = int(open(config['pidfile']).read()) - except ValueError: - sys.exit('Pidfile %s contains non numeric value' % config['pidfile']) - - try: - os.kill(pid, 0) - except OSError, why: - if why[0] == errno.ESRCH: - # The pid doesnt exists. - if not config['quiet']: - print 'Removing stale pidfile %s' % config['pidfile'] - os.remove(config['pidfile']) - else: - sys.exit('Can\'t check status of PID %s from pidfile %s: %s' % (pid, config['pidfile'], why[1])) - else: - sys.exit("""\ -Another twistd server is running, PID %s\n -This could either be a previously started instance of your application or a -different application entirely. To start a new one, either run it in some other -directory, or use my --pidfile and --logfile parameters to avoid clashes. -""" % pid) + killApp(config) if config['logfile'] == '-': if not config['nodaemon']: @@ -472,3 +475,66 @@ os._exit(1) runApp(config) + +def runDirect(save=1): + """ run directly from python file + + You can use this function to include start/stop/restart + functionality directly from your twisted application. + Following is example code, let us call it test.py, + + if '__main__' == __name__: + # run this before twisted.internet.reactor is imported + from twisted.scripts.twistd import runDirect + runDirect(save=1) + + from twisted.internet import app + from twisted.web.server import Site + from twisted.web.static import File + + application = app.Application('test') + application.listenTCP(8080, Site(File('.'))) + + + Given this code, the following will now work, + python test.py # runs the app directly (no daemon) + python test.py start # runs the app as a deamon + python test.py stop # stops the app as a deamon + python test.py restart # restarts the app as a deamon + + Options can be included as well, for example, + python test.py --quiet start # deamon + python test.py --quiet # no deamon + """ + from sys import argv, exit + config = ServerOptions() + config.synopsis = "Usage: %s [options] start|stop|restart" % argv[0] + bStop = 0; bStart = 1; bDaemon = 0 + cmd = argv[-1] + if cmd in ('start', 'stop', 'restart'): + argv.pop() + bDaemon = 1 + if 'restart' == cmd: + bStop = 1 + if 'stop' == cmd: + bStart = 0 + bStop = 1 + try: + config.parseOptions() + except usage.error, ue: + print config.opt_help() + exit(1) + if bStop: + from signal import SIGTERM + from os.path import exists + from time import sleep + killApp(config, SIGTERM) + nWait = 0 # processes do not die instantly + while exists(config['pidfile']) and nWait < 20: + sleep(.1) + nWait += 1 + if bStart: + if not save: config["no_save"] = 1 + if not bDaemon: config['nodaemon'] = 1 + config["python"] = argv[0] + runApp(config)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, March 19, 2003, at 07:58 PM, Clark C. Evans wrote:
... what [Twisted] misses, IMHO, is a simple interface to stop and restart.
You're right, but this is the wrong way to go about fixing it.
Also, it lacks a nice way to be embedded within a python script so that the __main__ == __name__ hack runs the file properly.
I disagree. Despite the fact that I occasionally write quick scripts that work this way, it's for quick hacks only, and providing better support for it is a misleading band-aid on the deficiencies of multi-platform twistd functionality.
And add these three lines of code to make it runnable...
if '__main__' == __name__: from twisted.scripts.twistd import runDirect runDirect(save=1)
In this way, the following commands 'just work':
python test.py # runs the app directly (no daemon) python test.py start # runs the app as a deamon python test.py stop # stops the app as a deamon python test.py restart # restarts the app as a deamon
... on UNIX. But, what does "restart" mean if you're running on Jython? On win32? What interaction does this imply with native OS services?
And, even twistd options can be included...
python test.py --quiet start python test.py --quiet
In that case, why not 'twistd -y test.py --quiet'?
1. Refactoring runApp to remove a hunk of code which runs os.kill on a pid found in the given pidfile. This is moved to another function, killApp, which takes two arguments, config and signal. From the runApp the code then calls killApp(config, signal=0)
Currently, that code is there _only_ to determine, in a rather platform-specific way, whether the server should attempt to keep starting or not. It's not a generalized stop-the-application.
2. Adding a new function, runDirect which does several things:
b. it adds --python <filename> where filename is the name of the current python file (argv[0]), further, it gives a direct option in the function arguments to not save the tap file (saving a tap file is not always useful, and as an argument it is hard to miss); and
Saving a tap file is pretty much always useful. It provides a common ground for introspection tools to look at what a server is doing. I have gone over this in previous mails.
c. if stop or restart are chosen, then this kills the current process using the given pidfile (which defaults to twistd.pid in the current directory) through killApp(signal=SIGTERM)
Why put this into Twisted where we have to swaddle it in a layer of cross-platform compatibility rather than just letting the user use whatever tools are available on their system for interacting with Twisted?
This function could be broken into the start/stop/restart behavior from the --python option; but I don't have this requirement and people in the IRC list didn't seem to think that either of these two functions are useful.
Anyway, I'm posting to this list beacuse I think the way in which twistd is used isn't obvious... i.e. the primary way you'd use it with a python source file is an option buried among many others.
So you are trying to solve a documentation problem by adding functionality to a different place, necessitating more documentation? :-)
I raised this change on the IRC list, and the primary argument against the above was that it is out-of-scope; in other words, one could write a shell script to do the same. I think this is a bad argument beacuse _all_ of twistd could be done in a shell script. For example, if you ask D. J. Bernstein, he'd say that none of this deamon / logging behavior needs to be in an application, and this is why he has his 'daemontools'.
D.J. Bernstein doesn't give a rip about portability outside of UNIX, so he makes the decision to write his framework for logging and daemonization in sh. We write these frameworks in Python, but I agree with this point: none of this daemon/logging/startup/shutdown behavior needs to be in an application. My view on functionality like this is that it is a platform-specific thing. Even assuming that signals worked like you expect them to across all platforms that we are trying to support, there are other issues. What about the cultural expectation of command lines? On win32, the way Python programs are run isn't very natural. On the command line, the location of the .exe is significant, there's no such thing as a shebang line, and users aren't accustomed to running interpreters on scripts. They expect to have an icon to click on, usually a Service. There are also differing cultural expectations of how you run stuff like this on MacOS. Does the start/stop thing have a GUI? A monitoring icon in the dock? Now, we don't *currently* have functionality like that, but we hope to one day. I am beginning to despair of ever actually achieving that goal, though, since nobody who understands it has time, and so many other programmers have habits that are opposed to this approach... Supporting mechanisms in the framework that make configuration and introspection impossible without editing the start-up code for your particular application would make this eventual goal even more difficult than it already is. Components written for twisted should be written in such a way that they can be loaded into existing applications without running a script. The --python option to twistd is pushing about as far as we want to go in the direction of hard-coding custom functionality for a particular server process.
So, the argument shouldn't be _can_ this be done externally, of course it can. The question is does it make sence to standardize on particular ways of using twisted so that everyone doesn't have their own approach, to logging, etc. I feel that this comes all the way down to Twisted having a simple way to start/stop/restart a server without having to rely upon external, non-Twisted scripts.
If you want to improve twistd, please feel free to suggest ways that stopping/restarting could be made easier or even more consistent across platforms. For example, "killApp" might make sense if, on signals-challenged platforms, twistd automatically opened a localhost-only socket which allowed communication with the server.
At the very least, I'd like the killApp refactor to be accepted, as I'd rather not have to duplicate killApp code within a private/sandbox copy of runDirect.
What functionality, exactly, is your platform's kill(1) missing that makes you want a Python version of it? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (Darwin) iD8DBQE+eVMwvVGR4uSOE2wRAopTAJ4+JQiM0WrJYIAH2qPMCbil4bE1+gCeMRaF X5qxxeDWi10E8IeWcuBkQE4= =L2sp -----END PGP SIGNATURE-----
Glyph, Thanks for responding, I know you're quite busy before PyCon. On Wed, Mar 19, 2003 at 11:35:39PM -0600, Glyph Lefkowitz wrote: | >... what [Twisted] misses, IMHO, is a simple interface to stop and | >restart. | | You're right, but this is the wrong way to go about fixing it. Ok. | > 1. Refactoring runApp to remove a hunk of code which | > runs os.kill on a pid found in the given pidfile. | > This is moved to another function, killApp, which | > takes two arguments, config and signal. From the | > runApp the code then calls killApp(config, signal=0) | | Currently, that code is there _only_ to determine, in a rather | platform-specific way, whether the server should attempt to keep | starting or not. It's not a generalized stop-the-application. If twistd adds (in a "rather platform-specific" way) support for stop and restart, then I'm sure platform-specific linkages can be added later on; for example doing NT based service API calls instead of fork / pidfiles / signals. | > c. if stop or restart are chosen, then this kills the | > current process using the given pidfile (which defaults | > to twistd.pid in the current directory) through | > killApp(signal=SIGTERM) | | Why put this into Twisted where we have to swaddle it in a layer of | cross-platform compatibility rather than just letting the user use | whatever tools are available on their system for interacting with | Twisted? Why not? With twistd, Twisted is taking on responsibility for 'starting' an application, shouldn't it finish the job with stop and restart? | So you are trying to solve a documentation problem by adding | functionality to a different place, necessitating more documentation? Got me. ;) | We write these frameworks in Python, but I agree with this point: | none of this daemon/logging/startup/shutdown behavior needs to | be in an application. Cool. Does it belong in a framework? If not, where? | My view on functionality like this is that it is a platform-specific | thing. Even assuming that signals worked like you expect them to | across all platforms that we are trying to support, there are other | issues. What about the cultural expectation of command lines? On | win32, the way Python programs are run isn't very natural. On the | command line, the location of the .exe is significant, there's no such | thing as a shebang line, and users aren't accustomed to running | interpreters on scripts. They expect to have an icon to click on, | usually a Service. There are also differing cultural expectations of | how you run stuff like this on MacOS. Does the start/stop thing have a | GUI? A monitoring icon in the dock? | | Now, we don't *currently* have functionality like that, but we hope to | one day. I am beginning to despair of ever actually achieving that | goal, though, since nobody who understands it has time, and so many | other programmers have habits that are opposed to this approach... All of this is great. But I've got many applications that I am writing, and would like to write; and I'd like a nice way to start, stop, and restart my twisted daemons. Currently, "twistd" seems to be the preferred command line approach. | Supporting mechanisms in the framework that make configuration and | introspection impossible without editing the start-up code for your | particular application would make this eventual goal even more | difficult than it already is. | | Components written for twisted should be written in such a way that | they can be loaded into existing applications without running a script. Uncle! Uncle! Ok. Ok. '__main__' == __name__ is bad. | If you want to improve twistd, please feel free to suggest ways that | stopping/restarting could be made easier or even more consistent across | platforms. For example, "killApp" might make sense if, on | signals-challenged platforms, twistd automatically opened a | localhost-only socket which allowed communication with the server. Ok. Could we add to twistd 'start', 'stop' and 'restart' sub-commands, with 'start' being the default. For now, we implement stop with kill signals. When we get time we refactor the whole start/stop/restart code so that it uses a 'Process' object that is platform specific. The process object has 'start', 'stop', and 'restart' methods, and it has two or more implementations, including, but not limited to a UnixProcess, which uses kill/signals/pidfile, and a NT Process, which uses NT services, etc. As for GUIs. What would be useful is one more sub-command, 'query'. Which returns a status of the given process, 'starting', 'stopping', 'running', 'restarting', 'stopped'. With those in place, I'm sure someone could write a nice cross-platform wxWindows viewer which calls twistd to check the status of various processes, etc. We'd probably at that point also have to start thinking about registry entries, i.e. configuration settings (Coca?). Best, Clark
This patch adds 'stop', 'start', and 'restart' commands to twistd, so that a simple program, say, from twisted.internet import app from twisted.web.server import Site from twisted.web.static import File application = app.Application('test') application.listenTCP(8080, Site(File('.'))) Can be 'started' using: twistd -y test.py and 'stopped' using: twistd -y test.py stop and 'restarted' using: twistd -y test.py restart This is a good way to start thinking cross-platform, as twistd could be refactored later to, say on Win NT, use NT services for the start/stop/restart behavior. For now the current patch just uses the current "PID" file approach. Besides providing a cross-platform iterface, it doesn't require that users new to unix learn "kill" or have to get involved with finding the right pid file. Lastly, it provides the necessary delay so that a restart can happen using a single command. Best, Clark --- twistd.py.orig Wed Mar 19 16:21:46 2003 +++ twistd.py Thu Mar 20 12:37:56 2003 @@ -43,7 +43,7 @@ class ServerOptions(usage.Options): - synopsis = "Usage: twistd [options]" + synopsis = "Usage: twistd [options] [start|stop|restart]" optFlags = [['nodaemon','n', "don't daemonize"], ['savestats', None, "save the Stats object rather than the text output of the profiler."], @@ -87,6 +87,13 @@ 'This will only take effect if the application to be run has an application ' 'name.']] + subCommands = [['start', None, usage.Options, + 'starts the application requested (default)'], + ['stop', None, usage.Options, + 'shuts the given application down if it is running'], + ['restart',None, usage.Options, + 'restarts the application']] + def opt_plugin(self, pkgname): """read config.tac from a plugin package, as with -y """ @@ -211,6 +218,31 @@ import pdb pdb.set_trace() +def signalApp(config, signal = 0): + if os.path.exists(config['pidfile']): + try: + pid = int(open(config['pidfile']).read()) + except ValueError: + sys.exit('Pidfile %s contains non numeric value' % config['pidfile']) + + try: + os.kill(pid, signal) + except OSError, why: + if why[0] == errno.ESRCH: + # The pid doesnt exists. + if not config['quiet']: + print 'Removing stale pidfile %s' % config['pidfile'] + os.remove(config['pidfile']) + else: + sys.exit('Can\'t check status of PID %s from pidfile %s: %s' % (pid, config['pidfile'], why[1])) + else: + if not(signal): + sys.exit("""\ +Another twistd server is running, PID %s\n +This could either be a previously started instance of your application or a +different application entirely. To start a new one, either run it in some other +directory, or use my --pidfile and --logfile parameters to avoid clashes. +""" % pid) def runApp(config): global initRun @@ -248,29 +280,7 @@ # This will fix up accidental function definitions in evaluation spaces # and the like. initRun = 0 - if os.path.exists(config['pidfile']): - try: - pid = int(open(config['pidfile']).read()) - except ValueError: - sys.exit('Pidfile %s contains non numeric value' % config['pidfile']) - - try: - os.kill(pid, 0) - except OSError, why: - if why[0] == errno.ESRCH: - # The pid doesnt exists. - if not config['quiet']: - print 'Removing stale pidfile %s' % config['pidfile'] - os.remove(config['pidfile']) - else: - sys.exit('Can\'t check status of PID %s from pidfile %s: %s' % (pid, config['pidfile'], why[1])) - else: - sys.exit("""\ -Another twistd server is running, PID %s\n -This could either be a previously started instance of your application or a -different application entirely. To start a new one, either run it in some other -directory, or use my --pidfile and --logfile parameters to avoid clashes. -""" % pid) + signalApp(config) if config['logfile'] == '-': if not config['nodaemon']: @@ -457,6 +467,19 @@ log.err("--report-profile specified but application has no name (--appname unspecified)") log.msg("Server Shut Down.") +def stopApp(config): + from signal import SIGTERM + from os.path import exists + from time import sleep + signalApp(config, SIGTERM) + nWait = 0 # processes do not die instantly + while exists(config['pidfile']) and nWait < 20: + sleep(.1) + nWait += 1 + +def restartApp(config): + stopApp(config) + runApp(config) def run(): # make default be "--help" @@ -471,4 +494,7 @@ print "%s: %s" % (sys.argv[0], ue) os._exit(1) + cmd = getattr(config,'subCommand','start') + if 'stop' == cmd: return stopApp(config) + if 'restart' == cmd: return restartApp(config) runApp(config)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, March 20, 2003, at 12:08 PM, Clark C. Evans wrote:
Can be 'started' using: twistd -y test.py and 'stopped' using: twistd -y test.py stop and 'restarted' using: twistd -y test.py restart
I think I like this patch, because if I understand correctly, -y isn't a requirement :-). This should work fine with 'twistd -f test.tap restart', correct? There should probably be some more defined interactions with persistence. But I like the direction. Can some other twisted.internet wizard evaluate as well? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (Darwin) iD8DBQE+ejvjvVGR4uSOE2wRAtDeAKCUPwrCFcTka5IrjmB2VrNY5jXnCgCfSUuE X8a9qoPkCOk7kZpb9+a/GNs= =dMH0 -----END PGP SIGNATURE-----
On Thu, Mar 20, 2003 at 04:08:32PM -0600, Glyph Lefkowitz wrote: | On Thursday, March 20, 2003, at 12:08 PM, Clark C. Evans wrote: | >Can be 'started' using: twistd -y test.py | > and 'stopped' using: twistd -y test.py stop | > and 'restarted' using: twistd -y test.py restart | | I think I like this patch, because if I understand correctly, -y isn't | a requirement :-). This should work fine with 'twistd -f test.tap | restart', correct? Correct. It is 'othogonal' to the other functions of twistd. | There should probably be some more defined interactions with | persistence. But I like the direction. Can some other | twisted.internet wizard evaluate as well? Given that it sends the appropriate signal (shutdown), if taps are used it should save/restore appropriately. Clark
On Friday, March 21, 2003, at 01:01 PM, Clark C. Evans wrote:
Given that it sends the appropriate signal (shutdown), if taps are used it should save/restore appropriately.
One big question is what happens when you do a restart; does it stop the server and then start it with the original tap, or with the -shutdown.tap? I vote for it restarting the -shutdown.tap, but there are plenty of reasons not to do this, too. Donovan
On Fri, Mar 21, 2003 at 12:58:51PM -0800, Donovan Preston wrote: | One big question is what happens when you do a restart; does it stop | the server and then start it with the original tap, or with the | -shutdown.tap? | | I vote for it restarting the -shutdown.tap, but there are plenty of | reasons not to do this, too. I don't have an opinion, other than pointing out that if --no_save is used, then there won't be a -shutdown.tap and therefore, this sort of feature would be disabled. Proposed implementation: Suppose that xxx.tap was the initial application file and that it was started with twistd -f xxx.tap; on shutdown this would create a xxx-shutdown.tap Then, the 'restart' command would do the following: 1) if a xxx-initial.tap file isn't there, it should copy xxx.tap to xxx-initial.tap 2) if a xxx-shutdown.tap file exists, it should copy xxx-shutdown.tap to xxx.tap 3) it should then follow the same process as 'start' On a related topic, I would very much like --python to imply --no_save ; if someone wants to use taps then they should write their app.py file to use app.save functionality to make a .tap file. It is those people who arn't using '.tap' saved state files that, in most cases, will be using the --python option. Best, Clark
participants (3)
-
Clark C. Evans -
Donovan Preston -
Glyph Lefkowitz