Draft PEP: Standard daemon process library
Howdy all, I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon. This message is a call for comments, support, and suggestions, prior to submitting this PEP to the editor. :PEP: XXX :Title: Standard daemon process library :Version: 0.1 :Last-Modified: 2009-01-26 08:44 :Author: Ben Finney <ben+python@benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 2009-01-26 :Python-Version: 3.1 :Post-History: ======== Abstract ======== Writing a program to become a well-behaved Unix daemon is somewhat complex and tricky to get right, yet the steps are largely similar for any daemon regardless of what else the program may need to do. This PEP introduces a module to the Python standard library that provides a simple interface to the task of becoming a daemon process. .. contents:: .. Table of Contents: Abstract Specification Example usage Interface ``Daemon`` objects ``DaemonError`` objects Motivation Rationale Correct daemon behaviour Reference Implementation References Copyright ============= Specification ============= Example usage ============= Simple example of usage:: import daemon from spam import do_main_program this_daemon = daemon.Daemon() this_daemon.start() do_main_program() More complex example usage:: import os import grp import signal import daemon from spam import ( initial_program_setup, do_main_program, program_cleanup, reload_program_config, ) initial_program_setup() important_file = open('spam.data', 'w') interesting_file = open('eggs.data', 'w') this_daemon = daemon.Daemon() this_daemon.files_preserve = [important_file, interesting_file] this_daemon.working_directory = '/var/lib/foo' this_daemon.umask = 0o002 mail_gid = grp.getgrnam('mail').gr_gid this_daemon.gid = mail_gid this_daemon.terminate_callback = program_cleanup this_daemon.reload_callback = reload_program_config this_daemon.reload_signals = [signal.SIGHUP, signal.SIGUSR1] this_daemon.start() do_main_program() Interface ========= A new module, `daemon`, is added to the standard library. The module defines a class, `Daemon`, used to represent the settings for a daemon process. An exception class, `DaemonError`, is defined for exceptions raised from the module. ``Daemon`` objects ================== A `Daemon` instance represents the behaviour settings for the process when it becomes a daemon. The behaviour is customised by setting attributes on the instance, before calling the `start` method. The following attributes are defined. `files_preserve` :Default: ``None`` List of files that should *not* be closed when starting the daemon. If ``None``, all open file descriptors will be closed. Elements of the list are file descriptors (as returned by a file object's `fileno()` method) or Python `file` objects. Each specifies a file that is not to be closed during daemon start. `chroot_directory` :Default: ``None`` Full path to a directory to set as the effective root directory of the process. If ``None``, specifies that the root directory is not to be changed. `working_directory` :Default: ``'/'`` Full path of the working directory to which the process should change on daemon start. Since a filesystem cannot be unmounted if a process has its current working directory on that filesystem, this should either be left at default or set to a directory that is a sensible “home directory” for the daemon while it is running. `lockfile_directory` :Default: ``'/var/run'`` Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped. `lockfile_name` :Default: ``None`` Base name of the lockfile for this daemon, without directory or extension. If ``None``, the name is derived from the process command line. `umask` :Default: ``0`` File access creation mask (“umask”) to set for the process on daemon start. Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects. `ignore_signals` :Default: ``[signal.SIGTTOU, signal.SIGTTIN, signal.SIGTSTP]`` List of signals that the process should ignore (by setting the signal action to ``signal.SIG_IGN``) on daemon start. `terminate_signals` :Default: ``[signal.SIGTERM]`` List of signals that the process should interpret as a request to terminate cleanly. `terminate_callback` :Default: ``None`` Callable to invoke when the process receives any of the `terminate_signals` signals, before then terminating the process. `reload_signals` :Default: ``[signal.SIGHUP]`` List of signals that the process should interpret as a request to reload runtime configuration. `reload_callback` :Default: ``None`` Callable to invoke when the process receives any of the `reload_signals` signals. `uid` :Default: ``None`` The user ID (“uid”) value to switch the process to on daemon start. `gid` :Default: ``None`` The group ID (“gid”) value to switch the process to on daemon start. `prevent_core` :Default: ``True`` If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`. `stdout` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stdout`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stdout` is not re-bound. `stderr` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stderr`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stderr` is not re-bound. The following methods are defined. `start()` :Return: ``None`` Start the daemon. This performs the following steps: * If the `chroot_directory` attribute is not ``None``: * Set the effective root directory of the process to that directory (via `os.chroot`). This allows running the daemon process inside a “chroot gaol” as a means of limiting the system's exposure to rogue behaviour by the process. * If the `lockfile_directory` attribute is not ``None``: * Look in that directory for a file named '`lockfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process. * Close all open file descriptors, excluding those listed in the `files_preserve` attribute. * Change current working directory to the path specified by the `working_directory` attribute. * Reset the file access creation mask to the value specified by the `umask` attribute. * Detach the current process into its own process group, and disassociate from any controlling terminal. This step is skipped if it is determined to be redundant: if the process was started by `init`, by `initd`, or by `inetd`. * Set signal handlers as specified by the `ignore_signals`, `terminate_signals`, `terminate_callback`, `reload_signals`, `reload_callback` attributes. * If the `prevent_core` attribute is true: * Set the resource limits for the process to prevent any core dump from the process. * Set the process uid and gid to the true uid and gid of the process, to relinquish any elevated privilege. * If the `lockfile_directory` attribute is not ``None``: * Create the lockfile for this daemon in that directory, by writing a text line containing the current process ID (“pid”) to a file named '`lockfile_name`.pid'. * If either of the attributes `uid` or `gid` are not ``None``: * Set the process uid and/or gid to the specified values. * If either of the attributes `stdout` or `stderr` are not ``None``: * Bind the names `sys.stdout` and/or `sys.stderr` to the corresponding objects. `reload()` :Return: ``None`` Reload the daemon configuration. The meaning of this is entirely defined by the customisation of this daemon: if the `reload_callback` attribute is not ``None``, call that object. The return value is discarded. `stop()` :Return: ``None`` Stop the daemon. This performs the following steps: * If the `terminate_callback` attribute is not ``None``: * Call that object. The return value is discarded. * If the `lockfile_directory` attribute is not ``None``: * Delete the lockfile for this daemon. * Raise a `SystemExit` exception. ``DaemonError`` objects ======================= The `DaemonError` class inherits from `Exception`. The module implementation will raise an instance of `DaemonError` when an error occurs in processing daemon behaviour. ========== Motivation ========== The majority of programs written to be Unix daemons either implement behaviour very similar to that in the `Specification`_, or are poorly-behaved daemons by the `Rationale`_. Since these steps should be much the same in most implementations but are very particular and easy to omit or implement incorrectly, they are a prime target for a standard well-tested implementation in the standard library. ========= Rationale ========= Correct daemon behaviour ======================== According to Stevens in [stevens]_ §2.6, a program should perform the following steps to become a Unix daemon process. * Close all open file descriptors. * Change current working directory. * Reset the file access creation mask. * Run in the background. * Disassociate from process group. * Ignore terminal I/O signals. * Disassociate from control terminal. * Don't reacquire a control terminal. * Correctly handle the following circumstances: * Started by System V `init` process. * Daemon termination by ``SIGTERM`` signal. * Children generate ``SIGCLD`` signal. The `daemon` package [daemon]_ lists (in its summary of features) behaviour that should be performed when turning a program into a well-behaved Unix daemon process. The following are appropriate for a daemon started once the program is already running: * Sets up the correct process context for a daemon. * Behaves sensibly when started by `initd(8)` or `inetd(8)`. * Revokes any suid or sgid privileges to reduce security risks in case daemon is incorrectly installed with special privileges. * Prevents the generation of core files to prevent leaking sensitive information from daemons run as root (optional). * Names the daemon by creating and locking a PID file to guarantee that only one daemon with the given name can execute at any given time (optional). * Sets the user and group under which to run the daemon (optional, root only). * Creates a chroot gaol (optional, root only). * Captures the daemon's stdout and stderr and directs them to syslog (optional). ======================== Reference Implementation ======================== The `python-daemon` package [python-daemon]_. As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP. ========== References ========== .. [stevens] `Unix Network Programming`, W. Richard Stevens, 1994 Prentice Hall. .. [daemon] The (non-Python) `daemon` package, `<http://www.libslack.org/daemon/>`_ by “raf” <raf@raf.org>. .. [python-daemon] The `python-daemon` package at the Python Package Index, `<http://pypi.python.org/pypi/python-daemon>`_. This is the successor to [bda.daemon]_. .. [bda.daemon] The `bda.daemon` package at the Python Package Index, `<http://pypi.python.org/pypi/bda.daemon>`_. This is an implementation of [cookbook-66012]_. .. [cookbook-66012] Many good ideas were contributed by the community to Python cookbook recipe 66012, “Fork a daemon process on Unix” `<http://code.activestate.com/recipes/66012/>`_. ========= Copyright ========= This work is hereby placed in the public domain. To the extent that placing a work in the public domain is not legally possible, the copyright holder hereby grants to all recipients of this work all rights and freedoms that would otherwise be restricted by copyright. -- \ “Pinky, are you pondering what I'm pondering?” “Well, I think | `\ so (hiccup), but Kevin Costner with an English accent?” —_Pinky | _o__) and The Brain_ | Ben Finney
Ben Finney wrote:
Howdy all,
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
[complete spec omitted]
======================== Reference Implementation ========================
The `python-daemon` package [python-daemon]_.
As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP.
If I were wanting to make a *nix daemon, I would love to have this. As I understand things, you should finish the implementation, post it to PyPI, and get people to use and test it, (all of which require no approvals). When stable, propose it for the stdlib along with a credible promise to maintain it for at least 3 years. The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not. tjr
Terry Reedy <tjreedy@udel.edu> wrote:
The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not.
Right. Personally, I think it's pretty easy to make a daemon on Unix. If this module supported both Unix and Windows, though, that would be interesting. Bill
Bill Janssen wrote:
Terry Reedy <tjreedy@udel.edu> wrote:
The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not.
Right. Personally, I think it's pretty easy to make a daemon on Unix.
It's pretty easy, but still fiddly enough that I'd rather use a library than worry about the details myself.
If this module supported both Unix and Windows, though, that would be interesting.
That would be lovely, but even a Unix-only module would be worthwhile, IMO. -Andrew.
On Sun, Jan 25, 2009 at 9:01 PM, Andrew Bennetts <andrew-pythonideas@puzzling.org> wrote:
Bill Janssen wrote:
Terry Reedy <tjreedy@udel.edu> wrote:
The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not.
Right. Personally, I think it's pretty easy to make a daemon on Unix.
It's pretty easy, but still fiddly enough that I'd rather use a library than worry about the details myself.
I strongly second this sentiment. It's such an oft-used recipe that it deserves a canonical, fully correct library implementation.
If this module supported both Unix and Windows, though, that would be interesting.
That would be lovely, but even a Unix-only module would be worthwhile, IMO.
Though it would probably improve its chances of getting added to the std lib. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com
On Sun, 25 Jan 2009 21:23:14 -0800, Chris Rebert wrote:
If this module supported both Unix and Windows, though, that would be interesting.
That would be lovely, but even a Unix-only module would be worthwhile, IMO.
Though it would probably improve its chances of getting added to the std lib.
Cheers, Chris
I think you could have a much better chance if you revise the PEP proposal by also considering Windows' daemon (i.e. Windows Service). Make it so that Unix's and Unix-like's daemon and Windows Service have as similar interface as possible (then create the implementation, caveat: "it's not going to be me").
Andrew Bennetts writes:
Bill Janssen wrote:
If this module supported both Unix and Windows, though, that would be interesting.
That would be lovely, but even a Unix-only module would be worthwhile, IMO.
Jerry Seutter writes:
It would be nice if this functionality could work on windows machine as well as unix flavors.
My knowledge of programming on Windows is quite limited. AIUI, the steps required to make a Windows program into a daemon process are markedly different from the canonical Stevens procedure, correct? In that case, how would an implementation on Windows need to change the API given in this PEP? -- \ “I wish there was a knob on the TV to turn up the intelligence. | `\ There's a knob called ‘brightness’ but it doesn't work.” | _o__) —Eugene P. Gallagher | Ben Finney
Ben Finney <ben+python@benfinney.id.au> wrote:
My knowledge of programming on Windows is quite limited. AIUI, the steps required to make a Windows program into a daemon process are markedly different from the canonical Stevens procedure, correct?
In that case, how would an implementation on Windows need to change the API given in this PEP?
Yes, that's the question. You should figure out the answer, or find some more folks who can help you figure out the answer, and update the PEP to reflect it. Bill
Bill Janssen <janssen@parc.com> writes:
Ben Finney <ben+python@benfinney.id.au> wrote:
[…] how would an implementation on Windows need to change the API given in this PEP?
Yes, that's the question. You should figure out the answer, or find some more folks who can help you figure out the answer, and update the PEP to reflect it.
Based on what I can see in the Python Cookbook (e.g. <URL:http://code.activestate.com/recipes/551780/>), Windows doesn't even call it a “daemon”, instead calling it a “service”. The customisation options and expected behaviour are wildly different from what a Unix programmer would expect. So I think that a module for creating Unix-style daemons, with API and options geared toward making that easy, has no business also getting involved with creating Windows services. It would result in an unduly complicated API for no benefit to either Unix programmers or Windows programmers. If someone else wants to offer a separate PEP to include a ‘winservice’ module to the standard library, that seems a better way forward for that use case. -- \ “I'm beginning to think that life is just one long Yoko Ono | `\ album; no rhyme or reason, just a lot of incoherent shrieks and | _o__) then it's over.” —Ian Wolff | Ben Finney
On Tue, Jan 27, 2009 at 06:53:36PM +1100, Ben Finney wrote:
Based on what I can see in the Python Cookbook (e.g. <URL:http://code.activestate.com/recipes/551780/>), Windows doesn't even call it a ???daemon???, instead calling it a ???service???. The customisation options and expected behaviour are wildly different from what a Unix programmer would expect.
Not only it's a completely different API - it requires pywin32 which is not in the stdlib.
So I think that a module for creating Unix-style daemons, with API and options geared toward making that easy, has no business also getting involved with creating Windows services. It would result in an unduly complicated API for no benefit to either Unix programmers or Windows programmers.
+1 for unix-only daemonization, -1 for making a universal unix/w32 API. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:
On Tue, Jan 27, 2009 at 06:53:36PM +1100, Ben Finney wrote:
Based on what I can see in the Python Cookbook (e.g. <URL:http://code.activestate.com/recipes/551780/>), Windows doesn't even call it a ???daemon???, instead calling it a ???service???. The customisation options and expected behaviour are wildly different from what a Unix programmer would expect.
Not only it's a completely different API - it requires pywin32 which is not in the stdlib.
So I think that a module for creating Unix-style daemons, with API and options geared toward making that easy, has no business also getting involved with creating Windows services. It would result in an unduly complicated API for no benefit to either Unix programmers or Windows programmers.
+1 for unix-only daemonization, -1 for making a universal unix/w32 API.
Meh, Unix-only stuff is such a pain because most developers or projects could cobble together Unix-only stuff themselves; the same can rarely be said for Windows-equivalent stuff. It's not hard to abstract a general service interface; Qt has been doing it for a long time: http://doc.trolltech.com/solutions/qtservice/qtservice.html Trent.
On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:
On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:
Not only it's a completely different API - it requires pywin32 which is not in the stdlib. [skip] +1 for unix-only daemonization, -1 for making a universal unix/w32 API.
It's not hard to abstract a general service interface
Do you propose to include pywin32 into python? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:
On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:
Not only it's a completely different API - it requires pywin32 which is not in the stdlib. [skip] +1 for unix-only daemonization, -1 for making a universal unix/w32 API. It's not hard to abstract a general service interface
Do you propose to include pywin32 into python?
Ummm.. Python already has a slew of windows-specific code in it. You could -- if it seemed suitable -- copy the pywin32 code with appropriate attribution, but you don't need the whole thing. And you don't necessarily need it at all. TJG
On Tue, Jan 27, 2009 at 11:23 AM, Tim Golden <mail@timgolden.me.uk> wrote:
Oleg Broytmann wrote:
On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:
On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:
Not only it's a completely different API - it requires pywin32 which is not in the stdlib.
[skip]
+1 for unix-only daemonization, -1 for making a universal unix/w32 API.
It's not hard to abstract a general service interface
Do you propose to include pywin32 into python?
Ummm.. Python already has a slew of windows-specific code in it. You could -- if it seemed suitable -- copy the pywin32 code with appropriate attribution, but you don't need the whole thing. And you don't necessarily need it at all.
TJG
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing. -jesse
Jesse Noller wrote:
On Tue, Jan 27, 2009 at 11:23 AM, Tim Golden <mail@timgolden.me.uk> wrote:
Oleg Broytmann wrote:
On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:
On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:
Not only it's a completely different API - it requires pywin32 which is not in the stdlib. [skip] +1 for unix-only daemonization, -1 for making a universal unix/w32 API. It's not hard to abstract a general service interface Do you propose to include pywin32 into python? Ummm.. Python already has a slew of windows-specific code in it. You could -- if it seemed suitable -- copy the pywin32 code with appropriate attribution, but you don't need the whole thing. And you don't necessarily need it at all.
TJG
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
I'm inclined to agree with you. Or at least I think a PEP proposing a cross-platform daemon/service framework would be quite different from the one Ben is authoring. I was really just commenting on Oleg's suggestion that anything making use of win32 code needed to use pywin32. TJG
On Tue, Jan 27, 2009 at 05:10:49PM +0000, Tim Golden wrote:
I was really just commenting on Oleg's suggestion that anything making use of win32 code needed to use pywin32.
I haven't said that. I have said (not exactly in these words) "If you want to add an implementation of w32 services to the stdlib you must (or at least should) to add pywin32 to the stdlib." Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Jesse Noller <jnoller@gmail.com> writes:
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
Thanks, this is my position also. I have documented this now, and it will appear in the next version of the PEP. -- \ “The problem with television is that the people must sit and | `\ keep their eyes glued on a screen: the average American family | _o__) hasn't time for it.” —_The New York Times_, 1939 | Ben Finney
On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:
Jesse Noller <jnoller@gmail.com> writes:
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
Thanks, this is my position also. I have documented this now, and it will appear in the next version of the PEP.
I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: the PEP gets approved, the daemon module gets written, and, because it's so useful, uptake is immediate, people start writing code that uses it. Sweet. Or is it? Don't get me wrong, I *heart* Unix. But I get paid to develop in Windows. Because frankly, that's where the big bucks are and it's an extremely satisfying transition to jettison all emotional reasoning and just work for whoever is willing to pay you the most. I highly recommend it. But for now, I digress. So, back to your Unix-only daemon module. It's good. Great, in fact. All sorts of projects start using it. Subversion uses it to write some funky persistent daemon that speeds up hooks by 8000% percent. Well that's just super! I'd like to use that for one of my clients, but, argh, no, wait, I can't, it's trying to use some daemon module that Google tells me is Unix-only. After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy, just your normal, run-of-the-mill start/stop type stuff. Why on earth, I think to myself, is this great 3rd party Python utility using a Unix-only daemon module, thus requiring me to hack it to bits in order for it to run on Windows, wasting my time and my client's time. Because it was there. Trent.
On Tue, Jan 27, 2009 at 8:29 PM, Trent Nelson <python-ideas-list@trentnelson.com> wrote:
On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:
Jesse Noller <jnoller@gmail.com> writes:
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
Thanks, this is my position also. I have documented this now, and it will appear in the next version of the PEP.
I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: the PEP gets approved, the daemon module gets written, and, because it's so useful, uptake is immediate, people start writing code that uses it.
Or is it? Don't get me wrong, I *heart* Unix. But I get paid to develop in Windows. Because frankly, that's where the big bucks are and it's an extremely satisfying transition to jettison all emotional reasoning and just work for whoever is willing to pay you the most. I highly recommend it. But for now, I digress.
Don't worry; I think most of us would gladly work for money. In fact, I would hazard most of us like money a lot.
So, back to your Unix-only daemon module. It's good. Great, in fact. All sorts of projects start using it. Subversion uses it to write some funky persistent daemon that speeds up hooks by 8000% percent. Well that's just super! I'd like to use that for one of my clients, but, argh, no, wait, I can't, it's trying to use some daemon module that Google tells me is Unix-only.
After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy, just your normal, run-of-the-mill start/stop type stuff. Why on earth, I think to myself, is this great 3rd party Python utility using a Unix-only daemon module, thus requiring me to hack it to bits in order for it to run on Windows, wasting my time and my client's time.
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking. A unix-only daemon - and then a complimenting windows-only service library - makes sense. Ideally, they would have close to the same API, but I don't think log jamming all of the vagaries of both into one beast makes sense. Then again, my windows-ability is out of date and rusty. -jesse
On Tue, Jan 27, 2009 at 08:49:52PM -0500, Jesse Noller wrote:
On Tue, Jan 27, 2009 at 8:29 PM, Trent Nelson <python-ideas-list@trentnelson.com> wrote:
On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:
Jesse Noller <jnoller@gmail.com> writes:
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.
A unix-only daemon - and then a complimenting windows-only service library - makes sense. Ideally, they would have close to the same API, but I don't think log jamming all of the vagaries of both into one beast makes sense. Then again, my windows-ability is out of date and rusty.
See, I disagree with that as well. Take a look at what Qt did with their QtService class: http://doc.trolltech.com/solutions/qtservice/qtservice.html The Trolls are particularly good at abstracting wildly different Unix/Windows-isms into a single, usable, unified interface. The key is not thinking in terms of writing a Windows service or a Unix daemon, but in terms of providing an abstraction that makes sense to implement as a daemon on Unix and a service on Windows. I'd like to hear arguments for why that QtService interface is perceived as being deficient. If the argument is simply that the PEP author is not familiar with Windows, then I'm sure those of us that are would much rather help out with a cross-platform solution than have yet another avenue for people to write Python code that doesn't work on the most widely deployed platform in the Universe.* Trent. [*] Citation needed.
Trent Nelson writes:
On Tue, Jan 27, 2009 at 08:49:52PM -0500, Jesse Noller wrote:
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.
A unix-only daemon - and then a complimenting windows-only service library - makes sense. Ideally, they would have close to the same API, but I don't think log jamming all of the vagaries of both into one beast makes sense. Then again, my windows-ability is out of date and rusty.
See, I disagree with that as well. Take a look at what Qt did with their QtService class:
Thanks for mentioning that again; I saw another reference to it earlier, but didn't respond.
I'd like to hear arguments for why that QtService interface is perceived as being deficient.
Off the top of my head: The QtService model seems to be “start an additional process and maintain it at arms length, with a bunch of communication channels into and out of it”. Contrariwise, this PEP addresses the use case of “turn the *current* program into a daemon process, so that everything further this program does is running inside that process”. It's too big, as a result of the above difference in focus. When writing a Unix daemon program, I don't want the great majority of what that interface offers. It misses a lot, again as a result of the mismatch in purpose. After setting up a QtService, I need to continue chattering with it to find out what's going on inside it, and to do things like change its state or log a message. The current PEP, on the other hand, doesn't have the model of managing a separate process; the daemon is the *current* program, so one can just do things like output or state changes as one normally would. In short: It addresses a different problem to what I'm addressing with this PEP. -- \ “If nature has made any one thing less susceptible than all | `\ others of exclusive property, it is the action of the thinking | _o__) power called an idea” —Thomas Jefferson | Ben Finney
Jesse Noller <jnoller@...> writes:
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.
That would be like a portable library which would allow to write multiprocess programs by glossing over the differences between Unix and Windows processes... Wouldn't it?
On Jan 28, 2009, at 7:57 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Jesse Noller <jnoller@...> writes:
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.
That would be like a portable library which would allow to write multiprocess programs by glossing over the differences between Unix and Windows processes... Wouldn't it?
Touché good sir, touché
On Wed, Jan 28, 2009 at 7:57 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Jesse Noller <jnoller@...> writes:
Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.
That would be like a portable library which would allow to write multiprocess programs by glossing over the differences between Unix and Windows processes... Wouldn't it?
Since Antoine called me to the mat on this one, and I've been chewing on this more today - I think the cross-platformedness (is that a word?) idea is a good one, even if the Qt design doesn't necessarily apply here completely, having a single abstraction to both (but different modules under that abstraction based on os) is attractive (as Stephen pointed out). Heck, a daemon for OS/X is going to need it's own underlying implementation as well: the proper way to install a daemon/service is via launchd [1] which I had to monkey with both for the OS/X buildbot I have, and for an application today. jesse [1] http://developer.apple.com/documentation/Darwin/Reference/ManPages/man8/laun...
Jesse Noller wrote:
Since Antoine called me to the mat on this one, and I've been chewing on this more today - I think the cross-platformedness (is that a word?) idea is a good one, even if the Qt design doesn't necessarily apply here completely, having a single abstraction to both (but different modules under that abstraction based on os) is attractive (as Stephen pointed out).
Perhaps such an abstraction should be built on top of Ben Finney's proposed unix daemon module, rather than built in? -Andrew.
On Wed, Jan 28, 2009 at 8:57 PM, Andrew Bennetts <andrew-pythonideas@puzzling.org> wrote:
Jesse Noller wrote:
Since Antoine called me to the mat on this one, and I've been chewing on this more today - I think the cross-platformedness (is that a word?) idea is a good one, even if the Qt design doesn't necessarily apply here completely, having a single abstraction to both (but different modules under that abstraction based on os) is attractive (as Stephen pointed out).
Perhaps such an abstraction should be built on top of Ben Finney's proposed unix daemon module, rather than built in?
-Andrew.
Perhaps; but it makes sense to nest it under the abstraction - daemons.unixdaemon/winservice/osxdaemon
Jesse Noller <jnoller@gmail.com> writes:
Perhaps; but it makes sense to nest it under the abstraction - daemons.unixdaemon/winservice/osxdaemon
I really don't think it's helpful to conflate this “service” model with the much simpler concept of a Unix daemon. To implement a service on Unix, it's most logical to use a daemon; but the reverse is definitely not true (which is a large part of my resistance to expanding the current PEP to conflate the two concepts). Instead, I think that people who want a “service” would be better served by a top-level module named ‘service’, to preserve that concept as separate from the concept of a daemon. In other words, my understanding is: ‘import service’ should get an API (so far in no PEP that I know of) for a distinct purpose, that of implementing an MS Windows-style service, which uses whatever OS-specific implementation makes sense. ‘import daemon’ gets a totally different API for a different purpose, that of turning the current running program into a daemon, which AFAICT only makes a whole lot of sense on Unix and is what I'm proposing in this PEP. It makes much more sense for ‘daemon’ to stay simple and continue to mean the Unix-specific concept of “daemon”, without necessarily obstructing whoever wants to implement ‘service’, than for ‘daemon’ in Python to mean something other than the Unix meaning. -- \ “No matter how far down the wrong road you've gone, turn back.” | `\ —Turkish proverb | _o__) | Ben Finney
Ben Finney <ben+python@benfinney.id.au> wrote:
‘import daemon’ gets a totally different API for a different purpose, that of turning the current running program into a daemon, which AFAICT only makes a whole lot of sense on Unix and is what I'm proposing in this PEP.
I think that's an implementation detail. Unix daemons and Windows services seem more similar than different. The details about how they are established/started are different, but that's what a good API covers over. That would be worth designing. There are a number of Unix-only modules for this purpose already, aren't there? I see at least three of them on PyPI, including yours, Ben. What's the point of a new one? And what's the point of adding a new Unix-only module to the stdlib? Aren't we trying to remove platform-only modules? Didn't we remove the Mac modules in 3.x? And isn't win32all a separate distribution? Bill
Bill Janssen <janssen@parc.com> writes:
There are a number of Unix-only modules for this purpose already, aren't there? I see at least three of them on PyPI, including yours, Ben.
Yes, none of which meet the criteria listed in the PEP for “well-behaved daemon”. They're useful, to be sure; but they're lacking when judged against what a well-behaved Unix daemon should be doing.
What's the point of a new one?
Perhaps I've mistakenly given the impression that I'm re-inventing the wheel; sorry for that false impression if so. To implement this PEP fully, I've taken the existing implementation that was closest to the requirements (‘bda.daemon’), and based this work on altering it to conform with this PEP. I'm in active correspondence with the author of ‘bda.daemon’.
And what's the point of adding a new Unix-only module to the stdlib?
To have this common, fairly standardised but tricky, use case implemented once and easily available to any Python programmer, without them needing to know the fine details of what's required to make it work internally. Having multiple partial implementations in PyPI and the Cookbook leads only to the situation where a prospective user has to either know intimate details of this rather fiddly task to judge each of them, or make a choice arbitrarily. -- \ “I don't know half of you half as well as I should like, and I | `\ like less than half of you half as well as you deserve.” —Bilbo | _o__) Baggins | Ben Finney
Ben Finney <ben+python@...> writes:
Instead, I think that people who want a “service” would be better served by a top-level module named ‘service’, to preserve that concept as separate from the concept of a daemon.
I may be misstaken, but I think the usual desire is to have *something* which runs in the background and doesn't get killed when you kill the parent application or terminal which launched it. Whatever it is implemented in terms of "daemon", "service" or anything else doesn't really matter in those cases.
On Thu, Jan 29, 2009 at 01:35:21PM +1100, Ben Finney wrote:
It makes much more sense for ???daemon??? to stay simple and continue to mean the Unix-specific concept of ???daemon???, without necessarily obstructing whoever wants to implement ???service???, than for ???daemon??? in Python to mean something other than the Unix meaning.
There are two types of developers that will utilise this module. Those that are writing something for themselves, that never has to run anywhere else other than Unix. Or those that work primarily or exclusively in Unix, but understand that others might want to run their code elsewhere. If you build it, they will come. By building the latter solution, sure, you've inconvenienced the guy that wants a Unix-only daemon, but you've also enabled everyone else who wants to write stuff that has a chance of running unmodified on other platforms. And jumping on the bandwagon of a few others; I like the argument that something Unix-only like this could just as easily belong in PyPI, whereas a cross-platform solution is more viable for stdlib inclusion. We should make it harder to write non-portable code, not easier. Trent.
Trent Nelson writes:
There are two types of developers that will utilise this module. Those that are writing something for themselves, that never has to run anywhere else other than Unix.
Surely there is also the type that are writing *for others*, and want the program to be both well-behaved and work on any Unix-like OS?
Or those that work primarily or exclusively in Unix, but understand that others might want to run their code elsewhere.
Those people would be well served by a putative “service” interface. I would welcome such a PEP from you, since you seem to have fairly solid ideas of what it should look like.
We should make it harder to write non-portable code, not easier.
It's a bit of a stretch to go from “designed to work on any Unix-like OS” to “non-portable”. -- \ “Under democracy one party always devotes its chief energies to | `\ trying to prove that the other party is unfit to rule — and | _o__) both commonly succeed, and are right.” —Henry L. Mencken | Ben Finney
Ben Finney writes:
Those people would be well served by a putative "service" interface. I would welcome such a PEP from you, since you seem to have fairly solid ideas of what it should look like.
AFAICS it looks like yours, except with the requirement of "works on Windows" and a strong preference for "no other-OS-specific clutter with a simple 'import daemonize'". Bill added "plays nicely with Mac OS X". I don't see on what grounds you object to exploring the possibility, since you say you don't know much about Windows services. Note that nobody is suggesting that you write the code that implements daemons via Windows services (of course they'd love it if you did!), only that you open up the API design to changes that would make some programs work on all platforms without conditioning on platform, and make all programs work with minimal platform-specific code. So I suggest that you bat the ball back into their court, and announce on Python-Dev that you're willing but not able to generalize your PEP to Windows, and ask for coauthors to help. If there are no takers, then you're in a strong position to argue that Unix-like OSes are the important case anyway, nobody cares about Windows services in Python. If there are, then somebody else does the work and everybody is happy. No?
We should make it harder to write non-portable code, not easier.
It's a bit of a stretch to go from "designed to work on any Unix-like OS" to "non-portable".
That is not a Pythonic attitude. Here, "portable code" means "designed to work in any instance of Python conditioning only on 'this instance supports the feature'."
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
Ben Finney writes:
Those people would be well served by a putative "service" interface. I would welcome such a PEP from you, since you seem to have fairly solid ideas of what it should look like.
AFAICS it looks like yours
I can't see how anyone could compare the latest version of this PEP with the suggestions being made by Trent for an API, and conclude that they are similar. This might because the PEP isn't communicating very well. Does the section “A daemon is not a service” help to explain the fundamental difference between the model suggested by Trent, versus the model in the PEP? If not, is it made clearer by the addition of nearly a dozen extra methods in Trent's suggestion, and several extra concepts, that are extraneous to the task described in the PEP? All of which seem ideally suited to a putative “service” API.
I don't see on what grounds you object to exploring the possibility, since you say you don't know much about Windows services.
I have no objection to someone exploring a “service” interface. Indeed, I applaud such efforts. But that's not what I'm addressing with this PEP. As for the possibility of using the daemon interface as the Unix-specific implementation of the background-process *component* of a service interface, I believe it would be very well suited for that. But I am implementing the daemon API *now* because it's a distinct task, that I've seen needed many times and implemented poorly in many places. I would be delighted if someone who knows what they want from a “service” interface could build that, entirely different, API using a daemon as part of the implementation.
Note that nobody is suggesting that you write the code that implements daemons via Windows services (of course they'd love it if you did!), only that you open up the API design to changes that would make some programs work on all platforms without conditioning on platform, and make all programs work with minimal platform-specific code.
I maintain that it would be better to specify a platform-agnostic “service” API, that uses the “daemon” API as the Unix-specific implementation of the background-process component of the service. Meanwhile, I and others have a need for a simple, standardised, one-obvious-way “daemonise the current program” library, *entirely distinct from* any of ther extra baggage that comes with a “spawn a separate process that I can then communicate with through channels” API. I intend to satisfy that with this PEP. -- \ “People demand freedom of speech to make up for the freedom of | `\ thought which they avoid.” —Soren Aabye Kierkegaard (1813-1855) | _o__) | Ben Finney
Ben Finney writes:
[C]ompare the latest version of this PEP [...].
I'm assuming the one posted on 1/29 in <87iqnywsx2.fsf@benfinney.id.au>, or fairly close to that.
This might because the PEP isn't communicating very well. Does the section "A daemon is not a service" help to explain the fundamental difference between the model suggested by Trent, versus the model in the PEP?
That is quite clear, and you've said "daemonize-me != start-a- service" in discussion here. However, you've also said that "daemon != service" (including in the title of that section!), which clearly is mostly false when those terms are referring to the processes that are running in the background. I think that focusing on the distinction between "*becoming* a daemon" and "*starting* a service" is distracting. The point being that *becoming* a well-behaved service surely requires becoming a well- behaved background process. The real issue is not starting a separate process (just don't provide a way to do it), but rather it is that it also requires all the communication setup as well, which is out-of- scope for a daemonization PEP on any platform (apart from closing file descriptors, which is specific to the fork(2) implementation of process creation). To prevent the misinterpretation I made, I would phrase that title "A Service is Not Just a Daemon", implying "this PEP is not a full-service PEP".<wink> BTW, IMO you can delete the sentence "It is the opinion of this PEP's author that combining the two disparate systems of behaviour under a single API would be contrary to the focus of this PEP." from the PEP. It's redundant.
Meanwhile, I and others have a need for a simple, standardised, one-obvious-way "daemonise the current program" library, *entirely distinct from* any of ther extra baggage that comes with a "spawn a separate process that I can then communicate with through channels" API. I intend to satisfy that with this PEP.
Well, that's what I thought was in question here. It seems to me that there are likely to be additional aspects to "daemonize me" for Mac OS X and Windows, but that the basic start-reload-stop methods should probably apply. I'm not familiar enough with the process of daemonization on Mac OS X (Bill mentioned interaction with launchd) or Windows to help at reasonable expense, but I certainly hope that those who are will review to see if a similar API might be useful to them. Unfortunately there doesn't seem to be that much common ground among the advocates, though. Steve
It's tough to follow this whole thread as it has spread all over the place, so my apologies if this has been mentioned. Why not try to make both the unix only daemon and the generic service available? The latter could use the former for its implementation? Nate
nathan binkert <nate@binkert.org> writes:
It's tough to follow this whole thread as it has spread all over the place, so my apologies if this has been mentioned.
Why not try to make both the unix only daemon and the generic service available? The latter could use the former for its implementation?
Yes, this is my position. I'm glad to see that I'm not alone, thanks :-) -- \ “Puritanism: The haunting fear that someone, somewhere, may be | `\ happy.” —Henry L. Mencken | _o__) | Ben Finney
Andrew Bennetts writes:
Jesse Noller wrote:
having a single abstraction to both
Both? What is the referent here? (I couldn't find any in the context provided.)
(but different modules under that abstraction based on os) is attractive (as Stephen pointed out).
Perhaps such an abstraction should be built on top of Ben Finney's proposed unix daemon module, rather than built in?
+1. It seems Jesse and I have been simultaneously masticating. Having now chewed on this issue overnight, I can't see any reason why, given the existence of an implementation of the current PEP, that it couldn't be used to implement some *other* PEP dealing with a separate-process, service-based model. That I can't think of any reason why not is likely to mean that there are half a dozen reasons I'm unaware of :-) -- \ “I went to the museum where they had all the heads and arms | `\ from the statues that are in all the other museums.” —Steven | _o__) Wright | Ben Finney
Trent Nelson writes:
On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:
Jesse Noller writes:
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
Thanks, this is my position also.
I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: […]
I respectfully suggest that this isn't true, because your hypothetical situation continues:
After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy,
Your disagreement seems to rest on this assertion, and I don't see how your hypothetical described can be true. Do you really see the described behaviour in the current draft PEP as “not doing anything uniquely Unix-y”?
just your normal, run-of-the-mill start/stop type stuff.
As far as I can tell, beyond trivial use cases, there is both wide discrepancy and very little overlap in “start/stop type stuff” between a Unix daemon and an MS Windows service. At least, none of the (admittedly few) instructions on MS Windows services I've seen demonstrate otherwise. Someone who thinks otherwise is welcome to attempt a PEP that brings all that behaviour under a single banner *and* design the API to be both flexible enough to be useful in many use cases and simple enough to be worth learning instead of doing it manually. That person will not be me. -- \ “Contentment is a pearl of great price, and whosoever procures | `\ it at the expense of ten thousand desires makes a wise and | _o__) happy purchase.” —J. Balguy | Ben Finney
On Wed, Jan 28, 2009 at 01:15:41PM +1100, Ben Finney wrote:
Trent Nelson writes:
On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:
Jesse Noller writes:
I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.
Thanks, this is my position also.
I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: [???]
I respectfully suggest that this isn't true, because your hypothetical situation continues:
After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy,
Your disagreement seems to rest on this assertion, and I don't see how your hypothetical described can be true. Do you really see the described behaviour in the current draft PEP as ???not doing anything uniquely Unix-y????
No, the PEP is nothing but Unix-isms, that's my point. The 3rd party code I'm hypothetically wading through has been coded to that interface, but semantically, it's not doing anything unique to Unix.
`prevent_core` :Default: ``True``
If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`.
I was going to suggest service.set_secure(True); but then I wondered what is this actually achieving? Won't the resulting .core will be owned by root?
`lockfile_directory` :Default: ``'/var/run'``
Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped.
`lockfile_name` :Default: ``None``
Base name of the lockfile for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line.
All you're doing is ensuring multiple instances of the daemon don't run. On Windows, you'd use a mutex, on Unix, sure, lockfile makes sense. A cross-platform interface wouldn't bother exposing this implementation detail.
just your normal, run-of-the-mill start/stop type stuff.
As far as I can tell, beyond trivial use cases, there is both wide discrepancy and very little overlap in ???start/stop type stuff??? between a Unix daemon and an MS Windows service. At least, none of the (admittedly few) instructions on MS Windows services I've seen demonstrate otherwise.
Semantically the overlap is almost identical, which is why I'm interested in hearing why the QtService-type approach couldn't be leveraged here. (The implementation details differ wildly, which is what you're referring to.)
Someone who thinks otherwise is welcome to attempt a PEP that brings all that behaviour under a single banner *and* design the API to be both flexible enough to be useful in many use cases and simple enough to be worth learning instead of doing it manually. That person will not be me.
class Service(object): EventType = namedtuple( Success = 1, Error = 2, Warning = 3, Information = 4, ) StartupType = namedtuple( Auto = 0, Manual = 1 ) def __init__(self, name, desc=None, startup_type=StartupType.Auto) self.name = name self.description = desc def parse_arguments(args): pass def is_running(self): pass def is_installed(self): pass def install(self): pass def uninstall(self): pass def report_event(self, msg, event_type, id, category, data): pass def exec(args): pass def terminate(): pass def request_pause(): pass def request_resume(): pass def send_command(code): pass def run(args): raise NotImplementedError() def start(): pass def stop(): pass def pause(): pass def resume(): pass }:> Trent.
(I'm only addressing commentary on the current PEP here. Comparisons to MS Windows services have been addressed in other messages.) Trent Nelson writes:
On Wed, Jan 28, 2009 at 01:15:41PM +1100, Ben Finney wrote:
`prevent_core` :Default: ``True``
If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`.
I was going to suggest service.set_secure(True)
That name implies something emphatically not true: that a program's security is a binary toggle, and it can be turned on or off.
but then I wondered what is this actually achieving? Won't the resulting .core will be owned by root?
Yes. Which is less secure than never creating the core file at all.
`lockfile_directory` :Default: ``'/var/run'``
Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped.
`lockfile_name` :Default: ``None``
Base name of the lockfile for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line.
All you're doing is ensuring multiple instances of the daemon don't run.
Also allowing the location and name of the lock file to be customised.
On Windows, you'd use a mutex, on Unix, sure, lockfile makes sense. A cross-platform interface wouldn't bother exposing this implementation detail.
The details are exposed precisely so that they can be customised by programs that need it. This is common, for example, to allow for differing conventions or standards of filesystem layout. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but I find scratching just makes it worse.” —_Pinky and | _o__) The Brain_ | Ben Finney
On Wed, Jan 28, 2009 at 02:24:27PM +1100, Ben Finney wrote:
(I'm only addressing commentary on the current PEP here. Comparisons to MS Windows services have been addressed in other messages.)
(I've run out of steam trying to argue the virtues of cross- platform modules, so I'm only commenting on Ben's commentary on my commentary.)
Trent Nelson writes:
On Wed, Jan 28, 2009 at 01:15:41PM +1100, Ben Finney wrote:
`prevent_core` :Default: ``True``
If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`.
I was going to suggest service.set_secure(True)
That name implies something emphatically not true: that a program's security is a binary toggle, and it can be turned on or off.
but then I wondered what is this actually achieving? Won't the resulting .core will be owned by root?
Yes. Which is less secure than never creating the core file at all.
I must be missing something. This seems like a false sense of security. The core file is owned by root. Someone will need root access in order to read it. If someone has root access, and has malevolent intentions, it's game over, .core file or not. Trent.
Ben Finney writes:
The details [of the lockfiles implementing a mutex] are exposed precisely so that they can be customised by programs that need it. This is common, for example, to allow for differing conventions or standards of filesystem layout.
Sure, but the question being asked is why should users of the daemon process library need to worry about that, separately from other users of lockfiles or mutexes? If they need to worry about it only because there's no satisfactory module implementing it widely available, then that could be done in a subsidiary module that handles OS-specific details, and optionally exposes OS-specific special features (a la os and os.posix). That would also make refactoring (close to) trivial when such a module did become widely available. It's obvious you don't want to go in that direction, which is IMHO good and sufficient reason for you not to do so. But as it stands IMO it's a weakness in the PEP, even if Unix-only is otherwise acceptable for the stdlib.
"Stephen J. Turnbull" writes:
the question being asked is why should users of the daemon process library need to worry about that, separately from other users of lockfiles or mutexes?
Perhaps my mistake is in calling it a “lockfile”. It's actually a rather specific Unix-native concept: the “pidfile”, a combination sentinel, lockfile, and primitive text interface. It isn't even used much like other lockfiles (in that it wouldn't make sense, for example to wrap a context manager around it). I might return the terminology in the PEP back to “pidfile” to be clear. To answer the question, the location of a pidfile warrants specific mention (separate from discussion of lockfiles) in at least two OS standards that I'm aware of, as well as howtos and other discussions. For that reason, it's better to have a simple way to tell a program the desired location and name of its pidfile in particular. -- \ “[I]t is impossible for anyone to begin to learn that which he | `\ thinks he already knows.” —Epictetus, _Discourses_ | _o__) | Ben Finney
On Thu, Jan 29, 2009 at 01:42:44PM +1100, Ben Finney wrote:
rather specific Unix-native concept: the "pidfile", a combination sentinel, lockfile, and primitive text interface.
IWBN to have a main() function in the module to be used from the command line: $ python -m daemon --pidfile=/var/run/mydaemon.pid status Daemon pid 1221 is running $ python -m daemon --pidfile=/var/run/mydaemon.pid stop $ python -m daemon --pidfile=/var/run/mydaemon.pid status Cannot find /var/run/mydaemon.pid A note on implementation detail: IWBN if the pidfile is removed on stop(). Please do not put the burden unto the user.
I might return the terminology in the PEP back to "pidfile" to be clear.
+1 Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann <phd@phd.pp.ru> writes:
IWBN to have a main() function in the module to be used from the command line:
$ python -m daemon --pidfile=/var/run/mydaemon.pid status Daemon pid 1221 is running
$ python -m daemon --pidfile=/var/run/mydaemon.pid stop
$ python -m daemon --pidfile=/var/run/mydaemon.pid status Cannot find /var/run/mydaemon.pid
Interesting. Should that be defined in this PEP, or merely added to the reference implementation as a bonus?
A note on implementation detail: IWBN if the pidfile is removed on stop(). Please do not put the burden unto the user.
Yes, that's already addressed in the defined behaviour for `stop()`. -- \ “Know what I hate most? Rhetorical questions.” —Henry N. Camp | `\ | _o__) | Ben Finney
On Fri, Jan 30, 2009 at 08:14:07AM +1100, Ben Finney wrote:
Oleg Broytmann <phd@phd.pp.ru> writes:
IWBN to have a main() function in the module to be used from the command line:
$ python -m daemon --pidfile=/var/run/mydaemon.pid status Daemon pid 1221 is running
$ python -m daemon --pidfile=/var/run/mydaemon.pid stop
$ python -m daemon --pidfile=/var/run/mydaemon.pid status Cannot find /var/run/mydaemon.pid
Interesting. Should that be defined in this PEP, or merely added to the reference implementation as a bonus?
I think an implementation would be enough.
A note on implementation detail: IWBN if the pidfile is removed on stop(). Please do not put the burden unto the user.
Yes, that's already addressed in the defined behaviour for `stop()`.
Thank you! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
"Stephen J. Turnbull" writes:
If [the daemon interface deals with file locking] only because there's no satisfactory module implementing it widely available, then that could be done in a subsidiary module that handles OS-specific details, and optionally exposes OS-specific special features (a la os and os.posix). That would also make refactoring (close to) trivial when such a module did become widely available.
Skip Montanaro has a PyPI package, ‘lockfile’ (currently marked “Beta”) <URL:http://pypi.python.org/pypi/lockfile> that has such ambitions. I think the interface he proposes is elegant and I intend to submit an additional “PID File lock” capability, to cover the behaviour expected of PID files in particular. How should I alter this PEP if I'm proposing to use functionality currently present in another package that exists only in PyPI? Should I make an even lower-level PEP to suggest inclusion of Skip's ‘lockfile’ package? Is that wise? Is it polite? -- \ “Are you pondering what I'm pondering?” “Well, I think so, | `\ Brain, but if Jimmy cracks corn, and no one cares, why does he | _o__) keep doing it?” —_Pinky and The Brain_ | Ben Finney
Ben Finney writes:
Skip Montanaro has a PyPI package, ‘lockfile’ (currently marked "Beta") <URL:http://pypi.python.org/pypi/lockfile> that has such ambitions.
Should I make an even lower-level PEP to suggest inclusion of Skip's ‘lockfile’ package? Is that wise? Is it polite?
I would send your proposal for PIDlockfile to Skip. In your PEP, describe the relevant parts of the API explicitly in the PEP, with a footnote that you are deliberately extending the lockfile interface and currently intend to remain compatible with that interface. Then talk to Skip about a PEP for including his package, or perhaps extending your PEP to include it. Which is better depends on details and how aggressively Skip would want to pursue inclusion of lockfile if your daemon package doesn't make it or takes a long time. I don't know about "polite", but I think it would be unwise to try to PEP lockfile without Skip's explicit support.
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
Ben Finney writes:
Skip Montanaro has a PyPI package, ‘lockfile’ (currently marked "Beta") <URL:http://pypi.python.org/pypi/lockfile> that has such ambitions.
I would send your proposal for PIDlockfile to Skip. […]
For those people who have asked about the status of this: Yes, I'm currently working with Skip on the ‘lockfile’ package so that it can be used for the ‘daemon’ implementation. -- \ “Following fashion and the status quo is easy. Thinking about | `\ your users' lives and creating something practical is much | _o__) harder.” —Ryan Singer, 2008-07-09 | Ben Finney
Ben> For those people who have asked about the status of this: Yes, I'm Ben> currently working with Skip on the ?lockfile? package so that it Ben> can be used for the ?daemon? implementation. Indeed. I should have my Mercurial repository in a more globally visible place later today so Ben and anyone else interested can bash bits. Skip
> I am preparing a PEP, and corresponding reference implementation, for > a standard implementation of the steps needed to turn a program into a > well-behaved Unix daemon. I missed Ben's original post and am now reading mail offline so I can't dredge it up, but I've used a module called daemon from PyPI. Might well be worth a look as an existing proof of concept if nothing else. -- Skip Montanaro - skip@pobox.com - http://smontanaro.dyndns.org/
Terry Reedy <tjreedy@udel.edu> writes:
If I were wanting to make a *nix daemon, I would love to have this.
I'm glad to have many comments like this in support of this PEP. Thanks, everyone!
As I understand things, you should finish the implementation, post it to PyPI, and get people to use and test it, (all of which require no approvals).
Sure, but before I “finish” the implementation, I want to reach consensus on the suitability of the proposed API. To quote PEP 1: It is better to finish the specification and rationale first and reach consensus on it before writing code.
When stable, propose it for the stdlib along with a credible promise to maintain it for at least 3 years.
My intention is to continue supporting this code after inclusion in the standard library, yes. -- \ “Liberty, n. One of imagination's most precious possessions.” | `\ —Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney
Ben Finney writes:
Sure, but before I "finish" the implementation, I want to reach consensus on the suitability of the proposed API. To quote PEP 1:
It is better to finish the specification and rationale first and reach consensus on it before writing code.
The people who are posting "show us the code" are mostly folks with long experience in the PEP process. I think you should take it as an indication that in this case the definition of "suitable API" is an "I'll know it when I've tried it" kind of thing. What you might find useful is to avoid writing any code yourself, but see how well your proposed spec fits the existing code, of which several (potential) examples have been posted.
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
The people who are posting "show us the code" are mostly folks with long experience in the PEP process. I think you should take it as an indication that in this case the definition of "suitable API" is an "I'll know it when I've tried it" kind of thing.
I am working on ‘python-daemon’ concurrently with this PEP now, and intend to keep publishing updates to that implementation as it approaches conformance with the PEP.
What you might find useful is to avoid writing any code yourself, but see how well your proposed spec fits the existing code, of which several (potential) examples have been posted.
Fortunately I've based ‘python-daemon’ on what I determined to be the closest starting point for the well-behaved daemon behaviour. -- \ “Any intelligent fool can make things bigger and more complex… | `\ It takes a touch of genius – and a lot of courage – to move in | _o__) the opposite direction.” —Albert Einstein | Ben Finney
On Mon, Jan 26, 2009 at 09:11:10AM +1100, Ben Finney wrote:
.. [cookbook-66012]
Many good ideas were contributed by the community to Python cookbook recipe 66012, ???Fork a daemon process on Unix??? `<http://code.activestate.com/recipes/66012/>`_.
I use this one (with some local modifications) and I'm pretty happy about it. Having such a module in the std lib would be a big plus. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
This message is a call for comments, support, and suggestions, prior to submitting this PEP to the editor.
I would like to note that there are many existing packages that provide support for daemonization (including one I wrote for Zope long ago, which still appears to be around under the name zdaemon). Twisted also seems to have one. I also recall hearing a talk at PyCon a few years ago about a descendant of my zdaemon on steroids. I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals. The basics are very simple (just follow the recipe from the Stevens book), but that's a bit rough, and beyond that everyone seems to have a different idea on what use cases need to be support and what the most flexible API is to support all desirable use cases. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
I would like to note that there are many existing packages that provide support for daemonization […]
Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found). I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.
I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.
This suspicion could well be true. On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases. Is ‘python-ideas’ the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to ‘comp.lang.python’? -- \ “Pinky, are you pondering what I'm pondering?” “Well, I think | `\ so, Brain, but it's a miracle that this one grew back.” —_Pinky | _o__) and The Brain_ | Ben Finney
On Mon, Jan 26, 2009 at 2:40 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Guido van Rossum writes:
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
I would like to note that there are many existing packages that provide support for daemonization […]
Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).
I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.
I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.
This suspicion could well be true.
On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.
Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?
Depends on how much bikeshedding you want. I expect that you'll get enough of that without involving c.k.py... -- --Guido van Rossum (home page: http://www.python.org/~guido/)
It would be nice if this functionality could work on windows machine as well as unix flavors. I wrote the package at http://daemonize.sourceforge.net in frustration after seeing daemons built poorly several times. Some of the instances had been done incorrectly even after the developer had been pointed to the ActiveState page. Understanding why a daemon should be written the way it is requires a fair bit of historical knowledge that younger people occasionally do not have. Some of the recent grads hired at my workplace did not know what a daemon was. "Oh, like a web container?" Jerry Seutter On Mon, Jan 26, 2009 at 3:40 PM, Ben Finney <ben+python@benfinney.id.au<ben%2Bpython@benfinney.id.au>
wrote:
Guido van Rossum writes:
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
I would like to note that there are many existing packages that provide support for daemonization […]
Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).
I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.
I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.
This suspicion could well be true.
On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.
Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?
-- \ "Pinky, are you pondering what I'm pondering?" "Well, I think | `\ so, Brain, but it's a miracle that this one grew back." —_Pinky | _o__) and The Brain_ | Ben Finney
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Was that you in the YouTube video that was one of the first hits for "PyCon daemon" ? That was hilarious. :-) On Mon, Jan 26, 2009 at 2:55 PM, Jerry Seutter <jseutter@gmail.com> wrote:
It would be nice if this functionality could work on windows machine as well as unix flavors.
I wrote the package at http://daemonize.sourceforge.net in frustration after seeing daemons built poorly several times. Some of the instances had been done incorrectly even after the developer had been pointed to the ActiveState page. Understanding why a daemon should be written the way it is requires a fair bit of historical knowledge that younger people occasionally do not have. Some of the recent grads hired at my workplace did not know what a daemon was. "Oh, like a web container?" Jerry Seutter On Mon, Jan 26, 2009 at 3:40 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Guido van Rossum writes:
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
I would like to note that there are many existing packages that provide support for daemonization […]
Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).
I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.
I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.
This suspicion could well be true.
On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.
Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?
-- \ "Pinky, are you pondering what I'm pondering?" "Well, I think | `\ so, Brain, but it's a miracle that this one grew back." —_Pinky | _o__) and The Brain_ | Ben Finney
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Yeah, that was me. :) On Mon, Jan 26, 2009 at 4:02 PM, Guido van Rossum <guido@python.org> wrote:
Was that you in the YouTube video that was one of the first hits for "PyCon daemon" ? That was hilarious. :-)
On Mon, Jan 26, 2009 at 2:55 PM, Jerry Seutter <jseutter@gmail.com> wrote:
It would be nice if this functionality could work on windows machine as well as unix flavors.
I wrote the package at http://daemonize.sourceforge.net in frustration after seeing daemons built poorly several times. Some of the instances had been done incorrectly even after the developer had been pointed to the ActiveState page. Understanding why a daemon should be written the way it is requires a fair bit of historical knowledge that younger people occasionally do not have. Some of the recent grads hired at my workplace did not know what a daemon was. "Oh, like a web container?" Jerry Seutter On Mon, Jan 26, 2009 at 3:40 PM, Ben Finney <ben+python@benfinney.id.au<ben%2Bpython@benfinney.id.au>
wrote:
Guido van Rossum writes:
On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:
I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.
I would like to note that there are many existing packages that provide support for daemonization […]
Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).
I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.
I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.
This suspicion could well be true.
On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.
Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?
-- \ "Pinky, are you pondering what I'm pondering?" "Well, I think | `\ so, Brain, but it's a miracle that this one grew back." —_Pinky | _o__) and The Brain_ | Ben Finney
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Significant changes: new section “Other daemon implementations” with brief descriptions of their relationship to this PEP. :PEP: XXX :Title: Standard daemon process library :Version: 0.2 :Last-Modified: 2009-01-27 11:28 :Author: Ben Finney <ben+python@benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 2009-01-26 :Python-Version: 3.1 :Post-History: ======== Abstract ======== Writing a program to become a well-behaved Unix daemon is somewhat complex and tricky to get right, yet the steps are largely similar for any daemon regardless of what else the program may need to do. This PEP introduces a module to the Python standard library that provides a simple interface to the task of becoming a daemon process. .. contents:: .. Table of Contents: Abstract Specification Example usage Interface ``Daemon`` objects ``DaemonError`` objects Motivation Rationale Correct daemon behaviour Reference Implementation Other daemon implementations References Copyright ============= Specification ============= Example usage ============= Simple example of usage:: import daemon from spam import do_main_program this_daemon = daemon.Daemon() this_daemon.start() do_main_program() More complex example usage:: import os import grp import signal import daemon from spam import ( initial_program_setup, do_main_program, program_cleanup, reload_program_config, ) initial_program_setup() important_file = open('spam.data', 'w') interesting_file = open('eggs.data', 'w') this_daemon = daemon.Daemon() this_daemon.files_preserve = [important_file, interesting_file] this_daemon.working_directory = '/var/lib/foo' this_daemon.umask = 0o002 mail_gid = grp.getgrnam('mail').gr_gid this_daemon.gid = mail_gid this_daemon.terminate_callback = program_cleanup this_daemon.reload_callback = reload_program_config this_daemon.reload_signals = [signal.SIGHUP, signal.SIGUSR1] this_daemon.start() do_main_program() Interface ========= A new module, `daemon`, is added to the standard library. The module defines a class, `Daemon`, used to represent the settings for a daemon process. An exception class, `DaemonError`, is defined for exceptions raised from the module. ``Daemon`` objects ================== A `Daemon` instance represents the behaviour settings for the process when it becomes a daemon. The behaviour is customised by setting attributes on the instance, before calling the `start` method. The following attributes are defined. `files_preserve` :Default: ``None`` List of files that should *not* be closed when starting the daemon. If ``None``, all open file descriptors will be closed. Elements of the list are file descriptors (as returned by a file object's `fileno()` method) or Python `file` objects. Each specifies a file that is not to be closed during daemon start. `chroot_directory` :Default: ``None`` Full path to a directory to set as the effective root directory of the process. If ``None``, specifies that the root directory is not to be changed. `working_directory` :Default: ``'/'`` Full path of the working directory to which the process should change on daemon start. Since a filesystem cannot be unmounted if a process has its current working directory on that filesystem, this should either be left at default or set to a directory that is a sensible “home directory” for the daemon while it is running. `lockfile_directory` :Default: ``'/var/run'`` Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped. `lockfile_name` :Default: ``None`` Base name of the lockfile for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line. `umask` :Default: ``0`` File access creation mask (“umask”) to set for the process on daemon start. Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects. `ignore_signals` :Default: ``[signal.SIGTTOU, signal.SIGTTIN, signal.SIGTSTP]`` List of signals that the process should ignore (by setting the signal action to ``signal.SIG_IGN``) on daemon start. `terminate_signals` :Default: ``[signal.SIGTERM]`` List of signals that the process should interpret as a request to terminate cleanly. `terminate_callback` :Default: ``None`` Callable to invoke when the process receives any of the `terminate_signals` signals, before then terminating the process. `reload_signals` :Default: ``[signal.SIGHUP]`` List of signals that the process should interpret as a request to reload runtime configuration. `reload_callback` :Default: ``None`` Callable to invoke when the process receives any of the `reload_signals` signals. `uid` :Default: ``None`` The user ID (“uid”) value to switch the process to on daemon start. `gid` :Default: ``None`` The group ID (“gid”) value to switch the process to on daemon start. `prevent_core` :Default: ``True`` If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`. `stdout` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stdout`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stdout` is not re-bound. `stderr` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stderr`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stderr` is not re-bound. The following methods are defined. `start()` :Return: ``None`` Start the daemon. This performs the following steps: * If the `chroot_directory` attribute is not ``None``: * Set the effective root directory of the process to that directory (via `os.chroot`). This allows running the daemon process inside a “chroot gaol” as a means of limiting the system's exposure to rogue behaviour by the process. * If the `lockfile_directory` attribute is not ``None``: * Look in that directory for a file named '`lockfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process. * Close all open file descriptors, excluding those listed in the `files_preserve` attribute. * Change current working directory to the path specified by the `working_directory` attribute. * Reset the file access creation mask to the value specified by the `umask` attribute. * Detach the current process into its own process group, and disassociate from any controlling terminal. This step is skipped if it is determined to be redundant: if the process was started by `init`, by `initd`, or by `inetd`. * Set signal handlers as specified by the `ignore_signals`, `terminate_signals`, `reload_signals` attributes. * If the `prevent_core` attribute is true: * Set the resource limits for the process to prevent any core dump from the process. * Set the process uid and gid to the true uid and gid of the process, to relinquish any elevated privilege. * If the `lockfile_directory` attribute is not ``None``: * Create the lockfile for this daemon in that directory, by writing a text line containing the current process ID (“pid”) to a file named '`lockfile_name`.pid'. * If either of the attributes `uid` or `gid` are not ``None``: * Set the process uid and/or gid to the specified values. * If either of the attributes `stdout` or `stderr` are not ``None``: * Bind the names `sys.stdout` and/or `sys.stderr` to the corresponding objects. `reload()` :Return: ``None`` Reload the daemon configuration. The meaning of this is entirely defined by the customisation of this daemon: if the `reload_callback` attribute is not ``None``, call that object. The return value is discarded. `stop()` :Return: ``None`` Stop the daemon. This performs the following steps: * If the `terminate_callback` attribute is not ``None``: * Call that object. The return value is discarded. * If the `lockfile_directory` attribute is not ``None``: * Delete the lockfile for this daemon. * Raise a `SystemExit` exception. ``DaemonError`` objects ======================= The `DaemonError` class inherits from `Exception`. The module implementation will raise an instance of `DaemonError` when an error occurs in processing daemon behaviour. ========== Motivation ========== The majority of programs written to be Unix daemons either implement behaviour very similar to that in the `Specification`_, or are poorly-behaved daemons by the `Correct daemon behaviour`_. Since these steps should be much the same in most implementations but are very particular and easy to omit or implement incorrectly, they are a prime target for a standard well-tested implementation in the standard library. ========= Rationale ========= Correct daemon behaviour ======================== According to Stevens in [stevens]_ §2.6, a program should perform the following steps to become a Unix daemon process. * Close all open file descriptors. * Change current working directory. * Reset the file access creation mask. * Run in the background. * Disassociate from process group. * Ignore terminal I/O signals. * Disassociate from control terminal. * Don't reacquire a control terminal. * Correctly handle the following circumstances: * Started by System V `init` process. * Daemon termination by ``SIGTERM`` signal. * Children generate ``SIGCLD`` signal. The `daemon` tool [slack-daemon]_ lists (in its summary of features) behaviour that should be performed when turning a program into a well-behaved Unix daemon process. It differs from this PEP's intent in that it invokes a *separate* program as a daemon process. The following features are appropriate for a daemon that starts itself once the program is already running: * Sets up the correct process context for a daemon. * Behaves sensibly when started by `initd(8)` or `inetd(8)`. * Revokes any suid or sgid privileges to reduce security risks in case daemon is incorrectly installed with special privileges. * Prevents the generation of core files to prevent leaking sensitive information from daemons run as root (optional). * Names the daemon by creating and locking a PID file to guarantee that only one daemon with the given name can execute at any given time (optional). * Sets the user and group under which to run the daemon (optional, root only). * Creates a chroot gaol (optional, root only). * Captures the daemon's stdout and stderr and directs them to syslog (optional). ======================== Reference Implementation ======================== The `python-daemon` package [python-daemon]_. As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP. Other daemon implementations ============================ Prior to this PEP, several existing third-party Python libraries or tools implemented some of this PEP's `correct daemon behaviour`_. The `reference implementation`_ is a fairly direct successor from the following implementations: * Many good ideas were contributed by the community to Python cookbook recipe 66012 [cookbook-66012]_. * The `bda.daemon` library [bda.daemon]_ is an implementation (by Robert Niederreiter et al) of [cookbook-66012]_. It is the predecessor of [python-daemon]_. Other Python daemon implementations that differ from this PEP: * The `zdaemon` tool [zdaemon]_ was written for the Zope project. Like [slack-daemon]_, it differs from this specification because it is used to run another program as a daemon process. * The Python library `daemon` [clapper-daemon]_ is (according to its homepage) no longer maintained. As of version 1.0.1, it implements the basic steps from [stevens]_. * The `daemonize` library [seutter-daemonize]_ also implements the basic steps from [stevens]_. * Twisted [twisted]_ includes, perhaps unsurprisingly, an implementation of a process daemonisation API that is integrated with the rest of the Twisted framework; it differs significantly from the API in this PEP. * The Python `initd` library [dagitses-initd]_, which uses [clapper-daemon]_, implements an equivalent of Unix `initd(8)` for controlling a daemon process. ========== References ========== .. [stevens] `Unix Network Programming`, W. Richard Stevens, 1994 Prentice Hall. .. [slack-daemon] The (non-Python) “libslack” implementation of a `daemon` tool `<http://www.libslack.org/daemon/>`_ by “raf” <raf@raf.org>. .. [python-daemon] The `python-daemon` library `<http://pypi.python.org/pypi/python-daemon/>`_ by Ben Finney et al. .. [cookbook-66012] Python Cookbook recipe 66012, “Fork a daemon process on Unix” `<http://code.activestate.com/recipes/66012/>`_. .. [bda.daemon] The `bda.daemon` library `<http://pypi.python.org/pypi/bda.daemon/>`_ by Robert Niederreiter et al. .. [zdaemon] The `zdaemon` tool `<http://pypi.python.org/pypi/zdaemon/>`_ by Guido van Rossum et al. .. [clapper-daemon] The `daemon` library `<http://pypi.python.org/pypi/daemon/>`_ by Brian Clapper. .. [seutter-daemonize] The `daemonize` library `<http://daemonize.sourceforge.net/>`_ by Jerry Seutter. .. [twisted] The `Twisted` application framework `<http://pypi.python.org/pypi/Twisted/>`_ by Glyph Lefkowitz et al. .. [dagitses-initd] The Python `initd` library `<http://pypi.python.org/pypi/initd/>`_ by Michael Andreas Dagitses. ========= Copyright ========= This work is hereby placed in the public domain. To the extent that placing a work in the public domain is not legally possible, the copyright holder hereby grants to all recipients of this work all rights and freedoms that would otherwise be restricted by copyright. -- \ “I went to a garage sale. ‘How much for the garage?’ ‘It's not | `\ for sale.’” —Steven Wright | _o__) | Ben Finney
Significant changes in this version: allow all configuration options to be set either via Daemon constructor arguments or attribute assignment. Thanks to Mike Meyer. :PEP: XXX :Title: Standard daemon process library :Version: 0.3 :Last-Modified: 2009-01-27 19:13 :Author: Ben Finney <ben+python@benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 2009-01-26 :Python-Version: 3.1 :Post-History: ======== Abstract ======== Writing a program to become a well-behaved Unix daemon is somewhat complex and tricky to get right, yet the steps are largely similar for any daemon regardless of what else the program may need to do. This PEP introduces a module to the Python standard library that provides a simple interface to the task of becoming a daemon process. .. contents:: .. Table of Contents: Abstract Specification Example usage Interface ``Daemon`` objects ``DaemonError`` objects Motivation Rationale Correct daemon behaviour Reference Implementation Other daemon implementations References Copyright ============= Specification ============= Example usage ============= Simple example of usage:: import daemon from spam import do_main_program this_daemon = daemon.Daemon() this_daemon.start() do_main_program() More complex example usage:: import os import grp import signal import daemon from spam import ( initial_program_setup, do_main_program, program_cleanup, reload_program_config, ) initial_program_setup() important_file = open('spam.data', 'w') interesting_file = open('eggs.data', 'w') this_daemon = daemon.Daemon( working_directory='/var/lib/foo', umask=0o002, terminate_callback=program_cleanup, reload_callback=reload_program_config, reload_signals=[signal.SIGHUP, signal.SIGUSR1], ) this_daemon.files_preserve = [important_file, interesting_file] mail_gid = grp.getgrnam('mail').gr_gid this_daemon.gid = mail_gid this_daemon.start() do_main_program() Interface ========= A new module, `daemon`, is added to the standard library. The module defines a class, `Daemon`, used to represent the settings and controls for a daemon process. An exception class, `DaemonError`, is defined for exceptions raised from the module. ``Daemon`` objects ================== A `Daemon` instance represents the behaviour settings for the process when it becomes a daemon. The behaviour is customised by setting options on the instance, before calling the `start` method. Each option can be passed as a keyword argument to the `Daemon` constructor, or subsequently altered by assigning to an attribute on the instance at any time prior to calling `start`. That is, for an option named `wibble`, the following invocation:: foo = daemon.Daemon(wibble=bar) foo.start() is equivalent to:: foo = daemon.Daemon() foo.wibble = bar foo.start() The following options are defined. `files_preserve` :Default: ``None`` List of files that should *not* be closed when starting the daemon. If ``None``, all open file descriptors will be closed. Elements of the list are file descriptors (as returned by a file object's `fileno()` method) or Python `file` objects. Each specifies a file that is not to be closed during daemon start. `chroot_directory` :Default: ``None`` Full path to a directory to set as the effective root directory of the process. If ``None``, specifies that the root directory is not to be changed. `working_directory` :Default: ``'/'`` Full path of the working directory to which the process should change on daemon start. Since a filesystem cannot be unmounted if a process has its current working directory on that filesystem, this should either be left at default or set to a directory that is a sensible “home directory” for the daemon while it is running. `lockfile_directory` :Default: ``'/var/run'`` Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped. `lockfile_name` :Default: ``None`` Base name of the lockfile for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line. `umask` :Default: ``0`` File access creation mask (“umask”) to set for the process on daemon start. Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects. `ignore_signals` :Default: ``[signal.SIGTTOU, signal.SIGTTIN, signal.SIGTSTP]`` List of signals that the process should ignore (by setting the signal action to ``signal.SIG_IGN``) on daemon start. `terminate_signals` :Default: ``[signal.SIGTERM]`` List of signals that the process should interpret as a request to terminate cleanly. `terminate_callback` :Default: ``None`` Callable to invoke when the process receives any of the `terminate_signals` signals, before then terminating the process. `reload_signals` :Default: ``[signal.SIGHUP]`` List of signals that the process should interpret as a request to reload runtime configuration. `reload_callback` :Default: ``None`` Callable to invoke when the process receives any of the `reload_signals` signals. `uid` :Default: ``None`` The user ID (“uid”) value to switch the process to on daemon start. `gid` :Default: ``None`` The group ID (“gid”) value to switch the process to on daemon start. `prevent_core` :Default: ``True`` If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`. `stdout` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stdout`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stdout` is not re-bound. `stderr` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stderr`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stderr` is not re-bound. The following methods are defined. `start()` :Return: ``None`` Start the daemon. This performs the following steps: * If the `chroot_directory` attribute is not ``None``: * Set the effective root directory of the process to that directory (via `os.chroot`). This allows running the daemon process inside a “chroot gaol” as a means of limiting the system's exposure to rogue behaviour by the process. * If the `lockfile_directory` attribute is not ``None``: * Look in that directory for a file named '`lockfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process. * Close all open file descriptors, excluding those listed in the `files_preserve` attribute. * Change current working directory to the path specified by the `working_directory` attribute. * Reset the file access creation mask to the value specified by the `umask` attribute. * Detach the current process into its own process group, and disassociate from any controlling terminal. This step is skipped if it is determined to be redundant: if the process was started by `init`, by `initd`, or by `inetd`. * Set signal handlers as specified by the `ignore_signals`, `terminate_signals`, `reload_signals` attributes. * If the `prevent_core` attribute is true: * Set the resource limits for the process to prevent any core dump from the process. * Set the process uid and gid to the true uid and gid of the process, to relinquish any elevated privilege. * If the `lockfile_directory` attribute is not ``None``: * Create the lockfile for this daemon in that directory, by writing a text line containing the current process ID (“pid”) to a file named '`lockfile_name`.pid'. * If either of the attributes `uid` or `gid` are not ``None``: * Set the process uid and/or gid to the specified values. * If either of the attributes `stdout` or `stderr` are not ``None``: * Bind the names `sys.stdout` and/or `sys.stderr` to the corresponding objects. `reload()` :Return: ``None`` Reload the daemon configuration. The meaning of this is entirely defined by the customisation of this daemon: if the `reload_callback` attribute is not ``None``, call that object. The return value is discarded. `stop()` :Return: ``None`` Stop the daemon. This performs the following steps: * If the `terminate_callback` attribute is not ``None``: * Call that object. The return value is discarded. * If the `lockfile_directory` attribute is not ``None``: * Delete the lockfile for this daemon. * Raise a `SystemExit` exception. ``DaemonError`` objects ======================= The `DaemonError` class inherits from `Exception`. The module implementation will raise an instance of `DaemonError` when an error occurs in processing daemon behaviour. ========== Motivation ========== The majority of programs written to be Unix daemons either implement behaviour very similar to that in the `specification`_, or are poorly-behaved daemons by the `correct daemon behaviour`_. Since these steps should be much the same in most implementations but are very particular and easy to omit or implement incorrectly, they are a prime target for a standard well-tested implementation in the standard library. ========= Rationale ========= Correct daemon behaviour ======================== According to Stevens in [stevens]_ §2.6, a program should perform the following steps to become a Unix daemon process. * Close all open file descriptors. * Change current working directory. * Reset the file access creation mask. * Run in the background. * Disassociate from process group. * Ignore terminal I/O signals. * Disassociate from control terminal. * Don't reacquire a control terminal. * Correctly handle the following circumstances: * Started by System V `init` process. * Daemon termination by ``SIGTERM`` signal. * Children generate ``SIGCLD`` signal. The `daemon` tool [slack-daemon]_ lists (in its summary of features) behaviour that should be performed when turning a program into a well-behaved Unix daemon process. It differs from this PEP's intent in that it invokes a *separate* program as a daemon process. The following features are appropriate for a daemon that starts itself once the program is already running: * Sets up the correct process context for a daemon. * Behaves sensibly when started by `initd(8)` or `inetd(8)`. * Revokes any suid or sgid privileges to reduce security risks in case daemon is incorrectly installed with special privileges. * Prevents the generation of core files to prevent leaking sensitive information from daemons run as root (optional). * Names the daemon by creating and locking a PID file to guarantee that only one daemon with the given name can execute at any given time (optional). * Sets the user and group under which to run the daemon (optional, root only). * Creates a chroot gaol (optional, root only). * Captures the daemon's stdout and stderr and directs them to syslog (optional). ======================== Reference Implementation ======================== The `python-daemon` package [python-daemon]_. As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP. Other daemon implementations ============================ Prior to this PEP, several existing third-party Python libraries or tools implemented some of this PEP's `correct daemon behaviour`_. The `reference implementation`_ is a fairly direct successor from the following implementations: * Many good ideas were contributed by the community to Python cookbook recipe 66012 [cookbook-66012]_. * The `bda.daemon` library [bda.daemon]_ is an implementation of [cookbook-66012]_. It is the predecessor of [python-daemon]_. Other Python daemon implementations that differ from this PEP: * The `zdaemon` tool [zdaemon]_ was written for the Zope project. Like [slack-daemon]_, it differs from this specification because it is used to run another program as a daemon process. * The Python library `daemon` [clapper-daemon]_ is (according to its homepage) no longer maintained. As of version 1.0.1, it implements the basic steps from [stevens]_. * The `daemonize` library [seutter-daemonize]_ also implements the basic steps from [stevens]_. * Twisted [twisted]_ includes, perhaps unsurprisingly, an implementation of a process daemonisation API that is integrated with the rest of the Twisted framework; it differs significantly from the API in this PEP. * The Python `initd` library [dagitses-initd]_, which uses [clapper-daemon]_, implements an equivalent of Unix `initd(8)` for controlling a daemon process. ========== References ========== .. [stevens] `Unix Network Programming`, W. Richard Stevens, 1994 Prentice Hall. .. [slack-daemon] The (non-Python) “libslack” implementation of a `daemon` tool `<http://www.libslack.org/daemon/>`_ by “raf” <raf@raf.org>. .. [python-daemon] The `python-daemon` library `<http://pypi.python.org/pypi/python-daemon/>`_ by Ben Finney et al. .. [cookbook-66012] Python Cookbook recipe 66012, “Fork a daemon process on Unix” `<http://code.activestate.com/recipes/66012/>`_. .. [bda.daemon] The `bda.daemon` library `<http://pypi.python.org/pypi/bda.daemon/>`_ by Robert Niederreiter et al. .. [zdaemon] The `zdaemon` tool `<http://pypi.python.org/pypi/zdaemon/>`_ by Guido van Rossum et al. .. [clapper-daemon] The `daemon` library `<http://pypi.python.org/pypi/daemon/>`_ by Brian Clapper. .. [seutter-daemonize] The `daemonize` library `<http://daemonize.sourceforge.net/>`_ by Jerry Seutter. .. [twisted] The `Twisted` application framework `<http://pypi.python.org/pypi/Twisted/>`_ by Glyph Lefkowitz et al. .. [dagitses-initd] The Python `initd` library `<http://pypi.python.org/pypi/initd/>`_ by Michael Andreas Dagitses. ========= Copyright ========= This work is hereby placed in the public domain. To the extent that placing a work in the public domain is not legally possible, the copyright holder hereby grants to all recipients of this work all rights and freedoms that would otherwise be restricted by copyright. -- \ “Program testing can be a very effective way to show the | `\ presence of bugs, but is hopelessly inadequate for showing | _o__) their absence.” —Edsger W. Dijkstra | Ben Finney
Significant changes in this version: New section distinguishing a Unix daemon from a “service”; clarify that the API makes the *current* program into a daemon process; consistently discuss PID files (instead of lock files). :PEP: XXX :Title: Standard daemon process library :Version: 0.4 :Last-Modified: 2009-01-29 16:32 :Author: Ben Finney <ben+python@benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 2009-01-26 :Python-Version: 3.1 :Post-History: ======== Abstract ======== Writing a program to become a well-behaved Unix daemon is somewhat complex and tricky to get right, yet the steps are largely similar for any daemon regardless of what else the program may need to do. This PEP introduces a module to the Python standard library that provides a simple interface to the task of becoming a daemon process. .. contents:: .. Table of Contents: Abstract Specification Example usage Interface ``Daemon`` objects ``DaemonError`` objects Motivation Rationale Correct daemon behaviour A daemon is not a service Reference Implementation Other daemon implementations References Copyright ============= Specification ============= Example usage ============= Simple example of usage:: import daemon from spam import do_main_program this_daemon = daemon.Daemon() this_daemon.start() do_main_program() More complex example usage:: import os import grp import signal import daemon from spam import ( initial_program_setup, do_main_program, program_cleanup, reload_program_config, ) initial_program_setup() important_file = open('spam.data', 'w') interesting_file = open('eggs.data', 'w') this_daemon = daemon.Daemon( working_directory='/var/lib/foo', umask=0o002, terminate_callback=program_cleanup, reload_callback=reload_program_config, reload_signals=[signal.SIGHUP, signal.SIGUSR1], ) this_daemon.files_preserve = [important_file, interesting_file] mail_gid = grp.getgrnam('mail').gr_gid this_daemon.gid = mail_gid this_daemon.start() do_main_program() Interface ========= A new module, `daemon`, is added to the standard library. The module defines a class, `Daemon`, used to represent the settings and controls for a daemon process. An exception class, `DaemonError`, is defined for exceptions raised from the module. ``Daemon`` objects ================== A `Daemon` instance represents the behaviour settings for the process when it becomes a daemon. The behaviour is customised by setting options on the instance, before calling the `start` method. Each option can be passed as a keyword argument to the `Daemon` constructor, or subsequently altered by assigning to an attribute on the instance at any time prior to calling `start`. That is, for an option named `wibble`, the following invocation:: foo = daemon.Daemon(wibble=bar) foo.start() is equivalent to:: foo = daemon.Daemon() foo.wibble = bar foo.start() The following options are defined. `files_preserve` :Default: ``None`` List of files that should *not* be closed when starting the daemon. If ``None``, all open file descriptors will be closed. Elements of the list are file descriptors (as returned by a file object's `fileno()` method) or Python `file` objects. Each specifies a file that is not to be closed during daemon start. `chroot_directory` :Default: ``None`` Full path to a directory to set as the effective root directory of the process. If ``None``, specifies that the root directory is not to be changed. `working_directory` :Default: ``'/'`` Full path of the working directory to which the process should change on daemon start. Since a filesystem cannot be unmounted if a process has its current working directory on that filesystem, this should either be left at default or set to a directory that is a sensible “home directory” for the daemon while it is running. `pidfile_directory` :Default: ``'/var/run'`` Absolute directory path to contain the daemon's PID file. If ``None``, the PID file behaviour for this daemon is skipped. `pidfile_name` :Default: ``None`` Base name of the PID file for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line. `umask` :Default: ``0`` File access creation mask (“umask”) to set for the process on daemon start. Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects. `ignore_signals` :Default: ``[signal.SIGTTOU, signal.SIGTTIN, signal.SIGTSTP]`` List of signals that the process should ignore (by setting the signal action to ``signal.SIG_IGN``) on daemon start. `terminate_signals` :Default: ``[signal.SIGTERM]`` List of signals that the process should interpret as a request to terminate cleanly. `terminate_callback` :Default: ``None`` Callable to invoke when the process receives any of the `terminate_signals` signals, before then terminating the process. `reload_signals` :Default: ``[signal.SIGHUP]`` List of signals that the process should interpret as a request to reload runtime configuration. `reload_callback` :Default: ``None`` Callable to invoke when the process receives any of the `reload_signals` signals. `uid` :Default: ``None`` The user ID (“uid”) value to switch the process to on daemon start. `gid` :Default: ``None`` The group ID (“gid”) value to switch the process to on daemon start. `prevent_core` :Default: ``True`` If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`. `stdout` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stdout`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stdout` is not re-bound. `stderr` :Default: ``None`` File-like object, open for writing (in append mode, 'w+'), that will be used as the new value of `sys.stderr`. If it represents an actual file, it should be listed in `files_preserve` to prevent it being closed during daemon start. If ``None``, then `sys.stderr` is not re-bound. The following methods are defined. `start()` :Return: ``None`` Start the daemon, turning the current program into a daemon process. This performs the following steps: * If the `chroot_directory` attribute is not ``None``: * Set the effective root directory of the process to that directory (via `os.chroot`). This allows running the daemon process inside a “chroot gaol” as a means of limiting the system's exposure to rogue behaviour by the process. * If the `pidfile_directory` attribute is not ``None``: * Look in that directory for a file named '`pidfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process. * Close all open file descriptors, excluding those listed in the `files_preserve` attribute. * Change current working directory to the path specified by the `working_directory` attribute. * Reset the file access creation mask to the value specified by the `umask` attribute. * Detach the current process into its own process group, and disassociate from any controlling terminal. This step is skipped if it is determined to be redundant: if the process was started by `init`, by `initd`, or by `inetd`. * Set signal handlers as specified by the `ignore_signals`, `terminate_signals`, `reload_signals` attributes. * If the `prevent_core` attribute is true: * Set the resource limits for the process to prevent any core dump from the process. * Set the process uid and gid to the true uid and gid of the process, to relinquish any elevated privilege. * If the `pidfile_directory` attribute is not ``None``: * Create the PID file for this daemon in that directory, by writing a text line containing the current process ID (“PID”) to a file named '`pidfile_name`.pid'. * If either of the attributes `uid` or `gid` are not ``None``: * Set the process uid and/or gid to the specified values. * If either of the attributes `stdout` or `stderr` are not ``None``: * Bind the names `sys.stdout` and/or `sys.stderr` to the corresponding objects. When the function returns, the running program is a daemon process. `reload()` :Return: ``None`` Reload the daemon configuration. The meaning of this is entirely defined by the customisation of this daemon: if the `reload_callback` attribute is not ``None``, call that object. The return value is discarded. `stop()` :Return: ``None`` Stop the daemon, ending the program. This performs the following steps: * If the `terminate_callback` attribute is not ``None``: * Call that object. The return value is discarded. * If the `pidfile_directory` attribute is not ``None``: * Delete the PID file for this daemon. * Raise a `SystemExit` exception. ``DaemonError`` objects ======================= The `DaemonError` class inherits from `Exception`. The module implementation will raise an instance of `DaemonError` when an error occurs in processing daemon behaviour. ========== Motivation ========== The majority of programs written to be Unix daemons either implement behaviour very similar to that in the `specification`_, or are poorly-behaved daemons by the `correct daemon behaviour`_. Since these steps should be much the same in most implementations but are very particular and easy to omit or implement incorrectly, they are a prime target for a standard well-tested implementation in the standard library. ========= Rationale ========= Correct daemon behaviour ======================== According to Stevens in [stevens]_ §2.6, a program should perform the following steps to become a Unix daemon process. * Close all open file descriptors. * Change current working directory. * Reset the file access creation mask. * Run in the background. * Disassociate from process group. * Ignore terminal I/O signals. * Disassociate from control terminal. * Don't reacquire a control terminal. * Correctly handle the following circumstances: * Started by System V `init` process. * Daemon termination by ``SIGTERM`` signal. * Children generate ``SIGCLD`` signal. The `daemon` tool [slack-daemon]_ lists (in its summary of features) behaviour that should be performed when turning a program into a well-behaved Unix daemon process. It differs from this PEP's intent in that it invokes a *separate* program as a daemon process. The following features are appropriate for a daemon that starts itself once the program is already running: * Sets up the correct process context for a daemon. * Behaves sensibly when started by `initd(8)` or `inetd(8)`. * Revokes any suid or sgid privileges to reduce security risks in case daemon is incorrectly installed with special privileges. * Prevents the generation of core files to prevent leaking sensitive information from daemons run as root (optional). * Names the daemon by creating and locking a PID file to guarantee that only one daemon with the given name can execute at any given time (optional). * Sets the user and group under which to run the daemon (optional, root only). * Creates a chroot gaol (optional, root only). * Captures the daemon's stdout and stderr and directs them to syslog (optional). A daemon is not a service ========================= This PEP addresses only Unix-style daemons, for which the above correct behaviour is relevant, as opposed to comparable behaviours on other operating systems. There is a related concept in many systems, called a “service”. A service differs from the model in this PEP, in that rather than having the *current* program continue to run as a daemon process, a service starts an *additional* process to run in the background, and the current process communicates with that additional process via some defined channels. The Unix-style daemon model in this PEP can be used, among other things, to implement the background-process part of a service; but this PEP does not address the other aspects of setting up and managing a service. It is the opinion of this PEP's author that combining the two disparate systems of behaviour under a single API would be contrary to the focus of this PEP. ======================== Reference Implementation ======================== The `python-daemon` package [python-daemon]_. As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP. Other daemon implementations ============================ Prior to this PEP, several existing third-party Python libraries or tools implemented some of this PEP's `correct daemon behaviour`_. The `reference implementation`_ is a fairly direct successor from the following implementations: * Many good ideas were contributed by the community to Python cookbook recipe 66012 [cookbook-66012]_. * The `bda.daemon` library [bda.daemon]_ is an implementation of [cookbook-66012]_. It is the predecessor of [python-daemon]_. Other Python daemon implementations that differ from this PEP: * The `zdaemon` tool [zdaemon]_ was written for the Zope project. Like [slack-daemon]_, it differs from this specification because it is used to run another program as a daemon process. * The Python library `daemon` [clapper-daemon]_ is (according to its homepage) no longer maintained. As of version 1.0.1, it implements the basic steps from [stevens]_. * The `daemonize` library [seutter-daemonize]_ also implements the basic steps from [stevens]_. * Twisted [twisted]_ includes, perhaps unsurprisingly, an implementation of a process daemonisation API that is integrated with the rest of the Twisted framework; it differs significantly from the API in this PEP. * The Python `initd` library [dagitses-initd]_, which uses [clapper-daemon]_, implements an equivalent of Unix `initd(8)` for controlling a daemon process. ========== References ========== .. [stevens] `Unix Network Programming`, W. Richard Stevens, 1994 Prentice Hall. .. [slack-daemon] The (non-Python) “libslack” implementation of a `daemon` tool `<http://www.libslack.org/daemon/>`_ by “raf” <raf@raf.org>. .. [python-daemon] The `python-daemon` library `<http://pypi.python.org/pypi/python-daemon/>`_ by Ben Finney et al. .. [cookbook-66012] Python Cookbook recipe 66012, “Fork a daemon process on Unix” `<http://code.activestate.com/recipes/66012/>`_. .. [bda.daemon] The `bda.daemon` library `<http://pypi.python.org/pypi/bda.daemon/>`_ by Robert Niederreiter et al. .. [zdaemon] The `zdaemon` tool `<http://pypi.python.org/pypi/zdaemon/>`_ by Guido van Rossum et al. .. [clapper-daemon] The `daemon` library `<http://pypi.python.org/pypi/daemon/>`_ by Brian Clapper. .. [seutter-daemonize] The `daemonize` library `<http://daemonize.sourceforge.net/>`_ by Jerry Seutter. .. [twisted] The `Twisted` application framework `<http://pypi.python.org/pypi/Twisted/>`_ by Glyph Lefkowitz et al. .. [dagitses-initd] The Python `initd` library `<http://pypi.python.org/pypi/initd/>`_ by Michael Andreas Dagitses. ========= Copyright ========= This work is hereby placed in the public domain. To the extent that placing a work in the public domain is not legally possible, the copyright holder hereby grants to all recipients of this work all rights and freedoms that would otherwise be restricted by copyright. .. Local variables: mode: rst coding: utf-8 time-stamp-start: "^:Last-Modified:[ ]+" time-stamp-end: "$" time-stamp-line-limit: 20 time-stamp-format: "%:y-%02m-%02d %02H:%02M" End: vim: filetype=rst fileencoding=utf-8 :
On Thu, Jan 29, 2009 at 05:03:05PM +1100, Ben Finney wrote:
There is a related concept in many systems, called a “service”. A service differs from the model in this PEP, in that rather than having the *current* program continue to run as a daemon process, a service starts an *additional* process to run in the background, and the current process communicates with that additional process via some defined channels.
In my opinion, the difference between a daemon and a service is just a by-product of the difference between fork and CreateProcess. If the multiprocessing package can solve this problem, I don't see why a daemon/service package wouldn't be able to, too.
As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP.
I'm still confused why there's a PEP before there's a stable implementation and a base of users. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
On Thu, Jan 29, 2009 at 9:56 AM, Andrew McNabb <amcnabb@mcnabbs.org> wrote:
As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP.
I'm still confused why there's a PEP before there's a stable implementation and a base of users.
Let alone the *need* to be included in the stdlib, when Pypi seems perfectly appropriate for such package, especially if it's unix-only. George
George Sakkis writes:
Let alone the *need* to be included in the stdlib, when Pypi seems perfectly appropriate for such package, especially if it's unix-only.
I addressed this in Message-ID: <87mydawt1o.fsf@benfinney.id.au> yesterday (a reply to Bill Janssen). -- \ “My doctor told me to stop having intimate dinners for four. | `\ Unless there are three other people.” —Orson Welles | _o__) | Ben Finney
Andrew McNabb <amcnabb@mcnabbs.org> writes:
I'm still confused why there's a PEP before there's a stable implementation and a base of users.
Because that's exactly what PEP 1 explicitly advises. Am I wrong to be following PEP 1? -- \ “We should strive to do things in [Gandhi's] spirit… not to use | `\ violence in fighting for our cause, but by non-participation in | _o__) what we believe is evil.” —Albert Einstein | Ben Finney
Andrew McNabb <amcnabb@mcnabbs.org> writes:
In my opinion, the difference between a daemon and a service is just a by-product of the difference between fork and CreateProcess.
This PEP addresses the case where a daemon is specifically what the programmer wants. I would like it to *also* be useful as the obvious thing to use for implementing a service on Unix, but it is designed to also allow for the case when a service is *not* what the programmer wants.
If the multiprocessing package can solve this problem, I don't see why a daemon/service package wouldn't be able to, too.
A service package would be ideal for the use case you suggest. That's not what this PEP proposes, though. -- \ “Why, I'd horse-whip you if I had a horse.” —Groucho Marx | `\ | _o__) | Ben Finney
Hello,
`files_preserve` :Default: ``None``
List of files that should *not* be closed when starting the daemon.
Why does it have to be explicit? If I have an open file lying around, it should be obvious that I don't want it closed. Otherwise I'd have closed it myself... The exception is of course the three stdio streams, which should be treated separately.
`umask` :Default: ``0``
File access creation mask (“umask”) to set for the process on daemon start.
Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects.
Why this behaviour? It's easy enough to call os.umask() manually if you want it, but there are certainly situations where you don't want to change the umask (situations where the daemon is meant to act on behalf of the user who started it). Or, perhaps, umask could be made mand (regardless of which, 0 isn't a sensible umask default!)
* If the `pidfile_directory` attribute is not ``None``:
* Look in that directory for a file named '`pidfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process.
It should first check that there does exist a processus with that number, so that stale pid files don't cause too many problems. Regards Antoine.
Antoine Pitrou <solipsis@pitrou.net> writes:
Hello,
`files_preserve` :Default: ``None``
List of files that should *not* be closed when starting the daemon.
Why does it have to be explicit? If I have an open file lying around, it should be obvious that I don't want it closed. Otherwise I'd have closed it myself...
Is it so obvious? In Python programs, I far more often see files left open simply because the programmer expects them to be cleaned up at the appropriate time. By the references cited in this PEP, the time of starting the daemon is such an appropriate time to close all open file descriptors.
`umask` :Default: ``0``
File access creation mask (“umask”) to set for the process on daemon start.
Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects.
Why this behaviour?
Because the umask is inherited from the environment of the parent process of the current program, which is good when the current program is conceptually an extension of the parent program's environment, but not in the case where one is intentionally disassociating from that environment.
It's easy enough to call os.umask() manually if you want it, but there are certainly situations where you don't want to change the umask (situations where the daemon is meant to act on behalf of the user who started it).
Those situations exist, but I would argue they are not the common case.
Or, perhaps, umask could be made [mandatory when the options ‘uid’ or ‘gid’ are set.]
That diverges from the referenced “correct behaviour”, in particular the Stevens reference. I think if this PEP is worth implementing, it's for exactly the purpose of providing a Python implementation of defined correct daemon behaviour; that definition includes re-setting the umask.
(regardless of which, 0 isn't a sensible umask default!)
The default umask ‘0’ is chosen in the references as the umask that allows file access modes for files created by the daemon to work as expected. Quoting Stevens: This prevents any files created by the daemon from having their access bits modified. For example, if a daemon specifically created a file with a mode of 0660, so that only the user and group could read and write the file, but the ``umask`` value was 060, the group read and write permissions would be turned off. If the daemon required the group permissions to be on, so that some other process in that group could access the file, this ``umask`` value prevents it. -- \ “I don't accept the currently fashionable assertion that any | `\ view is automatically as worthy of respect as any equal and | _o__) opposite view.” —Douglas Adams | Ben Finney
Ben Finney <ben+python@...> writes:
Is it so obvious? In Python programs, I far more often see files left open simply because the programmer expects them to be cleaned up at the appropriate time.
What if I use a library which keeps its own files open in the background? Do I have to monkeypatch the library so as to get at the file descriptors and pass them by hand?
It's easy enough to call os.umask() manually if you want it, but there are certainly situations where you don't want to change the umask (situations where the daemon is meant to act on behalf of the user who started it).
Those situations exist, but I would argue they are not the common case.
Well, perhaps, but it's easier to leave the umask as is rather than ask the programmer to query the current umask and then pass it as an argument (not to mention that the current os.umask() function does not support querying without modifying...).
That diverges from the referenced “correct behaviour”, in particular the Stevens reference. I think if this PEP is worth implementing, it's for exactly the purpose of providing a Python implementation of defined correct daemon behaviour; that definition includes re-setting the umask.
Well, I think the PEP should implement a behaviour which is at the same time useful and sane. Whether it scrupulously conforms to something which isn't an official standard shouldn't be considered important.
The default umask ‘0’ is chosen in the references as the umask that allows file access modes for files created by the daemon to work as expected.
And then, if I create a file using the standard Python idiom (the open() function), it will be readable and writable by anyone. It isn't sane at all. Regards, Antoine.
Antoine Pitrou <solipsis@pitrou.net> writes:
Hello, […]
* If the `pidfile_directory` attribute is not ``None``:
* Look in that directory for a file named '`pidfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process.
It should first check that there does exist a processus with that number, so that stale pid files don't cause too many problems.
Good point, thanks. What would be appropriate behaviour in the case of a stale PID file? Abort the daemonisation attempt? Delete the stale lock file silently and continue as though it didn't exist? -- \ “I bought some powdered water, but I don't know what to add.” | `\ —Steven Wright | _o__) | Ben Finney
Ben Finney <ben+python@...> writes:
What would be appropriate behaviour in the case of a stale PID file? Abort the daemonisation attempt? Delete the stale lock file silently and continue as though it didn't exist?
Delete the stale lock file silently and continue as though it didn't exist. (and, of course, create another one for the current process)
[responding very late] On Thu, Jan 29, 2009, Antoine Pitrou wrote:
Ben Finney <ben+python@...> writes:
What would be appropriate behaviour in the case of a stale PID file? Abort the daemonisation attempt? Delete the stale lock file silently and continue as though it didn't exist?
Delete the stale lock file silently and continue as though it didn't exist. (and, of course, create another one for the current process)
This should be determined by the API -- for example, a process running against an NFS disk that may be run from any of several servers but should only run on one server at any time ought to abort if there's a stale PID file. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons." --Aahz
Aahz <aahz@pythoncraft.com> writes:
[responding very late]
On Thu, Jan 29, 2009, Antoine Pitrou wrote:
Delete the stale lock file silently and continue as though it didn't exist. (and, of course, create another one for the current process)
This should be determined by the API -- for example, a process running against an NFS disk that may be run from any of several servers but should only run on one server at any time ought to abort if there's a stale PID file.
This and several other issues make PID file handling quite a thorny subject, and I'm working with Skip Montanaro on an implementation separated from the daemon PEP 3143. The current PEP 3143 delegates all these decisions (by not mentioning them at all) to the context manager ‘__entry__’ and ‘__exit__’ of an optional ‘pidfile’ parameter. -- \ “There's no excuse to be bored. Sad, yes. Angry, yes. | `\ Depressed, yes. Crazy, yes. But there's no excuse for boredom, | _o__) ever.” —Viggo Mortensen | Ben Finney
En Thu, 29 Jan 2009 04:03:05 -0200, Ben Finney <ben+python-TqlCGjI+HWGnbCmf7pGUHw@public.gmane.org> escribió:
:PEP: XXX :Title: Standard daemon process library :Version: 0.4
After reading this PEP, I see it as a big list of implementation details. A more high level view would be more suitable for a multiplatform implementation. Like when using most library modules, I trust the authors, I hope they've done their job well, that they've read their Stevens and Pietrek, and the code follows the best practices. For the most part, I don't care about the implementation - just that it don't gets in my way. I'd say that a daemon is just a background process, detached from any terminal and without any user interactivity, that keeps running independently of the logged user. *How* to get there, it's the library job. So I need a method to "become a daemon". Stopping a daemon process must be done orderly so a "stop" method is required too. It needs some way to react to external requests (signals, or control requests on Windows): they're a short list, we can use methods following a naming convention (e.g. "on_reload"). And that's all, at least conceptually; probably we need some attributes to customize the behaviour. I think it's a lot easier to implement such abstract view on different systems. Excluding Windows right from the start --because of so many details that should be emulated, and are mostly irrelevant-- would be a bad idea. -- Gabriel Genellina
Howdy all, Significant changes in this release: * Name the daemon process context class `DaemonContext`, since it doesn't actually represent a separate daemon. (The reference implementation will also have a `DaemonRunner` class, but that's outside the scope of this PEP.) * Implement the context manager protocol, allowing use as a ‘with’ context manager or via explicit ‘open’ and ‘close’ calls. * Delegate PID file handling to a `pidfile` object handed to the `DaemonContext` instance, and used simply as a context manager. * Simplify the set of options by using a mapping for signal handlers. * Target Python 3.2, since the reference implementation will very likely not be complete in time for anything earlier. :PEP: XXX :Title: Standard daemon process library :Version: 0.5 :Last-Modified: 2009-03-12 14:50 :Author: Ben Finney <ben+python@benfinney.id.au> :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 2009-01-26 :Python-Version: 3.2 :Post-History: ======== Abstract ======== Writing a program to become a well-behaved Unix daemon is somewhat complex and tricky to get right, yet the steps are largely similar for any daemon regardless of what else the program may need to do. This PEP introduces a package to the Python standard library that provides a simple interface to the task of becoming a daemon process. .. contents:: .. Table of Contents: Abstract Specification Example usage Interface ``DaemonContext`` objects ``DaemonError`` objects Motivation Rationale Correct daemon behaviour A daemon is not a service Reference Implementation Other daemon implementations References Copyright ============= Specification ============= Example usage ============= Simple example of direct `DaemonContext` usage:: import daemon from spam import do_main_program with daemon.DaemonContext() as daemon_context: do_main_program() More complex example usage:: import os import grp import signal import daemon import lockfile from spam import ( initial_program_setup, do_main_program, program_cleanup, reload_program_config, ) context = daemon.DaemonContext( working_directory='/var/lib/foo', umask=0o002, pidfile=lockfile.FileLock('/var/run/spam.pid'), ) context.signal_map = { signal.SIGTERM: program_cleanup, signal.SIGHUP: 'close', signal.SIGUSR1: reload_program_config, } mail_gid = grp.getgrnam('mail').gr_gid context.gid = mail_gid important_file = open('spam.data', 'w') interesting_file = open('eggs.data', 'w') context.files_preserve = [important_file, interesting_file] initial_program_setup() with context: do_main_program() Interface ========= A new package, `daemon`, is added to the standard library. An exception class, `DaemonError`, is defined for exceptions raised from the package. A class, `DaemonContext`, is defined to represent the settings and process context for the program running as a daemon process. ``DaemonContext`` objects ========================= A `DaemonContext` instance represents the behaviour settings and process context for the program when it becomes a daemon. The behaviour and environment is customised by setting options on the instance, before calling the `open` method. Each option can be passed as a keyword argument to the `DaemonContext` constructor, or subsequently altered by assigning to an attribute on the instance at any time prior to calling `open`. That is, for options named `wibble` and `wubble`, the following invocation:: foo = daemon.DaemonContext(wibble=bar, wubble=baz) foo.open() is equivalent to:: foo = daemon.DaemonContext() foo.wibble = bar foo.wubble = baz foo.open() The following options are defined. `files_preserve` :Default: ``None`` List of files that should *not* be closed when starting the daemon. If ``None``, all open file descriptors will be closed. Elements of the list are file descriptors (as returned by a file object's `fileno()` method) or Python `file` objects. Each specifies a file that is not to be closed during daemon start. `chroot_directory` :Default: ``None`` Full path to a directory to set as the effective root directory of the process. If ``None``, specifies that the root directory is not to be changed. `working_directory` :Default: ``'/'`` Full path of the working directory to which the process should change on daemon start. Since a filesystem cannot be unmounted if a process has its current working directory on that filesystem, this should either be left at default or set to a directory that is a sensible “home directory” for the daemon while it is running. `umask` :Default: ``0`` File access creation mask (“umask”) to set for the process on daemon start. Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects. `pidfile` :Default: ``None`` Context manager for a PID lock file. When the daemon context opens and closes, it enters and exits the `pidfile` context manager. `signal_map` :Default: ``{signal.SIGTOU: None, signal.SIGTTIN: None, signal.SIGTSTP: None, signal.SIGTERM: 'close'}`` Mapping from operating system signals to callback actions. The mapping is used when the daemon context opens, and determines the action for each signal's signal handler: * A value of ``None`` will ignore the signal (by setting the signal action to ``signal.SIG_IGN``). * A string value will be used as the name of an attribute on the ``DaemonContext`` instance. The attribute's value will be used as the action for the signal handler. * Any other value will be used as the action for the signal handler. `uid` :Default: ``None`` The user ID (“uid”) value to switch the process to on daemon start. `gid` :Default: ``None`` The group ID (“gid”) value to switch the process to on daemon start. `prevent_core` :Default: ``True`` If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`. `stdin` :Default: ``None`` `stdout` :Default: ``None`` `stderr` :Default: ``None`` Each of `stdin`, `stdout`, and `stderr` is a file-like object which will be used as the new file for the standard I/O stream `sys.stdin`, `sys.stdout`, and `sys.stderr` respectively. The file should therefore be open, with a minimum of mode 'r' in the case of `stdin`, and mode 'w+' in the case of `stdout` and `stderr`. If the object has a `fileno()` method that returns a file descriptor, the corresponding file will be excluded from being closed during daemon start (that is, it will be treated as though it were listed in `files_preserve`). If ``None``, the corresponding system stream is re-bound to the file named by `os.devnull`. The following methods are defined. `open()` :Return: ``None`` Open the daemon context, turning the current program into a daemon process. This performs the following steps: * If the `chroot_directory` attribute is not ``None``, set the effective root directory of the process to that directory (via `os.chroot`). This allows running the daemon process inside a “chroot gaol” as a means of limiting the system's exposure to rogue behaviour by the process. * Close all open file descriptors. This excludes those listed in the `files_preserve` attribute, and those that correspond to the `stdin`, `stdout`, or `stderr` attributes. * Change current working directory to the path specified by the `working_directory` attribute. * Reset the file access creation mask to the value specified by the `umask` attribute. * Detach the current process into its own process group, and disassociate from any controlling terminal. This step is skipped if it is determined to be redundant: if the process was started by `init`, by `initd`, or by `inetd`. * Set signal handlers as specified by the `signal_map` attribute. * If the `prevent_core` attribute is true, set the resource limits for the process to prevent any core dump from the process. * Set the process uid and gid to the true uid and gid of the process, to relinquish any elevated privilege. * If the `pidfile` attribute is not ``None``, enter its context manager. * If either of the attributes `uid` or `gid` are not ``None``, set the process uid and/or gid to the specified values. * If any of the attributes `stdin`, `stdout`, `stderr` are not ``None``, bind the system streams `sys.stdin`, `sys.stdout`, and/or `sys.stderr` to the files represented by the corresponding attributes. Where the attribute has a file descriptor, the descriptor is duplicated (instead of re-binding the name). When the function returns, the running program is a daemon process. `close()` :Return: ``None`` Terminate the daemon context. This performs the following step: * If the `pidfile` attribute is not ``None``, exit its context manager. The class also implements the context manager protocol via ``__enter__`` and ``__exit__`` methods. `__enter__()` :Return: The ``DaemonContext`` instance Call the instance's `open()` method, then return the instance. `__exit__(exc_type, exc_value, exc_traceback)` :Return: ``True`` or ``False`` as defined by the context manager protocol Call the instance's `close()` method, then return ``True`` if the exception was handled or ``False`` if it was not. ``DaemonError`` objects ======================= The `DaemonError` class inherits from `Exception`. The `daemon` package implementation will raise an instance of `DaemonError` when an error occurs in processing daemon behaviour. ========== Motivation ========== The majority of programs written to be Unix daemons either implement behaviour very similar to that in the `specification`_, or are poorly-behaved daemons by the `correct daemon behaviour`_. Since these steps should be much the same in most implementations but are very particular and easy to omit or implement incorrectly, they are a prime target for a standard well-tested implementation in the standard library. ========= Rationale ========= Correct daemon behaviour ======================== According to Stevens in [stevens]_ §2.6, a program should perform the following steps to become a Unix daemon process. * Close all open file descriptors. * Change current working directory. * Reset the file access creation mask. * Run in the background. * Disassociate from process group. * Ignore terminal I/O signals. * Disassociate from control terminal. * Don't reacquire a control terminal. * Correctly handle the following circumstances: * Started by System V `init` process. * Daemon termination by ``SIGTERM`` signal. * Children generate ``SIGCLD`` signal. The `daemon` tool [slack-daemon]_ lists (in its summary of features) behaviour that should be performed when turning a program into a well-behaved Unix daemon process. It differs from this PEP's intent in that it invokes a *separate* program as a daemon process. The following features are appropriate for a daemon that starts itself once the program is already running: * Sets up the correct process context for a daemon. * Behaves sensibly when started by `initd(8)` or `inetd(8)`. * Revokes any suid or sgid privileges to reduce security risks in case daemon is incorrectly installed with special privileges. * Prevents the generation of core files to prevent leaking sensitive information from daemons run as root (optional). * Names the daemon by creating and locking a PID file to guarantee that only one daemon with the given name can execute at any given time (optional). * Sets the user and group under which to run the daemon (optional, root only). * Creates a chroot gaol (optional, root only). * Captures the daemon's stdout and stderr and directs them to syslog (optional). A daemon is not a service ========================= This PEP addresses only Unix-style daemons, for which the above correct behaviour is relevant, as opposed to comparable behaviours on other operating systems. There is a related concept in many systems, called a “service”. A service differs from the model in this PEP, in that rather than having the *current* program continue to run as a daemon process, a service starts an *additional* process to run in the background, and the current process communicates with that additional process via some defined channels. The Unix-style daemon model in this PEP can be used, among other things, to implement the background-process part of a service; but this PEP does not address the other aspects of setting up and managing a service. ======================== Reference Implementation ======================== The `python-daemon` package [python-daemon]_. As of `python-daemon` version 1.3 (2009-03-12), the package is under active development and is not yet a full implementation of this PEP. Other daemon implementations ============================ Prior to this PEP, several existing third-party Python libraries or tools implemented some of this PEP's `correct daemon behaviour`_. The `reference implementation`_ is a fairly direct successor from the following implementations: * Many good ideas were contributed by the community to Python cookbook recipes #66012 [cookbook-66012]_ and #278731 [cookbook-278731]_. * The `bda.daemon` library [bda.daemon]_ is an implementation of [cookbook-66012]_. It is the predecessor of [python-daemon]_. Other Python daemon implementations that differ from this PEP: * The `zdaemon` tool [zdaemon]_ was written for the Zope project. Like [slack-daemon]_, it differs from this specification because it is used to run another program as a daemon process. * The Python library `daemon` [clapper-daemon]_ is (according to its homepage) no longer maintained. As of version 1.0.1, it implements the basic steps from [stevens]_. * The `daemonize` library [seutter-daemonize]_ also implements the basic steps from [stevens]_. * Ray Burr's `daemon.py` module [burr-daemon]_ provides the [stevens]_ procedure as well as PID file handling and redirection of output to syslog. * Twisted [twisted]_ includes, perhaps unsurprisingly, an implementation of a process daemonisation API that is integrated with the rest of the Twisted framework; it differs significantly from the API in this PEP. * The Python `initd` library [dagitses-initd]_, which uses [clapper-daemon]_, implements an equivalent of Unix `initd(8)` for controlling a daemon process. ========== References ========== .. [stevens] `Unix Network Programming`, W. Richard Stevens, 1994 Prentice Hall. .. [slack-daemon] The (non-Python) “libslack” implementation of a `daemon` tool `<http://www.libslack.org/daemon/>`_ by “raf” <raf@raf.org>. .. [python-daemon] The `python-daemon` library `<http://pypi.python.org/pypi/python-daemon/>`_ by Ben Finney et al. .. [cookbook-66012] Python Cookbook recipe 66012, “Fork a daemon process on Unix” `<http://code.activestate.com/recipes/66012/>`_. .. [cookbook-278731] Python Cookbook recipe 278731, “Creating a daemon the Python way” `<http://code.activestate.com/recipes/278731/>`_. .. [bda.daemon] The `bda.daemon` library `<http://pypi.python.org/pypi/bda.daemon/>`_ by Robert Niederreiter et al. .. [zdaemon] The `zdaemon` tool `<http://pypi.python.org/pypi/zdaemon/>`_ by Guido van Rossum et al. .. [clapper-daemon] The `daemon` library `<http://pypi.python.org/pypi/daemon/>`_ by Brian Clapper. .. [seutter-daemonize] The `daemonize` library `<http://daemonize.sourceforge.net/>`_ by Jerry Seutter. .. [burr-daemon] The `daemon.py` module `<http://www.nightmare.com/~ryb/code/daemon.py>`_ by Ray Burr. .. [twisted] The `Twisted` application framework `<http://pypi.python.org/pypi/Twisted/>`_ by Glyph Lefkowitz et al. .. [dagitses-initd] The Python `initd` library `<http://pypi.python.org/pypi/initd/>`_ by Michael Andreas Dagitses. ========= Copyright ========= This work is hereby placed in the public domain. To the extent that placing a work in the public domain is not legally possible, the copyright holder hereby grants to all recipients of this work all rights and freedoms that would otherwise be restricted by copyright. .. Local variables: mode: rst coding: utf-8 time-stamp-start: "^:Last-Modified:[ ]+" time-stamp-end: "$" time-stamp-line-limit: 20 time-stamp-format: "%:y-%02m-%02d %02H:%02M" End: vim: filetype=rst fileencoding=utf-8 :
Ben Finney schrieb:
Howdy all,
Significant changes in this release:
* Name the daemon process context class `DaemonContext`, since it doesn't actually represent a separate daemon. (The reference implementation will also have a `DaemonRunner` class, but that's outside the scope of this PEP.)
* Implement the context manager protocol, allowing use as a ‘with’ context manager or via explicit ‘open’ and ‘close’ calls.
* Delegate PID file handling to a `pidfile` object handed to the `DaemonContext` instance, and used simply as a context manager.
* Simplify the set of options by using a mapping for signal handlers.
* Target Python 3.2, since the reference implementation will very likely not be complete in time for anything earlier.
This looks like it should be submitted as a formal PEP now; that should also ensure more interest in it, and an eventual resolution. Georg
Georg Brandl <g.brandl@gmx.net> writes:
Ben Finney schrieb:
Howdy all,
Significant changes in this release [of the draft PEP]:
This looks like it should be submitted as a formal PEP now; that should also ensure more interest in it, and an eventual resolution.
Thanks for the support. Unless anyone has strong objections within the next day or so, I'll submit this as a PEP. -- \ “With Lisp or Forth, a master programmer has unlimited power | `\ and expressiveness. With Python, even a regular guy can reach | _o__) for the stars.” —Raymond Hettinger | Ben Finney
participants (21)
-
Aahz -
Andrew Bennetts -
Andrew McNabb -
Antoine Pitrou -
Ben Finney -
Bill Janssen -
Chris Rebert -
Gabriel Genellina -
Georg Brandl -
George Sakkis -
Guido van Rossum -
Jerry Seutter -
Jesse Noller -
Lie Ryan -
nathan binkert -
Oleg Broytmann -
skip＠pobox.com -
Stephen J. Turnbull -
Terry Reedy -
Tim Golden -
Trent Nelson