Terry Reedy wrote:

The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not.

Right. Personally, I think it's pretty easy to make a daemon on Unix. If this module supported both Unix and Windows, though, that would be interesting. Bill

On Sun, Jan 25, 2009 at 9:01 PM, Andrew Bennetts wrote:

Bill Janssen wrote:

...

Terry Reedy wrote:

...

The question will then be whether this unix-only module is generally useful enough to be in the stdlib. Many good modules are not.

Right. Personally, I think it's pretty easy to make a daemon on Unix.

It's pretty easy, but still fiddly enough that I'd rather use a library than worry about the details myself.

I strongly second this sentiment. It's such an oft-used recipe that it deserves a canonical, fully correct library implementation.

...

If this module supported both Unix and Windows, though, that would be interesting.

That would be lovely, but even a Unix-only module would be worthwhile, IMO.

Though it would probably improve its chances of getting added to the std lib. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com

Andrew Bennetts writes:

Bill Janssen wrote:

...

If this module supported both Unix and Windows, though, that would be interesting.

That would be lovely, but even a Unix-only module would be worthwhile, IMO.

Jerry Seutter writes:

It would be nice if this functionality could work on windows machine as well as unix flavors.

My knowledge of programming on Windows is quite limited. AIUI, the steps required to make a Windows program into a daemon process are markedly different from the canonical Stevens procedure, correct? In that case, how would an implementation on Windows need to change the API given in this PEP? -- \ “I wish there was a knob on the TV to turn up the intelligence. | `\ There's a knob called ‘brightness’ but it doesn't work.” | _o__) —Eugene P. Gallagher | Ben Finney

Bill Janssen writes:

Ben Finney wrote:

...

[…] how would an implementation on Windows need to change the API given in this PEP?

Yes, that's the question. You should figure out the answer, or find some more folks who can help you figure out the answer, and update the PEP to reflect it.

Based on what I can see in the Python Cookbook (e.g. URL:http://code.activestate.com/recipes/551780/), Windows doesn't even call it a “daemon”, instead calling it a “service”. The customisation options and expected behaviour are wildly different from what a Unix programmer would expect. So I think that a module for creating Unix-style daemons, with API and options geared toward making that easy, has no business also getting involved with creating Windows services. It would result in an unduly complicated API for no benefit to either Unix programmers or Windows programmers. If someone else wants to offer a separate PEP to include a ‘winservice’ module to the standard library, that seems a better way forward for that use case. -- \ “I'm beginning to think that life is just one long Yoko Ono | `\ album; no rhyme or reason, just a lot of incoherent shrieks and | _o__) then it's over.” —Ian Wolff | Ben Finney

On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:

On Tue, Jan 27, 2009 at 06:53:36PM +1100, Ben Finney wrote:

...

Based on what I can see in the Python Cookbook (e.g. URL:http://code.activestate.com/recipes/551780/), Windows doesn't even call it a ???daemon???, instead calling it a ???service???. The customisation options and expected behaviour are wildly different from what a Unix programmer would expect.

Not only it's a completely different API - it requires pywin32 which is not in the stdlib.

...

So I think that a module for creating Unix-style daemons, with API and options geared toward making that easy, has no business also getting involved with creating Windows services. It would result in an unduly complicated API for no benefit to either Unix programmers or Windows programmers.

+1 for unix-only daemonization, -1 for making a universal unix/w32 API.

Meh, Unix-only stuff is such a pain because most developers or projects could cobble together Unix-only stuff themselves; the same can rarely be said for Windows-equivalent stuff. It's not hard to abstract a general service interface; Qt has been doing it for a long time: http://doc.trolltech.com/solutions/qtservice/qtservice.html Trent.

Oleg Broytmann wrote:

On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:

...

On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:

...

Not only it's a completely different API - it requires pywin32 which is not in the stdlib. [skip] +1 for unix-only daemonization, -1 for making a universal unix/w32 API. It's not hard to abstract a general service interface

Do you propose to include pywin32 into python?

Ummm.. Python already has a slew of windows-specific code in it. You could -- if it seemed suitable -- copy the pywin32 code with appropriate attribution, but you don't need the whole thing. And you don't necessarily need it at all. TJG

Jesse Noller wrote:

On Tue, Jan 27, 2009 at 11:23 AM, Tim Golden wrote:

...

Oleg Broytmann wrote:

...

On Tue, Jan 27, 2009 at 04:06:44PM +0000, Trent Nelson wrote:

...

On Tue, Jan 27, 2009 at 12:00:59PM +0300, Oleg Broytmann wrote:

...

Not only it's a completely different API - it requires pywin32 which is not in the stdlib. [skip] +1 for unix-only daemonization, -1 for making a universal unix/w32 API. It's not hard to abstract a general service interface Do you propose to include pywin32 into python? Ummm.. Python already has a slew of windows-specific code in it. You could -- if it seemed suitable -- copy the pywin32 code with appropriate attribution, but you don't need the whole thing. And you don't necessarily need it at all.

TJG

I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.

I'm inclined to agree with you. Or at least I think a PEP proposing a cross-platform daemon/service framework would be quite different from the one Ben is authoring. I was really just commenting on Oleg's suggestion that anything making use of win32 code needed to use pywin32. TJG

Jesse Noller writes:

I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.

Thanks, this is my position also. I have documented this now, and it will appear in the next version of the PEP. -- \ “The problem with television is that the people must sit and | `\ keep their eyes glued on a screen: the average American family | _o__) hasn't time for it.” —_The New York Times_, 1939 | Ben Finney

On Tue, Jan 27, 2009 at 8:29 PM, Trent Nelson wrote:

On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:

...

Jesse Noller writes:

...

I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.

Thanks, this is my position also. I have documented this now, and it will appear in the next version of the PEP.

I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: the PEP gets approved, the daemon module gets written, and, because it's so useful, uptake is immediate, people start writing code that uses it.

Or is it? Don't get me wrong, I *heart* Unix. But I get paid to develop in Windows. Because frankly, that's where the big bucks are and it's an extremely satisfying transition to jettison all emotional reasoning and just work for whoever is willing to pay you the most. I highly recommend it. But for now, I digress.

Don't worry; I think most of us would gladly work for money. In fact, I would hazard most of us like money a lot.

So, back to your Unix-only daemon module. It's good. Great, in fact. All sorts of projects start using it. Subversion uses it to write some funky persistent daemon that speeds up hooks by 8000% percent. Well that's just super! I'd like to use that for one of my clients, but, argh, no, wait, I can't, it's trying to use some daemon module that Google tells me is Unix-only.

After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy, just your normal, run-of-the-mill start/stop type stuff. Why on earth, I think to myself, is this great 3rd party Python utility using a Unix-only daemon module, thus requiring me to hack it to bits in order for it to run on Windows, wasting my time and my client's time.

Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking. A unix-only daemon - and then a complimenting windows-only service library - makes sense. Ideally, they would have close to the same API, but I don't think log jamming all of the vagaries of both into one beast makes sense. Then again, my windows-ability is out of date and rusty. -jesse

Trent Nelson writes:

On Tue, Jan 27, 2009 at 08:49:52PM -0500, Jesse Noller wrote:

...

Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.

A unix-only daemon - and then a complimenting windows-only service library - makes sense. Ideally, they would have close to the same API, but I don't think log jamming all of the vagaries of both into one beast makes sense. Then again, my windows-ability is out of date and rusty.

See, I disagree with that as well. Take a look at what Qt did with their QtService class:

http://doc.trolltech.com/solutions/qtservice/qtservice.html

Thanks for mentioning that again; I saw another reference to it earlier, but didn't respond.

I'd like to hear arguments for why that QtService interface is perceived as being deficient.

Off the top of my head: The QtService model seems to be “start an additional process and maintain it at arms length, with a bunch of communication channels into and out of it”. Contrariwise, this PEP addresses the use case of “turn the *current* program into a daemon process, so that everything further this program does is running inside that process”. It's too big, as a result of the above difference in focus. When writing a Unix daemon program, I don't want the great majority of what that interface offers. It misses a lot, again as a result of the mismatch in purpose. After setting up a QtService, I need to continue chattering with it to find out what's going on inside it, and to do things like change its state or log a message. The current PEP, on the other hand, doesn't have the model of managing a separate process; the daemon is the *current* program, so one can just do things like output or state changes as one normally would. In short: It addresses a different problem to what I'm addressing with this PEP. -- \ “If nature has made any one thing less susceptible than all | `\ others of exclusive property, it is the action of the thinking | _o__) power called an idea” —Thomas Jefferson | Ben Finney

On Jan 28, 2009, at 7:57 PM, Antoine Pitrou wrote:

Jesse Noller writes:

...

Windows Services != Unix daemons. Admittedly, I haven't had to do it in awhile, but from what I remember, windows services are nowhere near the same type of beast as a unix daemon, and trying to make a unified daemon library that glosses over the differences might cause seizures and habitual smoking.

That would be like a portable library which would allow to write multiprocess programs by glossing over the differences between Unix and Windows processes... Wouldn't it?

Touché good sir, touché

Jesse Noller wrote:

Since Antoine called me to the mat on this one, and I've been chewing on this more today - I think the cross-platformedness (is that a word?) idea is a good one, even if the Qt design doesn't necessarily apply here completely, having a single abstraction to both (but different modules under that abstraction based on os) is attractive (as Stephen pointed out).

Perhaps such an abstraction should be built on top of Ben Finney's proposed unix daemon module, rather than built in? -Andrew.

Jesse Noller writes:

Perhaps; but it makes sense to nest it under the abstraction - daemons.unixdaemon/winservice/osxdaemon

I really don't think it's helpful to conflate this “service” model with the much simpler concept of a Unix daemon. To implement a service on Unix, it's most logical to use a daemon; but the reverse is definitely not true (which is a large part of my resistance to expanding the current PEP to conflate the two concepts). Instead, I think that people who want a “service” would be better served by a top-level module named ‘service’, to preserve that concept as separate from the concept of a daemon. In other words, my understanding is: ‘import service’ should get an API (so far in no PEP that I know of) for a distinct purpose, that of implementing an MS Windows-style service, which uses whatever OS-specific implementation makes sense. ‘import daemon’ gets a totally different API for a different purpose, that of turning the current running program into a daemon, which AFAICT only makes a whole lot of sense on Unix and is what I'm proposing in this PEP. It makes much more sense for ‘daemon’ to stay simple and continue to mean the Unix-specific concept of “daemon”, without necessarily obstructing whoever wants to implement ‘service’, than for ‘daemon’ in Python to mean something other than the Unix meaning. -- \ “No matter how far down the wrong road you've gone, turn back.” | `\ —Turkish proverb | _o__) | Ben Finney

Bill Janssen writes:

There are a number of Unix-only modules for this purpose already, aren't there? I see at least three of them on PyPI, including yours, Ben.

Yes, none of which meet the criteria listed in the PEP for “well-behaved daemon”. They're useful, to be sure; but they're lacking when judged against what a well-behaved Unix daemon should be doing.

What's the point of a new one?

Perhaps I've mistakenly given the impression that I'm re-inventing the wheel; sorry for that false impression if so. To implement this PEP fully, I've taken the existing implementation that was closest to the requirements (‘bda.daemon’), and based this work on altering it to conform with this PEP. I'm in active correspondence with the author of ‘bda.daemon’.

And what's the point of adding a new Unix-only module to the stdlib?

To have this common, fairly standardised but tricky, use case implemented once and easily available to any Python programmer, without them needing to know the fine details of what's required to make it work internally. Having multiple partial implementations in PyPI and the Cookbook leads only to the situation where a prospective user has to either know intimate details of this rather fiddly task to judge each of them, or make a choice arbitrarily. -- \ “I don't know half of you half as well as I should like, and I | `\ like less than half of you half as well as you deserve.” —Bilbo | _o__) Baggins | Ben Finney

On Thu, Jan 29, 2009 at 01:35:21PM +1100, Ben Finney wrote:

It makes much more sense for ???daemon??? to stay simple and continue to mean the Unix-specific concept of ???daemon???, without necessarily obstructing whoever wants to implement ???service???, than for ???daemon??? in Python to mean something other than the Unix meaning.

There are two types of developers that will utilise this module. Those that are writing something for themselves, that never has to run anywhere else other than Unix. Or those that work primarily or exclusively in Unix, but understand that others might want to run their code elsewhere. If you build it, they will come. By building the latter solution, sure, you've inconvenienced the guy that wants a Unix-only daemon, but you've also enabled everyone else who wants to write stuff that has a chance of running unmodified on other platforms. And jumping on the bandwagon of a few others; I like the argument that something Unix-only like this could just as easily belong in PyPI, whereas a cross-platform solution is more viable for stdlib inclusion. We should make it harder to write non-portable code, not easier. Trent.

Ben Finney writes:

Those people would be well served by a putative "service" interface. I would welcome such a PEP from you, since you seem to have fairly solid ideas of what it should look like.

AFAICS it looks like yours, except with the requirement of "works on Windows" and a strong preference for "no other-OS-specific clutter with a simple 'import daemonize'". Bill added "plays nicely with Mac OS X". I don't see on what grounds you object to exploring the possibility, since you say you don't know much about Windows services. Note that nobody is suggesting that you write the code that implements daemons via Windows services (of course they'd love it if you did!), only that you open up the API design to changes that would make some programs work on all platforms without conditioning on platform, and make all programs work with minimal platform-specific code. So I suggest that you bat the ball back into their court, and announce on Python-Dev that you're willing but not able to generalize your PEP to Windows, and ask for coauthors to help. If there are no takers, then you're in a strong position to argue that Unix-like OSes are the important case anyway, nobody cares about Windows services in Python. If there are, then somebody else does the work and everybody is happy. No?

...

We should make it harder to write non-portable code, not easier.

It's a bit of a stretch to go from "designed to work on any Unix-like OS" to "non-portable".

That is not a Pythonic attitude. Here, "portable code" means "designed to work in any instance of Python conditioning only on 'this instance supports the feature'."

Ben Finney writes:

[C]ompare the latest version of this PEP [...].

I'm assuming the one posted on 1/29 in <87iqnywsx2.fsf@benfinney.id.au>, or fairly close to that.

This might because the PEP isn't communicating very well. Does the section "A daemon is not a service" help to explain the fundamental difference between the model suggested by Trent, versus the model in the PEP?

That is quite clear, and you've said "daemonize-me != start-a- service" in discussion here. However, you've also said that "daemon != service" (including in the title of that section!), which clearly is mostly false when those terms are referring to the processes that are running in the background. I think that focusing on the distinction between "*becoming* a daemon" and "*starting* a service" is distracting. The point being that *becoming* a well-behaved service surely requires becoming a well- behaved background process. The real issue is not starting a separate process (just don't provide a way to do it), but rather it is that it also requires all the communication setup as well, which is out-of- scope for a daemonization PEP on any platform (apart from closing file descriptors, which is specific to the fork(2) implementation of process creation). To prevent the misinterpretation I made, I would phrase that title "A Service is Not Just a Daemon", implying "this PEP is not a full-service PEP".<wink> BTW, IMO you can delete the sentence "It is the opinion of this PEP's author that combining the two disparate systems of behaviour under a single API would be contrary to the focus of this PEP." from the PEP. It's redundant.

Meanwhile, I and others have a need for a simple, standardised, one-obvious-way "daemonise the current program" library, *entirely distinct from* any of ther extra baggage that comes with a "spawn a separate process that I can then communicate with through channels" API. I intend to satisfy that with this PEP.

Well, that's what I thought was in question here. It seems to me that there are likely to be additional aspects to "daemonize me" for Mac OS X and Windows, but that the basic start-reload-stop methods should probably apply. I'm not familiar enough with the process of daemonization on Mac OS X (Bill mentioned interaction with launchd) or Windows to help at reasonable expense, but I certainly hope that those who are will review to see if a similar API might be useful to them. Unfortunately there doesn't seem to be that much common ground among the advocates, though. Steve

nathan binkert writes:

It's tough to follow this whole thread as it has spread all over the place, so my apologies if this has been mentioned.

Why not try to make both the unix only daemon and the generic service available? The latter could use the former for its implementation?

Yes, this is my position. I'm glad to see that I'm not alone, thanks :-) -- \ “Puritanism: The haunting fear that someone, somewhere, may be | `\ happy.” —Henry L. Mencken | _o__) | Ben Finney

Trent Nelson writes:

On Wed, Jan 28, 2009 at 08:18:51AM +1100, Ben Finney wrote:

...

Jesse Noller writes:

...

I think adding the windows services functionality distracts from the reasonable goal of the PEP Ben is proposing.

Thanks, this is my position also.

I disagree. Partly because I'm in a bit of a devil's advocate mood at the moment. But mostly because I know what will happen: […]

I respectfully suggest that this isn't true, because your hypothetical situation continues:

After poking around the source, I'm perplexed. It's not doing anything uniquely Unix-xy,

Your disagreement seems to rest on this assertion, and I don't see how your hypothetical described can be true. Do you really see the described behaviour in the current draft PEP as “not doing anything uniquely Unix-y”?

just your normal, run-of-the-mill start/stop type stuff.

As far as I can tell, beyond trivial use cases, there is both wide discrepancy and very little overlap in “start/stop type stuff” between a Unix daemon and an MS Windows service. At least, none of the (admittedly few) instructions on MS Windows services I've seen demonstrate otherwise. Someone who thinks otherwise is welcome to attempt a PEP that brings all that behaviour under a single banner *and* design the API to be both flexible enough to be useful in many use cases and simple enough to be worth learning instead of doing it manually. That person will not be me. -- \ “Contentment is a pearl of great price, and whosoever procures | `\ it at the expense of ten thousand desires makes a wise and | _o__) happy purchase.” —J. Balguy | Ben Finney

(I'm only addressing commentary on the current PEP here. Comparisons to MS Windows services have been addressed in other messages.) Trent Nelson writes:

On Wed, Jan 28, 2009 at 01:15:41PM +1100, Ben Finney wrote:

...

`prevent_core` :Default: ``True``

If true, prevents the generation of core files, in order to avoid leaking sensitive information from daemons run as `root`.

I was going to suggest service.set_secure(True)

That name implies something emphatically not true: that a program's security is a binary toggle, and it can be turned on or off.

but then I wondered what is this actually achieving? Won't the resulting .core will be owned by root?

Yes. Which is less secure than never creating the core file at all.

...

`lockfile_directory` :Default: ``'/var/run'``

Absolute directory path to contain the daemon's lockfile. If ``None``, the lockfile behaviour for this daemon is skipped.

`lockfile_name` :Default: ``None``

Base name of the lockfile for this daemon, without directory or suffix. If ``None``, the name is derived from the process command line.

All you're doing is ensuring multiple instances of the daemon don't run.

Also allowing the location and name of the lock file to be customised.

On Windows, you'd use a mutex, on Unix, sure, lockfile makes sense. A cross-platform interface wouldn't bother exposing this implementation detail.

The details are exposed precisely so that they can be customised by programs that need it. This is common, for example, to allow for differing conventions or standards of filesystem layout. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but I find scratching just makes it worse.” —_Pinky and | _o__) The Brain_ | Ben Finney

Ben Finney writes:

The details [of the lockfiles implementing a mutex] are exposed precisely so that they can be customised by programs that need it. This is common, for example, to allow for differing conventions or standards of filesystem layout.

Sure, but the question being asked is why should users of the daemon process library need to worry about that, separately from other users of lockfiles or mutexes? If they need to worry about it only because there's no satisfactory module implementing it widely available, then that could be done in a subsidiary module that handles OS-specific details, and optionally exposes OS-specific special features (a la os and os.posix). That would also make refactoring (close to) trivial when such a module did become widely available. It's obvious you don't want to go in that direction, which is IMHO good and sufficient reason for you not to do so. But as it stands IMO it's a weakness in the PEP, even if Unix-only is otherwise acceptable for the stdlib.

On Thu, Jan 29, 2009 at 01:42:44PM +1100, Ben Finney wrote:

rather specific Unix-native concept: the "pidfile", a combination sentinel, lockfile, and primitive text interface.

IWBN to have a main() function in the module to be used from the command line: $ python -m daemon --pidfile=/var/run/mydaemon.pid status Daemon pid 1221 is running $ python -m daemon --pidfile=/var/run/mydaemon.pid stop $ python -m daemon --pidfile=/var/run/mydaemon.pid status Cannot find /var/run/mydaemon.pid A note on implementation detail: IWBN if the pidfile is removed on stop(). Please do not put the burden unto the user.

I might return the terminology in the PEP back to "pidfile" to be clear.

+1 Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

On Fri, Jan 30, 2009 at 08:14:07AM +1100, Ben Finney wrote:

Oleg Broytmann writes:

...

IWBN to have a main() function in the module to be used from the command line:

$ python -m daemon --pidfile=/var/run/mydaemon.pid status Daemon pid 1221 is running

$ python -m daemon --pidfile=/var/run/mydaemon.pid stop

$ python -m daemon --pidfile=/var/run/mydaemon.pid status Cannot find /var/run/mydaemon.pid

Interesting. Should that be defined in this PEP, or merely added to the reference implementation as a bonus?

I think an implementation would be enough.

...

A note on implementation detail: IWBN if the pidfile is removed on stop(). Please do not put the burden unto the user.

Yes, that's already addressed in the defined behaviour for `stop()`.

Thank you! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

Ben Finney writes:

Skip Montanaro has a PyPI package, ‘lockfile’ (currently marked "Beta") URL:http://pypi.python.org/pypi/lockfile that has such ambitions.

Should I make an even lower-level PEP to suggest inclusion of Skip's ‘lockfile’ package? Is that wise? Is it polite?

I would send your proposal for PIDlockfile to Skip. In your PEP, describe the relevant parts of the API explicitly in the PEP, with a footnote that you are deliberately extending the lockfile interface and currently intend to remain compatible with that interface. Then talk to Skip about a PEP for including his package, or perhaps extending your PEP to include it. Which is better depends on details and how aggressively Skip would want to pursue inclusion of lockfile if your daemon package doesn't make it or takes a long time. I don't know about "polite", but I think it would be unwise to try to PEP lockfile without Skip's explicit support.

Ben> For those people who have asked about the status of this: Yes, I'm Ben> currently working with Skip on the ?lockfile? package so that it Ben> can be used for the ?daemon? implementation. Indeed. I should have my Mercurial repository in a more globally visible place later today so Ben and anyone else interested can bash bits. Skip

Terry Reedy writes:

If I were wanting to make a *nix daemon, I would love to have this.

I'm glad to have many comments like this in support of this PEP. Thanks, everyone!

As I understand things, you should finish the implementation, post it to PyPI, and get people to use and test it, (all of which require no approvals).

Sure, but before I “finish” the implementation, I want to reach consensus on the suitability of the proposed API. To quote PEP 1: It is better to finish the specification and rationale first and reach consensus on it before writing code.

When stable, propose it for the stdlib along with a credible promise to maintain it for at least 3 years.

My intention is to continue supporting this code after inclusion in the standard library, yes. -- \ “Liberty, n. One of imagination's most precious possessions.” | `\ —Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney

"Stephen J. Turnbull" writes:

The people who are posting "show us the code" are mostly folks with long experience in the PEP process. I think you should take it as an indication that in this case the definition of "suitable API" is an "I'll know it when I've tried it" kind of thing.

I am working on ‘python-daemon’ concurrently with this PEP now, and intend to keep publishing updates to that implementation as it approaches conformance with the PEP.

What you might find useful is to avoid writing any code yourself, but see how well your proposed spec fits the existing code, of which several (potential) examples have been posted.

Fortunately I've based ‘python-daemon’ on what I determined to be the closest starting point for the well-behaved daemon behaviour. -- \ “Any intelligent fool can make things bigger and more complex… | `\ It takes a touch of genius – and a lot of courage – to move in | _o__) the opposite direction.” —Albert Einstein | Ben Finney

On Mon, Jan 26, 2009 at 2:40 PM, Ben Finney wrote:

Guido van Rossum writes:

...

On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:

...

I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.

I would like to note that there are many existing packages that provide support for daemonization […]

Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).

I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.

...

I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.

This suspicion could well be true.

On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.

Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?

Depends on how much bikeshedding you want. I expect that you'll get enough of that without involving c.k.py... -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Was that you in the YouTube video that was one of the first hits for "PyCon daemon" ? That was hilarious. :-) On Mon, Jan 26, 2009 at 2:55 PM, Jerry Seutter wrote:

It would be nice if this functionality could work on windows machine as well as unix flavors.

I wrote the package at http://daemonize.sourceforge.net in frustration after seeing daemons built poorly several times. Some of the instances had been done incorrectly even after the developer had been pointed to the ActiveState page. Understanding why a daemon should be written the way it is requires a fair bit of historical knowledge that younger people occasionally do not have. Some of the recent grads hired at my workplace did not know what a daemon was. "Oh, like a web container?" Jerry Seutter On Mon, Jan 26, 2009 at 3:40 PM, Ben Finney wrote:

...

Guido van Rossum writes:

...

On Sun, Jan 25, 2009 at 2:11 PM, Ben Finney wrote:

...

I am preparing a PEP, and corresponding reference implementation, for a standard implementation of the steps needed to turn a program into a well-behaved Unix daemon.

I would like to note that there are many existing packages that provide support for daemonization […]

Good point, thanks (and thanks to others in this thread for drawing my attention to some more that I hadn't yet found).

I will make note, in the next revision, of the existing equivalent Python works of which I'm aware, and how their implementation differs from the intent of this PEP. If anyone feels I'm missing any, I would be happy to be informed.

...

I have an inkling that this is one of those problems where the many solutions that are hard to compare, because they have different APIs or make different assumptions about the ultimate goals.

This suspicion could well be true.

On the other hand, there already seems to be a good amount of positive support for the PEP, so I'd like to attempt to come to a sensible API that covers most use cases.

Is 'python-ideas' the right forum to hash this out? Or is now the right time to subject it to the inevitable by posting to 'comp.lang.python'?

-- \ "Pinky, are you pondering what I'm pondering?" "Well, I think | `\ so, Brain, but it's a miracle that this one grew back." —_Pinky | _o__) and The Brain_ | Ben Finney

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, Jan 29, 2009 at 9:56 AM, Andrew McNabb wrote:

...

As of 2009-01-26, the package is under active development and is not yet a full implementation of this PEP.

I'm still confused why there's a PEP before there's a stable implementation and a base of users.

Let alone the *need* to be included in the stdlib, when Pypi seems perfectly appropriate for such package, especially if it's unix-only. George

Andrew McNabb writes:

I'm still confused why there's a PEP before there's a stable implementation and a base of users.

Because that's exactly what PEP 1 explicitly advises. Am I wrong to be following PEP 1? -- \ “We should strive to do things in [Gandhi's] spirit… not to use | `\ violence in fighting for our cause, but by non-participation in | _o__) what we believe is evil.” —Albert Einstein | Ben Finney

Hello,

`files_preserve` :Default: ``None``

List of files that should *not* be closed when starting the daemon.

Why does it have to be explicit? If I have an open file lying around, it should be obvious that I don't want it closed. Otherwise I'd have closed it myself... The exception is of course the three stdio streams, which should be treated separately.

`umask` :Default: ``0``

File access creation mask (“umask”) to set for the process on daemon start.

Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects.

Why this behaviour? It's easy enough to call os.umask() manually if you want it, but there are certainly situations where you don't want to change the umask (situations where the daemon is meant to act on behalf of the user who started it). Or, perhaps, umask could be made mand (regardless of which, 0 isn't a sensible umask default!)

* If the `pidfile_directory` attribute is not ``None``:

* Look in that directory for a file named '`pidfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process.

It should first check that there does exist a processus with that number, so that stale pid files don't cause too many problems. Regards Antoine.

Antoine Pitrou writes:

Hello,

...

`files_preserve` :Default: ``None``

List of files that should *not* be closed when starting the daemon.

Why does it have to be explicit? If I have an open file lying around, it should be obvious that I don't want it closed. Otherwise I'd have closed it myself...

Is it so obvious? In Python programs, I far more often see files left open simply because the programmer expects them to be cleaned up at the appropriate time. By the references cited in this PEP, the time of starting the daemon is such an appropriate time to close all open file descriptors.

...

`umask` :Default: ``0``

File access creation mask (“umask”) to set for the process on daemon start.

Since a process inherits its umask from its parent process, starting the daemon will reset the umask to this value so that files are created by the daemon with access modes as it expects.

Why this behaviour?

Because the umask is inherited from the environment of the parent process of the current program, which is good when the current program is conceptually an extension of the parent program's environment, but not in the case where one is intentionally disassociating from that environment.

It's easy enough to call os.umask() manually if you want it, but there are certainly situations where you don't want to change the umask (situations where the daemon is meant to act on behalf of the user who started it).

Those situations exist, but I would argue they are not the common case.

Or, perhaps, umask could be made [mandatory when the options ‘uid’ or ‘gid’ are set.]

That diverges from the referenced “correct behaviour”, in particular the Stevens reference. I think if this PEP is worth implementing, it's for exactly the purpose of providing a Python implementation of defined correct daemon behaviour; that definition includes re-setting the umask.

(regardless of which, 0 isn't a sensible umask default!)

The default umask ‘0’ is chosen in the references as the umask that allows file access modes for files created by the daemon to work as expected. Quoting Stevens: This prevents any files created by the daemon from having their access bits modified. For example, if a daemon specifically created a file with a mode of 0660, so that only the user and group could read and write the file, but the ``umask`` value was 060, the group read and write permissions would be turned off. If the daemon required the group permissions to be on, so that some other process in that group could access the file, this ``umask`` value prevents it. -- \ “I don't accept the currently fashionable assertion that any | `\ view is automatically as worthy of respect as any equal and | _o__) opposite view.” —Douglas Adams | Ben Finney

Antoine Pitrou writes:

Hello, […]

...

* If the `pidfile_directory` attribute is not ``None``:

* Look in that directory for a file named '`pidfile_name`.pid'; if it exists, raise a `DaemonError` to prevent multiple instances of the daemon process.

It should first check that there does exist a processus with that number, so that stale pid files don't cause too many problems.

Good point, thanks. What would be appropriate behaviour in the case of a stale PID file? Abort the daemonisation attempt? Delete the stale lock file silently and continue as though it didn't exist? -- \ “I bought some powdered water, but I don't know what to add.” | `\ —Steven Wright | _o__) | Ben Finney

[responding very late] On Thu, Jan 29, 2009, Antoine Pitrou wrote:

Ben Finney writes:

...

What would be appropriate behaviour in the case of a stale PID file? Abort the daemonisation attempt? Delete the stale lock file silently and continue as though it didn't exist?

Delete the stale lock file silently and continue as though it didn't exist. (and, of course, create another one for the current process)

This should be determined by the API -- for example, a process running against an NFS disk that may be run from any of several servers but should only run on one server at any time ought to abort if there's a stale PID file. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons." --Aahz

En Thu, 29 Jan 2009 04:03:05 -0200, Ben Finney escribió:

:PEP: XXX :Title: Standard daemon process library :Version: 0.4

After reading this PEP, I see it as a big list of implementation details. A more high level view would be more suitable for a multiplatform implementation. Like when using most library modules, I trust the authors, I hope they've done their job well, that they've read their Stevens and Pietrek, and the code follows the best practices. For the most part, I don't care about the implementation - just that it don't gets in my way. I'd say that a daemon is just a background process, detached from any terminal and without any user interactivity, that keeps running independently of the logged user. *How* to get there, it's the library job. So I need a method to "become a daemon". Stopping a daemon process must be done orderly so a "stop" method is required too. It needs some way to react to external requests (signals, or control requests on Windows): they're a short list, we can use methods following a naming convention (e.g. "on_reload"). And that's all, at least conceptually; probably we need some attributes to customize the behaviour. I think it's a lot easier to implement such abstract view on different systems. Excluding Windows right from the start --because of so many details that should be emulated, and are mostly irrelevant-- would be a bad idea. -- Gabriel Genellina

Ben Finney schrieb:

Howdy all,

Significant changes in this release:

* Name the daemon process context class `DaemonContext`, since it doesn't actually represent a separate daemon. (The reference implementation will also have a `DaemonRunner` class, but that's outside the scope of this PEP.)

* Implement the context manager protocol, allowing use as a ‘with’ context manager or via explicit ‘open’ and ‘close’ calls.

* Delegate PID file handling to a `pidfile` object handed to the `DaemonContext` instance, and used simply as a context manager.

* Simplify the set of options by using a mapping for signal handlers.

* Target Python 3.2, since the reference implementation will very likely not be complete in time for anything earlier.

This looks like it should be submitted as a formal PEP now; that should also ensure more interest in it, and an eventual resolution. Georg