[Twisted-Python] mktap/twistd in a Windows service
I know this is a FAQ, but it's not in the FAQ list and I've had no luck finding a good answer via web or list searches (Windows is a lousy search term, and service isn't much use in a Twisted context...) I want to run a tiny non-authoritative DNS server on my PC. I've never done this before, but twisted names looks ideal. The documentation (http://twistedmatrix.com/projects/names/documentation/howto/names.html) says I just need to do mktap dns --recursive --cache twistd -f dns.tap and indeed this works fine. I seem to recall some time ago seeing postings to the effect that tap format files were out of favour, but I don't follow the twisted list much, so I don't know if this is still true. Nevertheless, it works so who cares? However, I want to run this as a Windows service. Ideally, twistd would have a --install-service flag or something, but it doesn't so I went looking, and found postings covering tap2ntsvc which creates services from tap files, but it seems to have been replaced by ntsvc, which only handles .tac files :-( I get the impression that the idea is that .tac files are easy to create by hand, but as I know *nothing* about twisted names (and don't really want to - all I want is to run a DNS server!) it's not as easy as it might be... So, can someone tell me: 1. What is the currently favoured form for Twisted application files? 2. How do I make one for twisted names (the equivalent of the mktap line above) and better still how do I generalise this for any mktap command? 3. How do I run it? (twistd, presumably) 4. How do I make that file run as a Windows service? I believe, from what I've found, that the answers are: 1. .tac 2. Don't know - not sure there's a generic answer. 3. twistd -noy 4. ntsvc In which case, the documentation really needs to be updated to explain .tac files throughout, instead of mktap. And I need someone to write me a dns.tac file :-( Ideally, this should be written up as a FAQ or HOWTO somewhere. I'll volunteer to write it up, as long as someone will maintain it - I'm not in a position to do so (I use twisted occasionally, but have no need or desire to follow development versions or discussions on the list). I don't want to bother doing a write-up, if it won't get included in the docs/FAQ and maintained, though... Thanks in advance for any help, Paul.
Paul Moore <pf_moore@yahoo.co.uk> writes:
I want to run a tiny non-authoritative DNS server on my PC. I've never done this before, but twisted names looks ideal. The documentation (http://twistedmatrix.com/projects/names/documentation/howto/names.html) says I just need to do
mktap dns --recursive --cache twistd -f dns.tap
and indeed this works fine. I seem to recall some time ago seeing postings to the effect that tap format files were out of favour, but I don't follow the twisted list much, so I don't know if this is still true. Nevertheless, it works so who cares?
However, I want to run this as a Windows service. Ideally, twistd would have a --install-service flag or something, but it doesn't so I went looking, and found postings covering tap2ntsvc which creates services from tap files, but it seems to have been replaced by ntsvc, which only handles .tac files :-(
For what it's worth, I rarely use twistd for my Twisted applications, and it's not that much work to run Twisted-based code as an NT service without any external utilities. You can set things up with the pywin32's service support (win32service/util), and just have your main entry point run Twisted's reactor. The shutdown code stops the reactor and also terminates the service. Note that service startup runs the main entry point in a second thread, but that's the "main" thread from Twisted's perspective. It does mix up signal handling under Windows, so I typically disable that in the reactor. Also, service control messages (like stop) are run in the service thread (true application main thread), so you need to call into Twisted's thread to stop the reactor. Here's a tiny startup script that provides service support for some application - I've removed any special logging initialization and registry parameter support for clarity. - - - - - - - - - - - - - - - - - - - - - - - - - import os, sys import win32service,win32serviceutil from twisted.internet import reactor from twisted.python import log # Ensure basic thread support is enabled for twisted from twisted.python import threadable threadable.init(1) class Service(win32serviceutil.ServiceFramework): _svc_name_ = '{servicename}' _svc_display_name_ = '{Display name for Service}' def SvcStop(self): self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING) log.msg('Stopping {server}...') reactor.callFromThread(reactor.stop) def SvcDoRun(self): # # ... Initialize application here # log.msg('{Server} running...') reactor.run(installSignalHandlers=0) if __name__ == "__main__": win32serviceutil.HandleCommandLine(Service) - - - - - - - - - - - - - - - - - - - - - - - - - The above provides for install/uninstall based on the default command line handling in win32serviceutil, and of course normal Windows control ("net start", "net stop") afterwards. Registering as a Python module only runs on the machine with Python/pywin32 installed, but you can package the above with something like py2exe. You can then get as complicated as you want from here. There are some helper functions in win32serviceutil (Get/SetServiceCustomOption) that can simplify storing startup information in the registry if you like. Alternatively, I've implemented my own equivalent to HandleCommandLine in some cases where I wanted better control (or preferred "--install" to just "install"), wanted to work in a stdin/stdout debug mode when run directly, or needed to synchronize startup handling with the py2exe version, when I packaged the service into an executable. Note that I'm not sure how the above plays with Twisted's service framework, as I never got comfortable with using it. But aside from that anything beyond the basics above are general python service questions, and not specific to Twisted applications. -- David
On Sun, 04 Nov 2007 17:52:35 +0000, Paul Moore <pf_moore@yahoo.co.uk> wrote:
I know this is a FAQ, but it's not in the FAQ list and I've had no luck finding a good answer via web or list searches (Windows is a lousy search term, and service isn't much use in a Twisted context...)
I want to run a tiny non-authoritative DNS server on my PC. I've never done this before, but twisted names looks ideal. The documentation (http://twistedmatrix.com/projects/names/documentation/howto/names.html) says I just need to do
mktap dns --recursive --cache twistd -f dns.tap
In recent versions of Twisted, you can simplify the above to this: twistd dns --recursive --cache This skips the tap phase: it creates the same application mktap would have created, but instead of putting it in the tap file, it just runs it.
and indeed this works fine. I seem to recall some time ago seeing postings to the effect that tap format files were out of favour, but I don't follow the twisted list much, so I don't know if this is still true. Nevertheless, it works so who cares?
Quite so.
However, I want to run this as a Windows service. Ideally, twistd would have a --install-service flag or something, but it doesn't so I went looking, and found postings covering tap2ntsvc which creates services from tap files, but it seems to have been replaced by ntsvc, which only handles .tac files :-(
It would be very excellent if twistd had a --install-service flag. There is a relatively minimal level of interest in Windows-specific functionality among the currently active Twisted developers, though. This means that it isn't very likely that any of us will undertake to add this feature. That doesn't mean that we'll never implement it, nor that we wouldn't accept a patch which implements it from someone else who is more interested in it than we are. If someone provides the functionality, with unit tests, then we'll gladly incorporate and maintain it. I'm not sure if you would be interested in doing this work or not; I'm directing this as much at you as at all of the other users of Twisted on Windows who I know are out there and mostly remain silently.
I get the impression that the idea is that .tac files are easy to create by hand, but as I know *nothing* about twisted names (and don't really want to - all I want is to run a DNS server!) it's not as easy as it might be...
They are, more or less, although they are harder to make than tap files. ;)
So, can someone tell me:
1. What is the currently favoured form for Twisted application files?
It all depends on the use-case. For all of the services distributed with Twisted itself, tap files offer very little over the persistence-free form of twistd I gave an example of above. Since tap files contain pickled data, they have all the problems of pickle (lack of introspectability, problems with compatibility, security); if these aren't problems, then tap files might be okay. On the other hand, many uses of twistd don't actually need any persistence, so using tap files doesn't actually provide any benefits.
2. How do I make one for twisted names (the equivalent of the mktap line above) and better still how do I generalise this for any mktap command?
See twisted.names.tap.makeService - this is the function `mktap dns´ is calling to create the object it will pickle into the tap file. Your tac file can call it directly with the desired configuration and add the resulting service to an application object. This will give you a tac which does the same thing as your mktap-produced tap file. You can read more about writing tac files here: http://twistedmatrix.com/projects/core/documentation/howto/application.html
3. How do I run it? (twistd, presumably)
As you said below, twistd -noy.
4. How do I make that file run as a Windows service?
I've never used ntsvc, so I can't say if that's right, but presumably it is at least close. There isn't any reason that if ntsvc works with tac files it couldn't also work with tap files, though. So another solution might be to explore adding tap file support to ntsvc. The interface for loading a tac file is really the same as the interface for loading a tap file, in fact, so I am slightly surprised ntsvc doesn't work with tap files.
I believe, from what I've found, that the answers are: 1. .tac 2. Don't know - not sure there's a generic answer. 3. twistd -noy 4. ntsvc
In which case, the documentation really needs to be updated to explain .tac files throughout, instead of mktap. And I need someone to write me a dns.tac file :-(
Ideally, this should be written up as a FAQ or HOWTO somewhere. I'll volunteer to write it up, as long as someone will maintain it - I'm not in a position to do so (I use twisted occasionally, but have no need or desire to follow development versions or discussions on the list). I don't want to bother doing a write-up, if it won't get included in the docs/FAQ and maintained, though...
Anything (accurate, coherent) you want to write up we'd be glad to add to the Twisted repository and include on the website, in releases, etc. Jean-Paul
Jean-Paul Calderone wrote:
However, I want to run this as a Windows service. Ideally, twistd would have a --install-service flag or something, but it doesn't so I went looking, and found postings covering tap2ntsvc which creates services from tap files, but it seems to have been replaced by ntsvc, which only handles .tac files :-(
It would be very excellent if twistd had a --install-service flag. There is a relatively minimal level of interest in Windows-specific functionality among the currently active Twisted developers, though. This means that it isn't very likely that any of us will undertake to add this feature.
That doesn't mean that we'll never implement it, nor that we wouldn't accept a patch which implements it from someone else who is more interested in it than we are. If someone provides the functionality, with unit tests, then we'll gladly incorporate and maintain it. I'm not sure if you would be interested in doing this work or not; I'm directing this as much at you as at all of the other users of Twisted on Windows who I know are out there and mostly remain silently.
Thanks for the information. I would love to say that I would look at this, but I really don't have the knowledge of Twisted, or the time to do a decent job. I'll certainly add it to my to do list, but although I love twisted in theory, in practice, I don't write enough of the type of software it's good at, for me to more than dabble. My main use, as I hinted at in my original mail, is to just use the prepackaged functionality - for which you have my eternal gratitude. (As it happens, the need I originally had for a DNS server has passed - I found an easier workaround for the specific case I was interested in). If anyone does want to work on this, I'd certainly be happy to help where I can - I have a reasonable amount of Windows coding experience, both in Python and otherwise. Thanks once again for the detailed reply. Paul.
participants (3)
-
David Bolen -
Jean-Paul Calderone -
Paul Moore