[Twisted-Python] twisted daemonize problem
I try to daemonize twisted, but it uses 100% cpu. The problem occurs in this code: def daemonize(log_stdout=False, out_name=None, log_stderr=False, err_name=None): """ Turn current process into a daemon """ pid = os.fork() if (pid == 0): ## os.chdir (workdir) os.setsid() maxfd = os.sysconf("SC_OPEN_MAX") for fd in range(0, maxfd): try: os.close(fd) except OSError: # ERROR (ignore) pass # Redirect the standard file descriptors to /dev/null. <<<< HERE's THE PROBLEM os.open("/dev/null", os.O_RDONLY) # standard input (0) os.open("/dev/null", os.O_RDWR) # standard output (1) os.open("/dev/null", os.O_RDWR) # standard error (2) if (not out_name): out_name = gen_name() + '.out' if (not err_name): err_name = gen_name() + '.err' if (log_stdout): sys.stdout = logger.logger (out_name) if (log_stderr): sys.stderr = logger.logger (err_name) else: print "forked pid", pid sys.exit (0) If I comment out the lines that open /dev/null, everything is OK, but with those lines the process uses 100% cpu. The actual twisted code (not shown) is class Spawner(xmlrpc.XMLRPC): ... r = Spawner() reactor.listenTCP(7080, server.Site(r)) reactor.run() This is on linux.
On Mon, 2010-11-08 at 08:53 -0500, Neal Becker wrote:
I try to daemonize twisted, but it uses 100% cpu.
The problem occurs in this code:
Why not use twistd, which does all this for you already? E.g. http://twistedmatrix.com/documents/10.1.0/core/howto/application.html or the more sophisticated http://twistedmatrix.com/documents/10.1.0/core/howto/tap.html
Itamar Turner-Trauring wrote:
On Mon, 2010-11-08 at 08:53 -0500, Neal Becker wrote:
I try to daemonize twisted, but it uses 100% cpu.
The problem occurs in this code:
Why not use twistd, which does all this for you already? E.g. http://twistedmatrix.com/documents/10.1.0/core/howto/application.html
or the more sophisticated
http://twistedmatrix.com/documents/10.1.0/core/howto/tap.html
Because it looks like I need to learn a lot more to use it. But I'll try.
On Mon, Nov 08, 2010 at 09:49:44AM -0500, Neal Becker wrote:
Itamar Turner-Trauring wrote:
On Mon, 2010-11-08 at 08:53 -0500, Neal Becker wrote:
I try to daemonize twisted, but it uses 100% cpu.
The problem occurs in this code:
Why not use twistd, which does all this for you already? E.g. http://twistedmatrix.com/documents/10.1.0/core/howto/application.html
or the more sophisticated
http://twistedmatrix.com/documents/10.1.0/core/howto/tap.html
Because it looks like I need to learn a lot more to use it. But I'll try.
It's not difficult or long at all, at least you could read this[1] thread of few days ago. [1] http://twistedmatrix.com/pipermail/twisted-python/2010-November/023114.html m. -- La vera terra dei barbari non è quella che non ha mai conosciuto l'arte, ma quella che, disseminata di capolavori, non sa nè apprezzarli nè conservarli. -- Marcel Proust
I seem to have things working OK with twistd. My app is a batch scheduling system that listens for commands via xmlrpc. It is very simple at this point, and I'm attaching it. I would welcome any suggestions for improvement, since I'm a newb to twisted, and a lot of this was guesswork. task.py is a seperate file because otherwise pickle doesn't work. t6.tac is the server, and a toy client is client4.py.
On Mon, Nov 08, 2010 at 01:50:38PM -0500, Neal Becker wrote:
I seem to have things working OK with twistd.
My app is a batch scheduling system that listens for commands via xmlrpc. It is very simple at this point, and I'm attaching it. I would welcome any suggestions for improvement, since I'm a newb to twisted, and a lot of this was guesswork.
task.py is a seperate file because otherwise pickle doesn't work. t6.tac is the server, and a toy client is client4.py.
If I could you give a suggestion on how to structure the server side of your program: The ``Task`` class in a separate file, this is fine, clients could import the file without the need to import the non useful, for the client, server code. All the server logic in a file and a third file for the tap, here a possible structure: procxmlrpc/ |- client.py |- server.py `- task.py twisted/ `- plugins/ `- procxmlrpc_plugin.py Here the simplified code, all the remaining code goes into ``server.py`` from procxmlrpc.server import Spawner from twisted.web import server from twisted.application import service, internet application = service.Application("Demo application") spawn_server = server.Site(Spawner()) service = internet.TCPServer(7080, spawn_server) service.setServiceParent(application) m. -- Nessuno come me si è creato una società reale evocando delle ombre; al punto che la vita dei miei ricordi assorbe il sentimento della mia vita reale. -- René de Chateaubriand, Mémoires d'Outre-tombe
Thanks for the suggestions. I'd also like to add authentication, but it seems rather daunting. Maybe I could just restrict the xmlrpc to listen only on connection from the local host?
On Tue, Nov 09, 2010 at 06:58:05AM -0500, Neal Becker wrote:
Thanks for the suggestions.
I'd also like to add authentication, but it seems rather daunting.
You should not. Add basic or digest authentication is really easy. Take a look at Calderone's Twisted Web in 60 seconds: HTTP authentication[1], I read the series time ago and I don't remember exactly the content but the serie is really well written, the best 60 seconds (per episode) spent to learn the bases of twisted web.
Maybe I could just restrict the xmlrpc to listen only on connection from the local host?
yes, you could and twisted tap plugin system helps you on this. Just create a tap plugin with the options you need. An simple example: from twisted.python import usage from twisted.plugin import IPlugin from twisted.application import strports, service class Options(usage.Options): optParameters = [ ['port', 'p', '80', 'Listen port.', int], ['interface', 'i', None, 'Interface to which to bind.'], ] class ProcessXMLRPCServiceMaker(object): implements(service.IServiceMaker, IPlugin) tapname = "processxmlrpc" description = "Remote process control in xmlrpc" options = Options def makeService(self, options): factory = ... # XXX description = 'tcp:' + str(config['port']) if config['interface']: description += ':interface=' + config['interface'] return strports.service(description, factory) You can setup the server to listen to localhost and prepare a reverse proxy that require authentication and that accept connections from all interfaces, as example. m. -- Dalle virtù che si esigono in un domestico, l'Eccellenza Vostra conosce molti padroni degni d'esser servitori? -- Pierre Augustin Caron de Beaumarchais
participants (3)
-
Itamar Turner-Trauring -
Marco Giusti -
Neal Becker