On Tue, 08 Feb 2005 14:36:04 -0500, Itamar Shtull-Trauring <itamar@itamarst.org> wrote:

On Tue, 2005-02-08 at 11:17 -0800, snacktime wrote:

...

I have a server that uses threads, and when run standalone or via twistd as an application I have to send it a KILL signal if it has already answered at least one request. Or in some cases if I just send it an INT it will stop if it's currently handling a lot of requests at the time I send it the signal. Seems like there is something I am not doing here that I should be when using threads?

The system will stop when all threads are finished, which happens when they finish their current task. Since that may be blocking (possibly indefinitely) the program may never exit cleanly. This is a general problem with blocking calls and not threads, and is not Twisted specific.

The only place I call threads is through deferToThread, and the server still hangs even after all the threads called with deferToThread have run and their callbacks processed. What other threads could there be that are blocking?

On Tue, 08 Feb 2005 15:06:18 -0500, Itamar Shtull-Trauring > > The only place I call threads is through deferToThread, and the server

...

still hangs even after all the threads called with deferToThread have run and their callbacks processed. What other threads could there be that are blocking?

In that case Twisted should exit correctly.

Here is the code. ProcessTransaction is the class (not shown here) that contains the blocking method Do(). After playing with it some more I found something interesting. Say I send a number of requests with the client, then hit control-C on the server. The server won't shutdown, but on the very next connection from the client the server drops the connection immediately before the ssl handshake is completed and shuts down. It seems like some sort of event loop where the INT signal isn't getting processed until the next event happens, which in this case would be the connection. class OT(Protocol): def dataReceived(self, data): """As soon as any data is received, process it in a thread.""" reactor.callLater(0, self.Start,data) def PrintData(self,data): self.transport.write("%s\r\n" % data) self.transport.loseConnection() def Start(self,data): c = ProcessTransaction() # Blocking function called here in thread d = threads.deferToThread(c.Do,data) d.addCallback(self.PrintData) d.addErrback(log.err) application = service.Application("otransact") OTService = service.IServiceCollection(application) OTfactory = Factory() OTfactory.protocol = OT OTServer = internet.SSLServer(8000, OTfactory,ServerContextFactory()) OTServer.setServiceParent(OTService)

Complete examples are preferable. If the problem cannot be reproduced (that is to say, the source copied into a file, the file run with Python, and the undesirable behavior observed as a result), it is much more time consuming to attempt to provide a useful response.

Ok here is a test server/client. Server: -------------------------- from twisted.application import service,internet from twisted.internet.protocol import Protocol, Factory from twisted.internet import defer, reactor from twisted.python import threadable from twisted.internet import threads from twisted.python import log threadable.init(1) import sys from OpenSSL import SSL class ServerContextFactory: def getContext(self): """Create an SSL context. This is a sample implementation that loads a certificate from a file called 'server.pem'.""" ctx = SSL.Context(SSL.SSLv23_METHOD) ctx.use_certificate_file('server.pem') ctx.use_privatekey_file('server.pem') return ctx class ProcessTransaction: def Do(self,data): return 'Done' ### Protocol Implementation class OT(Protocol): def dataReceived(self, data): """As soon as any data is received, process it in a thread.""" reactor.callLater(0, self.Start,data) def PrintData(self,data): self.transport.write("%s\r\n" % data) self.transport.loseConnection() def Start(self,data): c = ProcessTransaction() d = threads.deferToThread(c.Do,data) d.addCallback(self.PrintData) d.addErrback(log.err) application = service.Application("otransact") OTService = service.IServiceCollection(application) OTfactory = Factory() OTfactory.protocol = OT OTServer = internet.SSLServer(8000, OTfactory,ServerContextFactory()) OTServer.setServiceParent(OTService) Client: ----------------------- from OpenSSL import SSL import sys import time from twisted.internet.protocol import ClientFactory from twisted.protocols.basic import LineReceiver from twisted.internet import ssl, reactor class EchoClient(LineReceiver): end="GoodBye" def connectionMade(self): self.sendLine("Testing") def connectionLost(self, reason): print 'connection lost (protocol)' reactor.stop() def lineReceived(self, line): print "receive:", line self.transport.loseConnection() class EchoClientFactory(ClientFactory): protocol = EchoClient def clientConnectionFailed(self, connector, reason): print 'connection failed:', reason.getErrorMessage() reactor.stop() def clientConnectionLost(self, connector, reason): print 'connection lost:', reason.getErrorMessage() reactor.stop() def main(): factory = EchoClientFactory() reactor.connectSSL('localhost', 8000, factory, ssl.ClientContextFactory()) reactor.run() if __name__ == '__main__': main()

Thanks. What platform are you running this on? On Debian (Linux 2.6.9, libc 2.3.2), the server exits as expected when I send it a SIGINT or SIGTERM.

Jp

Freebsd 5.3 release-p2. If it works on debian I suspect it's either a freebsd problem or less likely a python problem. Freebsd still has some issues with threads as far as I know. Chris