[medusa] Question: killing/shuting down asyncore servers

[Sam Rushing]

Jim Fulton writes:
> Are there any recommendations for shutting down
> asyncore-based servers cleanly from outside the
> servers?
> 
> - Can a clean signal handler be written to
> handle kill events? I tried writing a signal
> handler that closed all the entries in the socket
> map, but I have a feeling that this isn't working
> properly. One of the problems I want to solve is
> getting the socket I'm listening on to close
> cleanly so that it isn't in use when I restart
> the server.
> 
> I suspect I could make this work with some sort of
> dedicated select trigger. I think it probably wants
> to be dedicated because, if I used an existing select
> trigger, I'd need to worry about the possability that
> the signal would interrupt I/O on the select trigger.

This is why I added the ExitNow exception to asyncore recently. But
you can still have a problem with catch-all try/except clauses.
Here's some code I've been using on unix:

[excerpts]

def handle_signal (*ignore):
shutdown()

import signal
signal.signal (signal.SIGTERM, handle_signal)
signal.signal (signal.SIGHUP, handle_signal)

SHUTDOWN_PERFORMED = 0

def shutdown():
global SHUTDOWN_PERFORMED
if not SHUTDOWN_PERFORMED:
[other processing]
asyncore.socket_map.clear()
SHUTDOWN_PERFORMED = 1
raise asyncore.ExitNow

Now this generally works. There are two cases where it can get in
trouble:

1) you have a try/except that doesn't watch for ExitNow.
I try to limit my use of catch-all try/except, so it's pretty
easy to find these and fix them *if you have control of the code*.

2) Your exit handler will clear the socket map, and then raises
an exception that crawls all the way up to the poll() function.
poll() is probably in the middle of a for loop processing pending
events on other channels. Processing those events can actually
repopulate the socket map, and the normal exit condition of an
empty socket map isn't seen.

Making the pending-events stuff globally accessible (like the
socket_map itself) could fix this. But I wouldn't enjoy
explaining such code to others.

> - I suppose I could include a shutdown server, such that
> if connections were made to this server, the application 
> would shut-down cleanly. I could obviously generalize this
> to a more general control server, which might provide
> other commands. This approach is more portable, but
> requires clients more complicated than kill. :)

I've used the marshal-based rpc classes (recently added to the
distribution) for exactly this. Rather than calling a shutdown()
function directly, it's cleaner to schedule a shutdown to happen in a
couple of seconds.

-Sam