[Twisted-Python] Twisted.web signal handling
Hello. I wrote a simple application using twisted.web. One of activity of my program is to contacting with some other processes. I need some timeout to perform this activity. The structure of my program looks like this: ///////////////////////////////////////////////////////////////////////////////////// def TimeoutHandler(pid): os.kill(pid, signal.SIGUSR1) def handler(): raise ConnectionError signal.signal(signal.SIGUSR1, handler) MyPID = os.getpid() def HistosRendering: timer = threading.Timer(0.001, TimeoutHandler, [MyPID]) timer.start() try: <Some activity that need timeout> except ConnectionError: <In case of timeout> return else: timer.cancel() class Resource(resource.Resource): def render(self, request): HistosRender() resource = Resource() ///////////////////////////////////////////////////////////////////////////////////// Unfortunately, there is something wrong, cause in case of timeout I am receiving error message, the TimeoutHandler is activated (in the Timer thread), but handler() in the main thread is not activated. I am using this kind of commands to start a server:
Do you see any solution? Thanks in advance, Krzysztof Nowak
On Sat, 24 Sep 2005 14:43:09 +0200, Krzysztof Nowak <pegazik@gmail.com> wrote:
Timeouts should be managed using twisted.internet.reactor.callLater().
twistd uses SIGUSR1 for logfile rotation. Your handler is probably getting clobbered.
Depending on exactly where this code is run, this is probably the wrong PID. twistd daemonizes, which involves forking and exiting in the parent. If your code is imported before daemonization happens, the above will retrieve the PID of the parent, which will not be your PID at any time during the actual operation of your program.
Why aren't you using Twisted to handle this connection attempt? The threading.Timer/signal/raise combo is a very bad way to manage timeouts. Twisted provides much better ways. Also, there is a race condition between the end of your try suite and the timer.cancel() call. The exception could be raised after the else suite is entered but before the timer is actually cancelled.
You are blocking the entire process for the during of the render() call. This is another reason not to make whatever synchronous connection attempt you are making and instead use something Twisted provides. No other activities can occur, process-wide, while your code is running. If this is a multiuser application, *everyone* using it will experience periods of unresponsiveness when *anyone* causes blocking code to run.
The main thing to figure out is how to rewrite the operation which you wish to timeout so that it is asynchronous and easily cancellable. Then all the signal related difficulties will disappear. Jp
On 9/24/05, Jp Calderone <exarkun@divmod.com> wrote:
Timeouts should be managed using twisted.internet.reactor.callLater().
Thanks. I am not really deep in the twisted. I just want to make this application and that would be all (for now). I have rewrite my previous code in this way: ///////////////////////////////////////////////////////////////////////////////////// def Timeout(): raise ConnectionError def HistosRendering: Event = reactor.callLater(0.0001, Timeout) try: <Some activity that need timeout> except ConnectionError: <In case of timeout> return else: Event.cancel() class Resource(resource.Resource): def render(self, request): HistosRender() resource = Resource() ///////////////////////////////////////////////////////////////////////////////////// But still, it doesn't work. Timeout() is called, but exception is making whole process to fail and the except: statestment is not even started. I'm quite shure, that this timeouting-stuff should be some standard problem, but I failed to find any information in the net. So please, help me once again.
JP Calderone: Also, there is a race condition between the end of your try suite and the timer.cancel() call. The exception could be raised after the else suite is entered but before the timer is actually cancelled.
Do you have any ideas how to avoid that problem?
If adding asynchronous approach would force me to restructure whole project, it would stay for time just like this. This could be next step, but timeouting is much more important to me.... Cheers, Krzysztof Nowak
On Sat, 24 Sep 2005 14:43:09 +0200, Krzysztof Nowak <pegazik@gmail.com> wrote:
Timeouts should be managed using twisted.internet.reactor.callLater().
twistd uses SIGUSR1 for logfile rotation. Your handler is probably getting clobbered.
Depending on exactly where this code is run, this is probably the wrong PID. twistd daemonizes, which involves forking and exiting in the parent. If your code is imported before daemonization happens, the above will retrieve the PID of the parent, which will not be your PID at any time during the actual operation of your program.
Why aren't you using Twisted to handle this connection attempt? The threading.Timer/signal/raise combo is a very bad way to manage timeouts. Twisted provides much better ways. Also, there is a race condition between the end of your try suite and the timer.cancel() call. The exception could be raised after the else suite is entered but before the timer is actually cancelled.
You are blocking the entire process for the during of the render() call. This is another reason not to make whatever synchronous connection attempt you are making and instead use something Twisted provides. No other activities can occur, process-wide, while your code is running. If this is a multiuser application, *everyone* using it will experience periods of unresponsiveness when *anyone* causes blocking code to run.
The main thing to figure out is how to rewrite the operation which you wish to timeout so that it is asynchronous and easily cancellable. Then all the signal related difficulties will disappear. Jp
On 9/24/05, Jp Calderone <exarkun@divmod.com> wrote:
Timeouts should be managed using twisted.internet.reactor.callLater().
Thanks. I am not really deep in the twisted. I just want to make this application and that would be all (for now). I have rewrite my previous code in this way: ///////////////////////////////////////////////////////////////////////////////////// def Timeout(): raise ConnectionError def HistosRendering: Event = reactor.callLater(0.0001, Timeout) try: <Some activity that need timeout> except ConnectionError: <In case of timeout> return else: Event.cancel() class Resource(resource.Resource): def render(self, request): HistosRender() resource = Resource() ///////////////////////////////////////////////////////////////////////////////////// But still, it doesn't work. Timeout() is called, but exception is making whole process to fail and the except: statestment is not even started. I'm quite shure, that this timeouting-stuff should be some standard problem, but I failed to find any information in the net. So please, help me once again.
JP Calderone: Also, there is a race condition between the end of your try suite and the timer.cancel() call. The exception could be raised after the else suite is entered but before the timer is actually cancelled.
Do you have any ideas how to avoid that problem?
If adding asynchronous approach would force me to restructure whole project, it would stay for time just like this. This could be next step, but timeouting is much more important to me.... Cheers, Krzysztof Nowak
participants (2)
-
Jp Calderone -
Krzysztof Nowak