[Twisted-Python] Checking whether a server started successfully
Hi,
I am starting a server process using "twistd -oy" (don't save state; application is described by Python source). If all is well, the server is started in the background. However, if all is not well, I'd like to detect that startup failed.
The particular scenario that I'm using is testing a server using a test framework that does HTTP requests. Before the tests are run, the server is started and it tries to bind to a fixed port. If there is still an old instance of the server running, the bind will fail and any requests made will be handled by the old instance instead. That will lead to invalid test results.
I worked around it currently by waiting for 3 seconds (probably sufficient time to try the binding) and then checking if the process of which the ID stored in "twisted.pid" is still running. However, there is no guarantee that 3 seconds is always enough time for the server to have bound the port. So I'm wondering if there is a reliable and elegant way of solving this.
One approach would be to check which process has bound a particular port. I'm running this server on Linux and the "netstat" command line tool has access to the right information, but not in an easily parseable format. I guess the same info can be found in /proc somehow, but are those files guaranteed to stay compatible with newer kernels?
Another approach would be to check in advance if something is listening on that particular port, but between the time this check is done and the time the server tries to bind, some other process might have grabbed the port.
In any case, the port already being bound is just one of the reasons why the startup might fail; I'd prefer to have a way to detect startup failures for other reasons as well.
One question is when exactly "startup" is finished. For my purpose, the moment the reactor is running would be the moment I consider startup to be successful. Maybe I can write a file somewhere to explicitly signal that, but that's a bit messy (where to put it? is the server process allowed to write there? who cleans up the file?).
Maybe it would be useful to have twistd (optionally) wait until the reactor is running before returning. I don't want the test script to rely on a patched version of Twisted, so this would only be an option if it would make sense to integrate such a feature into Twisted itself.
Have you encountered this problem as well? And if so, how did you deal with it?
Bye, Maarten
On 8/21/07, Maarten ter Huurne maarten@treewalker.org wrote:
Hi,
I am starting a server process using "twistd -oy" (don't save state; application is described by Python source). If all is well, the server is started in the background. However, if all is not well, I'd like to detect that startup failed.
The particular scenario that I'm using is testing a server using a test framework that does HTTP requests. Before the tests are run, the server is started and it tries to bind to a fixed port. If there is still an old instance of the server running, the bind will fail and any requests made will be handled by the old instance instead. That will lead to invalid test results.
I worked around it currently by waiting for 3 seconds (probably sufficient time to try the binding) and then checking if the process of which the ID stored in "twisted.pid" is still running. However, there is no guarantee that 3 seconds is always enough time for the server to have bound the port. So I'm wondering if there is a reliable and elegant way of solving this.
One approach would be to check which process has bound a particular port. I'm running this server on Linux and the "netstat" command line tool has access to the right information, but not in an easily parseable format. I guess the same info can be found in /proc somehow, but are those files guaranteed to stay compatible with newer kernels?
Another approach would be to check in advance if something is listening on that particular port, but between the time this check is done and the time the server tries to bind, some other process might have grabbed the port.
In any case, the port already being bound is just one of the reasons why the startup might fail; I'd prefer to have a way to detect startup failures for other reasons as well.
One question is when exactly "startup" is finished. For my purpose, the moment the reactor is running would be the moment I consider startup to be successful. Maybe I can write a file somewhere to explicitly signal that, but that's a bit messy (where to put it? is the server process allowed to write there? who cleans up the file?).
Maybe it would be useful to have twistd (optionally) wait until the reactor is running before returning. I don't want the test script to rely on a patched version of Twisted, so this would only be an option if it would make sense to integrate such a feature into Twisted itself.
Have you encountered this problem as well? And if so, how did you deal with it?
Bye, Maarten
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Yes, I've encountered this problem before. And yes, I initially did the same thing as you, by waiting a hardcoded time. But eventually I got tired of this race condition, and that it doesn't detect when the server failed to start. What I'm doing right now for my non-unit functional testing is running a bash script which:
- make sure twistd isn't running using ps: ps aux | grep twistd | grep -v grep - start up the app in non-daemon mode, send its output to the console and to a file: twistd -noy my-app.tac | tee server.log & - poll the server.log by grep'ing for a string I print from my app when the app is completely started - launch the tests against the running server (I think I'm using mechanize right now for web client library)
This works for me in my development environment, on Linux.
Cheers, Christian
Christian Simms wrote:
- make sure twistd isn't running using ps: ps aux | grep twistd |
grep -v grep
You can save yourself a pipe by doing
$ ps aux|grep [t]wistd
That way it won't find twistd on the grep line. the [brackets] get in the way, but don't semantically change what you're looking for.
Cheers, Cliff
participants (3)
-
Christian Simms -
J. Cliff Dyer -
Maarten ter Huurne