[Web-SIG] multi-threaded or multi-process wsgi apps

Graham Dumpleton graham.dumpleton at gmail.com
Thu Nov 29 01:02:16 CET 2007


> >>> The second is a problem I see an app I'm working on heading towards.
> >>> The app has web-alterable configuration, so in a multi-threaded and
> >>> particular multi-process environment, I need some way to get the
> >>> other threads or processes to re-read their configuration when it has
> >>> changed.
> >>
> >> In Paste/Pylons the configuration is stored in the environment (which
> >> is per-request), and put into a threadlocal object for access.
> >
> > Again, how about across processes?
> > And if the configuration changes once the app is up and running, how do
> > you propogate changes to all the other app processes?
>
> Generally you'd want to kill all the worker processes and start over.
> If you have a reasonable way to do that (and I think mod_wsgi would give
> you a reasonable way to do that), restarting the process is always
> cleanest.  I believe even something like Apache's "graceful"
> restart/reload just restarts the server, while letting existing requests
> finish.

Yes, with Apache one can use 'graceful'. That will cause applications
running under either mod_wsgi embedded mode or daemon mode to be
restarted.

For daemon mode, in mod_wsgi 2.0 there is also 'Process' reload
mechanism as an option. Just touching the main WSGI script file entry
point will cause just the processes for that daemon process group to
be restarted on next request, thereby avoiding a restart of the whole
Apache web server and any other hosted applications.

Thus, change your config file, whether it be actual Python code or an
ini file and touch the main WSGI script file for that application.
Upon the next request against that application it will detect WSGI
script file has changed and will do the restart.

If you wanted to force the restart without waiting for next request to
arrive, then you can also just send a SIGINT to all processes in that
daemon process group. The trick is knowing which processes they are,
because if still run as Apache user, hard to tell them from normal
Apace child processes since they are just a fork of main Apache
process. If mod_wsgi told to change them to run as different user,
somewhat easier.

This sort of process reload mechanism based on script file change is
also available in things like mod_fastcgi. Based on comments made by
various people, not sure how reliable it is in mod_fastcgi though.

For other options on how to trigger automatic reloading of application
process in mod_wsgi when code and/or config changes see:

  http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

Latter example in this based on similar stuff that Ian has done, just
customised to the setting of mod_wsgi and that one can rely on
mod_wsgi to automatically restart a daemon process when killed off.
Thus bit different to where this sort of idea is used elsewhere to
trigger in process reload of modules on top of existing modules, which
will not always work because of dependencies between modules.

Graham


More information about the Web-SIG mailing list