[Web-SIG] multi-threaded or multi-process wsgi apps

[Graham Dumpleton]

On 27/11/2007, Robert Brewer <fumanchu at aminus.org> wrote:
> Chris Withers wrote:
> > Right, I'm curious as to how wsgi applications end up being
> > multi-threaded or multi-process and if they are, how they share
> > resources such as databases and configuration.
> >
> > There's a couple of reasons I'm asking...
> >
> > The first was something Chris McDonough said about one ofthe issues
> > they're having with the repoze project: when using something like
> > mod_wsgi, it's the first person to hit each thread that takes the hit
> > of loading the configuration and opening up the zodb. Opening the
> ZODB,
> > in particular, can take a lot of time. How should repoze be structured
> > such that all the threads load their config and open their databases
> > when apache is restarted rather than when each thread is first hit?
>
> If I were coding it, repoze would use a database connection pool that is
> populated at (sub)process startup.

The issue with running under Apache, whether it be mod_wsgi or
mod_python, is that the server itself doesn't necessarily know
anything about what applications may actually need to be loaded. This
is because both support the concept of sticking the file representing
the entry point to the application in some file system directory. The
first that the server knows about the application is when a URL
arrives which maps to that application file.

Thus, in the general case one cant have pre initialisation at
(sub)process startup. To have pre initialisation means providing an
explicit means of configuring the server to say that it is possible
that some application may get invoked through a URL and so just in
case it should preload the application.

Because it involves changing main server configuration, obviously can
only be used as an option where you control the actual web server.
There would be no way you could use such an option if you were just a
user in a paid shared web hosting environment. In that case you can't
avoid doing delayed initialisation at time that first request arrives.

This is the big difference between Apache and pure Python hosting
solutions. That is that Apache has to deal with potential shared
hosting issues. Pure Python hosting solutions would probably always be
under direct control of the user and be only running their own code.

Graham