Simple discussion of python cgi approaches?

[Steve Holden]

Kylotan wrote:

> [Apologies if anyone sees this twice - at first I attempted to post it
> via my ISP's news server, but my ISP is notorious for poor news
> handling and I don't see it on Google Groups as having got through
> yet.]
> 
> I've been writing a CGI app with Python 2.3 and Apache. It's ok, but a
> bit slow, and I assume a significant part of this is the process
> start-up time, among other things. So I'm interested in alternatives
> or optimisations. The problem is, there doesn't seem to be the kind of
> information out there that I want. There seem to be plenty of places
> that throw a list of URLs at you (eg.
> http://www.python.org/cgi-bin/moinmoin/WebProgramming,
> http://phaseit.net/claird/comp.lang.python/web_python.html), and a few
> sites that will give the absolute basics of CGI programming itself
> (which I don't need).
> 
There are many differeing opinions on this subject, so maybe you'll just 
have to accept that c.l.py responses reflect the prejudices of their 
authors. They will at least usually be better-informed than random web 
sites, one might hope.

> I'm particularly confused by a few people advocating things like
> Xitami + LRWP for being very simple, yet it can't possibly be simpler
> than just dropping a cgi script into a directory, as I do with Apache,
> and the docs on LRWP
> (http://www.imatix.com/html/xitami/index12.htm#TOC114) make it look
> like I'd have to do significant rewriting of the scripts. It describes
> itself as being like FastCGI which, if true, doesn't exactly endear
> FastCGI to me either.
> 
Well, you said you are looking for "alternatives or optimaizations", so 
it would appear that you feel you are approaching the limits of your 
simple approach. It's a very unsual solution that will be simpler, 
faster and cheaper than the "obvious" one, though such solutions aren't 
unheard of, even in the web world.

> All I want is a short description on the technologies that very
> closely resemble standard CGI, what they have to offer me, and what
> sort of alterations I would have to make to my installation or code in
> order to use these. Does such a comparison exist already? If not, is
> anybody able to comment on their own experiences with Python CGI apps?
> 
Well, let me briefly summaries the advantages of Xitami/LRWP and 
mod_python, two approaches with which I am familiar, as compared with 
good 'ole vanilla CGI.

Both of these systems have the advantage that once your web 
functionality is stable (i.e. the programs aren't changing each time you 
want the run) you avoid the overhead of reloading both the interpreter 
and the source code of your page generators (CGI scripts) for each page.

mod_python effectively works by integrating the interpreter into Apache 
in such a way that each directory can, if necessary, have a separate 
interpreter instance (to avoid namespace clashes between different 
applications). Modules are loaded on-demand, and stay resident in the 
Apache process unless changed. Sometimes there are problems if an 
indirectly-imported module changes, since mod_python only checks those 
it imports itself.

The intergration is very tight, and you can write Apache handlers of all 
kinds in Python. I have experimented myself with writing request 
handlers, with results briefly reported at PyCON this year. It's a good 
general-purpose approach for uni-processor solutions.

Xitami is slightly different: the server is deliberately written as a 
lightweight asynchronous request handler which is capable of queueing 
requests until a service routine becomes available. The LRWP linkage 
isn't actually as complicated as it might seem, and the only major 
difference is that a service process must register with the server to 
handle a class of URLs before it receives requests.

This gives you two immediate benefits: firstly, since network sockets 
are used to communicate between the web server and the service process, 
the web server can act as a conduit to external functionality very 
easily - it's trivial to build a remote LRWP, since the only difference 
between a local and a remote LRWP is the fact that the remote one will 
use an external IP address for the web server rather than "localhost". 
Second, since the web server is perfectly happy to accept multiple 
registrations for a given class of URLs, and since each registration can 
come from a different host, the asynchronous nature of the Xitami system 
allows you to achieve potentially great scalability.

I described this approach in "Python Web Programming", since it seemed 
to me to offer the easiest way to build web-based Python applications 
which could potentially scale to hundreds of processors. The really nice 
part is that you can add more LRWPs to any URL class that needs more 
power, dynamically and without trauma.

I even built a system on Windows that had a URL to start up LRWPs on 
demand, and to shut them down, under client control. It's very flexible.

Clearly anything that has performance benefits over CGI is likely to be 
a little more difficult to get running but believe me, the extra effort 
isn't that great when you consider the rewards.

HTH

regards
  Steve