Website using Python--what's best architecture? CGI?

Paul Boddie paul at
Fri Mar 21 14:03:21 CET 2003

Andy Robinson <andy at> wrote in message news:<vnck7vonlvfmfl0o2h87b7u5nsr0luqkds at>...
> >
> >So, don't CGI ! Programming a CGI highly dynamic site is a hassle, even with
> >the help of a templating or HTML generator package (like HTMLGen and
> >others).
> I disagree.  Several years of making reporting 'add-ons' to other
> people's app servers, which are usually CGI scripts, has taught
> me something very interesting.

I'm guessing that your "add-ons" are CGI scripts, as opposed to the
application servers of those other people. Clarification is necessary
here because CGI scripts can act as the "gateway" to application
servers - eg. Webware's WebKit.cgi.

> Our apps consistently respond several times faster than the big Websphere or
> ASP systems that talk to them.  The other systems have usually been written by
> large numbers of smart programmers over time and tend to get big and slow as
> they add more features, or maybe just because they have a heck of a lot of
> code to execute in their main loops..

I can understand this, and I can see how it's interesting to use as
much of Apache's infrastructure (for example) as possible to do
application server type activities.

> I also think that how the process is initiated (which is what CGI is
> about) has absolutely nothing to do with the framework you use or
> don't use.  You can make a simple, clean dispatch mechanism
> and a few utilities to preserve state with tokens in very few lines
> of code.  And if your web frame work is <1000 lines of Python
> code, which is plenty, there won't be a big startup overhead.

Indeed, there's a lot to be said for making use of the capabilities of
the operating system. Start-up of Python processes doesn't necessarily
involve heavy I/O, and as far as things like session storage and
persistence are concerned, modern filesystems certainly have something
to offer in competition to relational database systems. Moreover, the
traditional separate process model can be quite compelling with Python
and the issues the current interpreter has with distributing threads
across multiple CPUs - this being an element in recent discussions of
Python's performance in Web applications.

> Please measure for yourself the actual time taken to execute a
> 'hello world' type page which does one query and formats
> the output on the different frameworks you are considering.

And then contribute the results to the WebProgramming section of the
PythonInfo Wiki. :-)


> >In addition, playing with CGI means creating a new DB connector for each
> >request when (most) Web application servers handle open DB connections pools
> >for you.
> True, but...please measure the connection overhead.


> OTOH I was told once that Oracle could take seconds and allocate
> megabytes of RAM for each connection.

Yes, it apparently depends on the database system concerned. Previous
involvement in Oracle-based applications tended to result in avoidance
of "straight CGI" for this very reason.

> This would be a really fun topic for a Python conference:  
> set a simple task and actually test and benchmark all those
> web frameworks.  ;-)

Indeed. However, I think that performance obsessions obscure the
reason for the existence of most of these frameworks: to provide
high-level APIs to the information and "transactions" involved in Web
applications. One could argue that the diversity of the high-level
APIs currently employed in the various frameworks is a major obstacle
to portability of applications (ie. right now you write your
application for either Zope, Webware, SkunkWeb, etc. and have to
commit to that choice), but one could also argue that one also has the
freedom to choose a paradigm that feels comfortable.

Another possibility, of course, is to undermine the need to employ an
application server in various cases by enhancing the 'cgi' module in
the standard library, but the last time such improvements were
proposed, the response was less than lukewarm.


More information about the Python-list mailing list