[Web-SIG] Pre-PEP: Python Web Container Interface v1.0

Phillip J. Eby pje at telecommunity.com
Sun Dec 7 13:53:43 EST 2003

Your comments and feedback are requested.  Thanks.

Title: Python Web Container Interface v1.0
Version: $Revision: 1.1 $
Last-Modified: $Date: 2003/12/07 13:29:50 $
Author: Phillip J. Eby <pje at telecommunity.com>
Discussions-To: Python Web-SIG <web-sig at python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 07-Dec-2003
Post-History: 07-Dec-2003


This document specifies a proposed standard interface between
web applications and web application "containers" implemented
in Python, making it possible to use a variety of application
frameworks with a single container, and to use a variety of
containers with a single application.


Python currently boasts a wide variety of web frameworks,
such as Zope, Quixote, Webware, Skunkware, PSO, and Twisted --
to name just a few [1]_.

This wide range of available choices would not be a
problem, if only it weren't necessary to choose between them!
Because few Python web frameworks can interoperate in the same
process, users are generally forced to select one and only one

Making matters worse, not all frameworks support the same
launching mechanisms.  Some use an embedded webserver, others
use CGI, FastCGI, or some custom server-to-application
protocol.  But, it is quite rare for a single framework to
provide built-in support for all of these methods.

Thus, the launching mechanism, or "container", becomes a key
constraint for users selecting a web development tool.  They
are limited to the frameworks that support (or can be made to
support) their desired runtime environment.  This can narrow
the field of choices considerably.

This is a problem for framework authors as well as for
framework users.  For their framework to become popular,
the author must at least implement container mechanisms for
the most popular runtime environments.  Although container
implementation is not complex, it is tedious and sometimes
riddled with platform-specific issues.  Being able to
separate container development from framework development
would therefore benefit framework developers as well as users.

This PEP, therefore, proposes a simple and universal interface
between web "containers" and web "applications".  The proposed
interface is 100% framework neutral, and does not favor any
development style over any other.  Conformance to this interface
will permit framework-neutral containers to be developed,
independently of any application framework, and any application
framework will then be usable with any container (potentially
subject to certain environmental issues such as threading

Finally, the interface also makes it potentially possible
to combine the use of multiple web framework tools in a single
application container.

Specification Overview

A "container" is a mechanism for executing Python code in
response to a request made on a web server.  The mechanism
by which this occurs is specific to the container.  For example, a CGI
container would use the Common Gateway Interface, while a mod_python
container would use Apache's internal API.

An "application" is a Python object that does useful work in response
to a request made on a Web server.  A container invokes an application
by calling its ``runCGI`` method, whose signature is defined as
follows (the ``self`` argument is omitted for clarity.)::

     def runCGI(input,output,errors,environ):

In other words, an application calls
``app.runCGI(input,output,errors,environ)`` to invoke the application.
The ``runCGI`` method should read from ``input``, if required, and
write its response to ``output``, using the ``environ`` dictionary
to obtain other information about the request.  Error messages or log
output may be written to ``errors``.  The return value of ``runCGI``
is ignored by the container.  The contents and format of ``input``,
``output``, and ``environ`` are defined by the Common Gateway
Interface [2]_.

The application object *must* support repeated calls to
``runCGI``, as virtually all containers will make such repeated
requests.  Containers *should* trap and log exceptions raised by
applications, and *may* continue to execute, or attempt to shut down
gracefully.  Applications *should* avoid allowing exceptions to
escape their ``runCGI`` method, since the precise effect of this is

Thread support, or lack thereof, is also container-dependent.
Containers that can run multiple requests in parallel, *should* also
provide the option of running an application in a single-threaded
fashion, so that applications or frameworks that are not thread-safe
may still be used.

This specification does not define how a container selects or
obtains an application to invoke.  These and other configuration
options are highly container-specific matters.  It is expected that
container authors will document how to configure the container to
execute a particular application object, and with what options (such
as threading options, if applicable).

Framework authors, on the other hand, will document how to create an
application object that wraps their framework's functionality.  The
user, who has chosen both the container and the application framework,
must connect the two together.  However, since both the framework and
the container now have a common interface, this should now be merely
a mechanical matter, rather than a significant engineering effort.

Specification Details

The ``input``, ``output``, and ``errors`` objects supplied to the
``runCGI`` method must be "file-like" objects, while the ``environ``
object *must* be a Python dictionary.  The ``runCGI`` method is
allowed to modify the dictionary in-place, making it easier for
authors to create simple "routing" components that forward ``runCGI``
calls to other components.

The rationale for requiring a dictionary is to maximize portability
between containers.  The alternative would be to define here some
subset of a dictionary's methods as being the standard and portable
interface.  In practice, however, most containers will probably want
to use a simple dictionary anyway, and some frameworks may end up
relying upon the fact that most containers do this.  So, in the
interest of a simple specification, and because there is little need
for a custom type here anyway, a Python dictionary is mandatory for
communicating the CGI environment.

The "file-like" objects are another matter, though.  There is much
more diversity between containers as to how these "file-like objects"
are likely to be implemented.  They may be pipes, or sockets, or
buffered asynchronous communication objects of some kind.  Therefore,
we must define the following subset of file methods that containers
are required to provide, and urge framework authors to use these,
and only these methods:

===================  =====================  ========
Method               Files                  Notes
===================  =====================  ========
``close()``          All
``read(size)``       ``input``
``readline()``       ``input``              1
``readlines(hint)``  ``input``              2
``__iter__()``       ``input``
``flush()``          ``output``,``errors``  3
``write(str)``       ``output``,``errors``
``writelines(seq)``  ``output``,``errors``
===================  =====================  ========

The semantics of each method are as documented in the Python Library
Reference, except for these notes as listed in the table above:

1. The optional "size" argument to ``readline()`` is not supported, as
    it may be complex for container authors to implement, and is not
    often used in practice.

2. Note that the ``hint`` argument to ``readlines()`` is optional for
    both caller and implementer.  The application author is free not
    to supply it, and the container author is free to ignore it.

3. Since ``output`` and ``errors`` may not be rewound, a container is
    free to forward write operations immediately, without buffering.
    In this case, the ``flush()`` method may be a no-op.  Portable
    applications, however, cannot assume that output is unbuffered
    or that ``flush()`` is a no-op.  They must call ``flush()`` if they
    need to ensure that output has in fact been written.

    Luckily, the use of ``output.flush()`` is only an issue for
    applications performing "server push" operations, since closing
    ``output`` will also flush it.  Applications writing logs or other
    output to ``errors``, however, may wish to perform a flush after
    each complete item is output, to minimize intermingling of data
    from multiple processes writing to the same log.

The methods listed in the table above *must* be supported by all
containers conforming to this specification.  Applications conforming
to this specification *must not* use any other methods or attributes
of the ``input``, ``output``, or ``errors`` objects.

Implementation and Application Notes

Proofs-of-concept of this specification are currently available in the
PEAK application framework [3]_.  PEAK includes a CGI container and
two FastCGI containers, as well as a sample non-framework application,
and a ``peak.web`` framework application.  Together, these components
demonstrate the ability to mix and match containers and
applications/frameworks by way of the interface specified here.
(Note: the containers and applications were implemented prior to the
creation of this specification, and so should not be taken as examples
of conforming implementations at this time.)

It is expected that future versions of Python will include updated
versions of current "containers" so that they can support this
interface.  For example, the Python standard library now contains
various web server implementations, and these could be modified to
allow invoking application objects that conform to this specification.

Widespread adoption of this specification would also make it possible
to implement simple "router" applications that forward ``runCGI``
calls to other application objects, using information in the
``environ`` to determine the recipient.

Because the CGI environment variables include both URL path
information and cookies, such "router" components could be very
sophisticated, if desired.  And, they would potentially allow more
than one framework to be used in the same application, permitting
Python developers to take the best from all possible worlds.

For load balancing and remote processing, it would also be possible
to write "bridge" applications, that forward a ``runCGI`` call over
a network.  Or, to add CGI capability to a Python webserver, one might
write a bridge that simply invoked another process in response to
``runCGI``.  Such bridges would again be usable in any container
conforming to this specification.


.. [1] The Python Wiki "Web Programming" topic

.. [2] The Common Gateway Interface Specification

.. [3] PEAK: The Python Enterprise Application Kit


This document has been placed in the public domain.

    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70

More information about the Web-SIG mailing list