[Web-SIG] WSGI deployment use case

[Ian Bicking]

Well, I thought I'd chime in with everything I'd want in a deployment 
strategy; some of this clearly is the realm of practice, not code, but 
it all fits together.  It's not necessarily the Universal Use Case, but 
I don't think it's too strange.


Here's some types of applications:

* Things I (or someone in my company) codes.

* Full application someone else creates and I use.

* Applications that are more like a service, that use inside an 
application of mine.  These are end-point applications, like a REST 
service or something.  An application like this probably appears to live 
inside my application, through some form of delegation; unless it's a 
service many applications share...?

* Middleware written by me for my internal use, maybe application-specific.

* Middleware written specifically for the framework I (or someone else) 
uses.

* General-purpose middleware that could apply to anything.


Some kinds of deployments I want to do:

* Two clients with the same application, same version (like "latest 
stable version").

* Two clients with different versions; e.g., one client hasn't paid for 
an upgrade (which might mean upgrading).

* A client with branched code, i.e., we've tweaked one instance of the 
application just for them.

* Two installations of the same application, in the same process, with 
different URLs and different configurations.  This might be something as 
small as a formmail kind of script, or a large program.

* Sometimes apps go into different processes, but often they can go into 
the same process (especially if I start using Python WSGI for the kind 
of seldom-used apps that I now use CGI for).

* I have to mount these applications at some location.  This should be 
part of the deployment configuration; both path based and domain name based.


Here's some aspects of the configuration:

* Many applications have a lot of configuration.  Much of it is "just in 
case" configuration that I'd never want to tweak.  Some of that 
configuration may be derivative of things I do want to tweak, e.g., URL 
layouts where I configure the base URL, but all the other URLs could be 
derived from that.

* What appears to be an application from the outside might be composed 
of many applications.  Maybe an app includes an external formmail app 
for a "support" link.  That app requires configuration (like smtp server).

* I'd like to configure some things globally.  Like that smtp server. 
Or an email address to send unexpected exceptions to.

* I might want to override configuration locally, like that email 
address.  I might want to augment configuration, like just add an 
address to the list, not reset the whole value.

* I'd like to install some middleware globally as well.  Like a session 
handler, perhaps.  Or authentication.  Or an exception catcher -- I'd 
like everyone to use my well-configured exception catcher.  So not only 
am I adding middleware, I might be asking that middleware be excluded 
(or should simply short-circuit itself).

* And of course, all my applications take configuration, separate from 
middleware and frameworks.

* And usually there are non-WSGI pieces that need access to the exact 
same configuration; scripts and cronjobs and whatnot.  Usually they just 
need the application configuration, but nothing related to middleware or 
the web.


I think quite a bit of this is handled well by what we're talking about, 
even if it wasn't just a little while ago; versioning for instance. 
Branches I'm a little less sure about, since version numbers are linear.

But configuration and composition of multiple independent applications 
into a single process isn't.  I don't think we can solve these 
separately, because the Hard Problem is how to handle configuration 
alongside composition.  How can I apply configuration to a set of 
applications?  How can I make exceptions?  How can an application 
consume configuration as well as delegate configuration to a 
subapplication?  The pipeline is often more like a tree, so the logic is 
a little complex.  Or, rather, there's actual *logic* in how 
configuration is applied, almost all of which are viable.

I can figure out a bunch of ad hoc and formal ways of accomplishing this 
in Paste; most of it is already possible, and entry points alone clean 
up a lot of what's there (encouraging a separation between how an 
application is invoked generally, and install-specific configuration). 
But with a more limited and declarative configuration it is harder. 
Also when configuration is pushed into factories as keyword arguments, 
instead of being pulled out of a dictionary, it is much harder -- the 
configuration becomes unhackable.



-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org