[Web-SIG] WSGI and Configuration

[Ian Bicking]

Phillip J. Eby wrote:
> At 05:03 PM 11/12/04 -0600, Ian Bicking wrote:
> 
>> Has anyone thought about what configuration might look like in a WSGI 
>> context?  I'd like to set up some interface where configuration can be 
>> shared between frameworks.  It's also important to me that 
>> configuration can be local in some fashion, not process- or 
>> installation-wide.  For instance, override some configuration values 
>> for a specific URL hierarchy.
> 
> 
> I don't know if this is what you're talking about, but the most basic 
> form of configuration that every WSGI server/gateway *should* support is 
> simple key-value pairs in 'environ'.  For example, a Zope X3 WSGI 
> application might want a 'ZOPE_SITE_CONFIG' environ variable to tell it 
> where its configuration file is.  In the simplest case for monolithic 
> servers, this can be set as an OS-level environment variable.  For more 
> complex servers, this can be server configuration-driven.  (E.g. Apache 
> configuration files can set environment variables per path.)
> 
> So, the simplest approach for apps is to receive this sort of 
> per-instance deployment configuration is via an 'environ' variable, and 
> server/gateway implementers are urged to allow per-instance settings of 
> these variables.

In this model, each dispatching middleware should have a way to indicate 
environmental variables that should be added to each request.  This 
would be something like wsgikit.urlparser, but also 
twisted_wsgi.WSGIResource, and potentially others.  Or a little piece of 
middleware like:

def environ_changer(application, environ_updates):
     def environ_application(environ, start_response):
         environ.update(environ_updates)
         return application(environ, start_response)

Though these little middlewares can get out of hand.

>> Maybe the most obvious way would be something like 
>> environ['config.dict'], which would be a two-level dictionary 
>> (sections and keys), reminiscent of a .ini file.  It might not be an 
>> actual dictionary, to more easily allow configurations to be layered 
>> and unlayered.
>>
>> But that's not very sophisticated, so I'm curious if anyone has any 
>> thoughts?
> 
> 
> Heh.  It's more sophisticated than what I'm suggesting, but I think 
> maybe we have different use cases here.  I'm seeking to support the 
> diverse existing configuration formats that applications and frameworks 
> "in the wild" already have, in a way that leans towards putting that 
> configuration in the server setup, while minimizing the scripting needed 
> for deployment.

I can certainly see that need as well, though you can kind of do that in 
an ad hoc way, with stuff like environ.get('ZOPE_SITE_CONF', 
os.environ.get('ZOPE_SITE_CONF')).

But with WSGIKit I'm not entirely trying to support Webware seemlessly; 
there's a bunch of crufty configuration and file layout parts that I'd 
rather get rid of.  And I'm trying to keep it Webware-neutral; both so 
other frameworks can work under the middleware I'm making, and so 
Webware can work under other middleware that works like mine does.  So 
I'm hoping for concensus of some sort.

> Actually, I think there are now starting to be enough deployment 
> examples to maybe add a "Deployment Recommendations" section to the 
> PEP.  It seems to me that the best practice at this point is:
> 
> * Application developer should supply their application as an importable 
> object, and specify what configuration variables it requires.

Sounds good.  "Importable" is a pain.  I think most web frameworks have 
to deal with the annoyances of importing arbitrary files, which is crufy 
in Python and has all sorts of annoying corner cases.  Maybe we can 
solve that more cleanly while we're at it?  Maybe as part of wsgiref, 
maybe pulling out the importing stuff you have in PEAK. 
wsgikit.urlparser.load_module has one implementation, but I know it's 
sub-optimal.  In some models of URL resolution, like Apache, there's no 
relation of files to sys.path or any importable name.  Webware just 
creates fake names in that case.  In any case relative imports are all 
funky.  Anyway, I'd love to have a solid solution to those issues.  But 
that's kind of an aside.

> * Servers/gateways should support specifying either a source file+name, 
> or module+name, to obtain the application
> 
> * Servers/gateways should support specifying name+value string pairs to 
> configure the aforementioned application.  Each application instance 
> should be able to have its own configuration, even if the actual 
> application callable is the same object.

Definitely need this, and this is part of what I'm thinking of for 
configuration.  In WSGIKit I edited Peter's Twisted server a bit to make 
it runnable as a script.  But I had to hardcode webkit into it, since 
webkit isn't functional without at least minimal configuration 
(indicating a base path).

That's actually not too bad, because wsgikit.webkit.wsgiwebkit.webkit 
(ack, bad name...) is actually fairly Webware-neutral as well, so 
hardcoding it in doesn't make this Webware-specific.


Well, thinking further, this is where I'd rather not expose certain 
parts to the typical user.  I kind of like that webkit is hard coded 
into the Twisted server I distribute with WSGIKit.  That's one less 
thing that users have to think about.  If I wasn't keeping a copy of 
twisted_wsgi, I'd probably create a small script that plugged them 
together, but I'd want to do that instead of just giving instructions on 
how to use the command line.  So providing a standard command-line API 
maybe isn't that interesting.

Really I want to insulate users from WSGI; I don't think they should 
know about it unless they want to work on the framework itself.  I want 
really minimal instructions, like:

* dump your files in a directory (Webware servets, Spyce pages, whatever).
* Run this server, pointing it to aforementioned directory.

Voila!  Oh, and pass in some configuration file, which hopefully is 
general enough that it can encompass Webware, Spyce, and all the other 
high-level frameworks, and the low-level pieces that come along for the 
ride.

> Alternatively, I wonder if perhaps there should be an "application 
> setup" importable, that gets passed the configuration, and returns the 
> application?  This would be useful for applications that want to do some 
> initial setup, and don't want to have to write a bunch of code to manage 
> configuration caching.

In this case, I think the person can create a script that invokes both 
server and sets up the application.  I guess.

I'm actively changing my mind on what I want, so if I'm inconsistent...

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org