[Web-SIG] [Proposal] "website" and first-level conf (was: more comments on Paste Deploy)

[Ian Bicking]

Chad Whitacre wrote:
>> >> >    2. I'm not clear on how Paste Deploy's abstractions map to the
>> >> >       filesystem. What does my website root look like?
>> >>
>> >> The way I have generally configured websites like this is like:
>> >>
>> >>    [composite:main]
>> >>    use = egg:Paste#urlmap
>> >>    / = config:root.ini
>> >>    /blog = config:superblog.ini
>> >
>> > Right, that's the configuration, but where is "egg:Paste#urlmap" on
>> > the filesystem? Are the three ini files alone in some directory? Where
>> > is paste? Where is SuperBlog? Where is the rest of the site? I find it
>> > easier to start with the filesystem and then move up into
>> > object/config abstractions.
>>
>> You just have to understand what egg:Paste#urlmap is, probably from some
>> documentation.  Admittedly that's boilerplate in the eyes of most people
>> who use it.  It's there explicitly because Paste Deploy doesn't build
>> *any* WSGI anything into it, it only composes pieces, one of the most
>> common being urlmap.  You can see docs for it with "paster points
>> paste.composite_factory urlmap", though I now notice I haven't written
>> any docs for it (bad of me), and that is hardly a simple command line.
>> I would certainly want to build a command-line help/browser (and
>> probably web one too) as part of a rewrite of the system.
>>
>> The three ini files do go in the same directory, though of course you
>> could do config:superblog/app.ini or something like that if you wanted
>> to set it up differently.  It's a relative filename, relative to the
>> file where it is given.
>>
>> The applications themselves are eggs.  You install them however you want
>> to install them (of course I'd strongly recommend workingenv,
>> virtual-python, or zc.buildout, but that's a separate concern).  Some
>> people have mentioned some frustration about having to build full
>> libraries with a namespace, setup.py, eggs, etc. just to use
>> applications.  But I think even pretty modest shops writing very one-off
>> apps gain a real benefit from these patterns, once you get over the
>> initial hump (and we can build tools to make the initial hump not so
>> bad, that's the point of paster create).  Anyway, here's one reply I
>> made to that request:
>> http://pythonpaste.org/archives/message/20070215.192041.1534ce27.en.html
>>
>> There's a lot of practices around library management that *has* to be
>> done, because people use libraries.  Most of this applies pretty well to
>> applications as well -- and since everyone *needs* to learn how to
>> manage their libraries, using the same mechanisms for managing
>> applications has some advantage.
>>
>> Incidentally, one change to the config format that would make it
>> possible to remove the explicit idea of "composite" apps, is to make
>> some key syntax that will instantiate the named object.  E.g.,:
>>
>>    app / = config:root.ini
>>
>> Then the keywords passed would just be {"/": <actual WSGI app>}, instead
>> of the current {"/": "config:root.ini"} (where the "config:root.ini" is
>> passed to the loader object that the composite factory gets).
> 
> Dude, my eyes are seriously glazing over. I want you to say something
> simple, like:
> 
>  $ cd /usr/local/www
>  $ workingenv.py example.com
>  ...
>  $ cd example.com
>  $ source bin/activate
>  (example.com)$ mkdir etc
> 
>  Then stick a config file in etc/ and run a simple command to start
> your website.

But you are just hand-waving over the exact part that I am describing 
("stick a config file in etc/").  What does that config file look like? 
  How do you handle different cases with it?  I cover a lot of pretty 
normal use cases up there.

> That's the kind of thing I imagine you doing (eh?), and it's also the
> thing that Aspen does. The difference is mostly in the config files.
> 
> Now, Jim: it looks like Zope still uses a Unix-y userland for
> INSTANCE_HOME, yes? So that's Paste, Pylons(?), Aspen, Zope2 and Zope3
> all using the same filesystem layout. IINM the filesystem structures
> of Django and CP/TurboGears are module-level (Bob?), so they could
> easily fit into lib/python.
> 
> If we could agree on a really simple first-line config file that
> handles basic process configuration--address, user/group, threads,
> etc.--and then points to the next layer config--be it zope.conf,
> paste.ini, apps.conf, or settings.py--then we'd be pretty far towards
> a common app server.

Part of why I push Paste Deploy is because every simpler or more 
abstract config idea could just as easily be composed as a Paste Deploy 
entry point.  That is, one can create the abstract idea of a config 
loader, but that requires all the same boiler plate that a minimal Paste 
Deploy config file has anyway.  Which is not to say someone might not 
want to write a different loader, but I don't think adding another layer 
of abstraction that's more neutral helps.

> That is to say, I think we are really discussing three increasing
> levels of cooperation:
> 
>  1) Server benchmarks and inter-op standards (Jim)
>  2) Common process management library (Bob)
>  3) Common web app server
> 
> Without discouraging the first two efforts, I'd like to champion the
> third. Here would be my proposal:
> 
> First, we define a "website" on the filesystem as a Unix-y userland
> with, at minimum, the following:
> 
>  etc/<foo>.conf
>  lib/python
> 
> Second, we adopt a simple ini-style format for <foo>.conf, which
> handles low-level process config. This file would then point to a
> second, framework-specific configuration layer.

If it's framework-specific, how do you determine what the framework is? 
   You need some kind of slug to do that, or else a separate runner. 
That also doesn't really do anything for composing multiple different 
applications that happen to use different frameworks.  Personally I find 
framework-specific configuration rather dumb, because the point of all 
this isn't to build *frameworks*, it's to build *applications*, and 
frameworks are just an implementation details of an application.

One could say that it would be better if the application shipped its own 
setup, meaning its own appctl script.  This doesn't allow very well for 
wrapping or composing applications, but it's a valid thing to provide. 
But I don't think your proposal goes in that direction.

-- 
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org