[Web-SIG] A Python Web Application Package and Format

[Ian Bicking]

On Sun, Apr 10, 2011 at 6:40 PM, Alice Bevan–McGregor
<alice at gothcandy.com>wrote:

> On 2011-04-10 16:25:21 -0700, James Mills said:
>
>  +1 too. I would however like to see this idea developed in a generic
>> and useable way. ie: No zope/twisted deps or making it fit around
>> Django :)
>> Ideally it should be useable by the most basic (plain old WSGI).
>>
>
> The following are the collected ideas of myself and a few other users in
> the WebCore chat room:
>
>        https://gist.github.com/911991
>
> Being generic (i.e. using WSGI under-the-hood) and allowing generic port
> assignments for other (non-web) networked applications is a design goal.
>

There's a significant danger that you'll be creating a configuration
management tool at that point, not simply a web application description.
 The escape valve in Silver Lining for these sort of things is services,
which can kind of implement anything, and presumably ad hoc services could
be allowed for.


> The aversion to packaged zips is not entirely understandable to us; in this
> case, a packaged copy of the application is produced via a setup.py command,
> though in theory one could develop with that model and just zip everything
> up in the end by hand.
>

You create a build process as part of the deployment (and development and
everything else), which I think is a bad idea.  My model does not use
setup.py as the basis for the process (you could build a tool that uses
setup.py, but it would be more a development methodology than a part of the
packaging).

Also lots of libraries don't work when zipped, and an application is
typically an aggregate of many libraries, so zipping everything just adds a
step that probably has to be undone later.  If a deploy process uses zip
file that's fine, but adding zipping to deployment processes that don't care
for zip files is needless overhead.  A directory of files is the most
general case.  It's also something a developer can manipulate, so you don't
get a mismatch between developers of applications and people deploying
applications -- they can use the exact same system and format.

Silver Lining seems to require too much in the way of hacking (modifying
> .pth files, etc) to be reasonable.
>

The pattern that it implements is fairly simple, and in several models you
have to lay things out somewhat manually.  I think some more convention and
tool support (e.g., in pip) would be helpful.

Though there are quite a few details, the result is more reliable, stable,
and easier to audit than anything based on a build process (which any use of
"dependencies" would require -- there are *no* dependencies in a Silver
Lining package, only the files that are *part* of the package).

Some notes from your link:

- There seems to be both the description of a format, and a program based on
that format, but it's not entirely clear where the boundary is.  I think
it's useful to think in terms of a format and a reference implementation of
particular tools that use that format (development management tools, like
installing into the format; deployment tools; testing tools; local serving
tools; etc).
- In Silver Lining I felt no need at all for shared libraries.  Some disk
space can be saved with clever management (hard links), but only when it's
entirely clear that it's just an optimization.  Adding a concept like
"server-packages" adds a lot of operational complexity and room for bugs
without any real advantages.
- I avoided exposing the concept of daemonization because it's not really an
application concern; or at least it certainly is not appropriate for a WSGI
application.  There are other applications that might need this, mostly
because they have no standard protocol equivalent to WSGI, but a generic
container is almost certain to be of higher quality and better situated to
its environment than a generic daemon.  (PID files, ugh)  At least
supervisord I think has a better representation of how to express daemon
configuration, but still I'm not a big fan of exposing this until it really
feels necessary.
- All dependencies are always version-sensitive; I think it's delusional
that people think otherwise.  Build the tooling to manage that process
(e.g., finding and testing newer versions), not the deployment.
- I try to avoid error conditions in the deployment, which is a big part of
not having any build process involved, as build processes are a source of
constant errors -- you can do a stage deployment, then five minutes later do
a production deployment, and if you have a build process there is a
significant chance that the two won't match.

  Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20110410/39fb1c3b/attachment.html>