[Web-SIG] A Python Web Application Package and Format

[Alice Bevan–McGregor]

On 2011-04-11 00:53:02 -0700, Eric Larson said:

> Hi,
> On Apr 10, 2011, at 10:29 PM, Alice Bevan–McGregor wrote:
> 
>> However, the package format I describe in that gist does include the 
>> source for the dependencies as "snapshotted" during bundling.  If your 
>> application is working in development, after snapshotting it /will/ 
>> work on sandbox or production deployments.
> 
> I wanted to chime in on this one aspect b/c I think the concept is 
> somewhat flawed. If your application is working in development and 
> "snapshot" the dependencies that is no guarantee that things will work 
> in production. The only way to say that snapshot or bundle is 
> guaranteed to work is if you snapshot the entire system and make it 
> available as a production system.

`pwaf bundle` bundles the source tarballs, effectively, of your 
application and dependencies into a single file.  Not unlike a certain 
feature of pip.

And… wait, am I the only one who uses built-from-snapshot virtual 
servers for sandbox and production deployment?  I can't be the only one 
who likes things to work as expected.

> Using a real world example, say you develop your application on OS X 
> and you deploy on Ubuntu 8.04 LTS. Right away you are dealing with two 
> different operating systems with entirely different system calls. If 
> you use something like lxml and simplejson, you have no choice but to 
> repackage or install from source on the production server.

Installing from source is what I was suggesting.  Also, Ubuntu on a 
server?  All your `linux single` (root) are belong to me.  ;^P

> While it is fair to say that generally you could avoid packages that 
> don't use C, both lxml and simplejson are rather obvious choices for 
> web development.

Except that json is built-in in 2.6 (admittedly with fewer features, 
but I've never needed the extras) and there are alternate xml parsers, 
too.

> It sounds like Ian doesn't want to have any build steps which I think 
> is a bad mantra. A build step lets you prepare things for deployment. A 
> deployment package is different than a development package and mixing 
> the two by forcing builds on the server or seems like asking for 
> trouble.

I'm having difficulty following this statement: build steps good, 
building on server bad?  So I take it you know the exact target 
architecture and have cross-compilers installed in your development 
environment?  That's not practical (or simple) at all!

> I'm not saying this is what you (Alice) are suggesting, but rather 
> pointing out that as a model, depending on virtualenv + pip's bundling 
> capabilities seems slightly flawed.

Virtualenv (or something utilizing a similar Python path 'chrooting' 
capability) and pip using the extracted "deps" as the source for 
"offline" installation actually seems quite reasonable to me.  The 
benefit of a known set of working packages (i.e. specific version 
numbers, tested in development) and the ability to compile C extensions 
in-place.  (Because sure as hell you can't reliably compile them 
before-hand if they have any form of system library dependency!)

> I think it should offer hooks for running tests, learning basic status 
> and allow simple configuration for typical sysadmin needs (logging via 
> syslog, process management, nagios checks, etc.). Instead of focusing 
> on what format that should take in terms of packages, it seems more 
> effective to spend time defining a standard means of managing WSGI apps 
> and piggyback or plain old copy some format like RPMs or dpkg.

RPMs are terrible, dpkg is terrible.  Binary package distribution, in 
general, is terrible.  I got the distinct impression at PyCon that 
binary distributable .eggs were thought of as terrible and should be 
phased out.

Also, nobody so far seems to have noticed the centralized logging 
management or deamon management lines from my notes.

> Just my .02. Again, I haven't offered code, so feel free to ignore me. 
> But I do hope that if there are others that suspect this model of 
> putting source on the server is a problem pipe up. If I were to add a 
> requirement it would be that Python web applications help system 
> administrators become more effective. That means finding consistent 
> ways of deploying apps that plays well with other languages / 
> platforms. After all, keeping a C compiler on a public server is rarely 
> a good idea.

If you could demonstrate a fool-proof way to install packages with 
system library dependencies using cross-compilation from a remote 
machine, I'm all ears.  ;)

	— Alice.