[Distutils] Packaging buildout-based project for release

[Jim Fulton]

On Wed, May 28, 2014 at 1:58 PM, Travis Jensen <travis.jensen at oracle.com> wrote:
> Hi, all,
>
> I'm hoping for some advice. I've got a Django web app that I've used
> buildout to build. It includes effectively three source trees (a Celery task
> tree, a set of re-usable Django database models, and the actual Django web
> application project. Yes, these could all be different repos, and if they
> should be, I will make them such, it just makes early development much
> easier. :)  Each of them has their own setup.py since they each have their
> own deployment story (celery tasks on worker systems, shared models just
> about anywhere, and the web app on the web tier). buildout's
> collective.recipe.template and collective.recipe.cmd are also being used to
> install node modules (lessc, coffeescript, and yuglify).
>
> So, one option for deploying is "git clone myproj.git && cd myproj &&
> buildout".  I'm not a fan for a variety of reasons: it is *slow*; it is not
> necessarily deterministic (even with version pinning on our own simple
> index, somebody could replace a file); and it requires additional software
> to be installed on production systems that I'd rather not be installed. An
> ideal world would solve all of them. but I could live with the (unlikely)
> non-determinism.

An option you should consider is doing this in a Docker container and
deploying with Docker.  (We don't do this now due to some Docker
networking limitations that appear to have gone away lately.)

> I'm really happy with the automation of the development environment that
> buildout gives. The question is whether I can take that and turn it into a
> deployable system or if I should be looking somewhere else.  An ideal world
> would be three RPMs (and a "meta"-RPM that installed all three) with all the
> Python and Node dependencies built in.

I wrote zc.sourcerelease to help solve this problem.

Here's a script that we use to build RPMs from buildouts in git:

  https://gist.github.com/jimfulton/6629791

Some notes:

- The script is run from a git checkout.
- The buildout-source-release script needs to be provided by the buildout.
- You need to provide a RPM spec file. It can be pretty generic.

Here's an example project that uses it:

   https://bitbucket.org/zc/zrs-rpm/src

Note the buildout.cfg and spec file.

Also note that my philosophy is that RPMs should contain
software, not configuration.  We configure processes and
such separately from building RPMs.  We use buildout
to build run-time configurations. :)

Of course, Docker, potentially, makes this a *lot* simpler.  You can
do most or all of your configuration at build time.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton