[Distutils] zc.buildout - order of extends processing

Martin Aspeli optilude+lists at gmail.com
Fri Feb 26 11:16:44 CET 2010


Hi,

In my quest to make re-usable buildout components, I've started using a 
pattern like this:

In a server-specific buildout file (e.g. prod-web-master.cfg) I have e.g.:

[buildout]
extends =
     buildout.d/base.cfg
     buildout.d/zope.cfg
     buildout.d/relstorage.cfg
     buildout.d/haproxy.cfg

[ports]
haproxy = 8100

[eggs]
main = my.package

...

Each file in buildout.cfg adds additional functionality, usually by 
defining new parts and doing e.g.

    parts += haproxy-build haproxy-config

    [haproxy-build]
    recipe = zc.recipe.cmmi
    ...

Sometimes, they also extend some shared variables, e.g.

    [eggs]
    main += RelStorage

With this type of setup, the order in which configuration is loaded and 
processed becomes important, in particular for += and -= type operations 
and straightforward overrides. For example, I may have a default 
${ports:haproxy} defined in haproxy.cfg, but overridden in 
prod-web-slave.cfg because the slave node wants it to run on a different 
port, say.

I can't find much documentation about how this is supposed to work. I've 
had situations where I've been surprised by the order in which things 
are processed, in particular if I extend one file that in turn extends 
another. So, a few questions

  - Is the order well-defined? e.g. is it always doing a depth-first or 
breath-first inclusion? Is the load order the same as the execution order?

  - Do settings in the top-level config (the one run with bin/buildout 
-c <file>) always override extended ones?

  - More generally, if A extends B and B extends C, does configuration 
in B always take precedence over configuration in C with the same 
part/option? Does a += in B always extend the options set in C?

  - If I have multiple files all extending a common base, e.g. A extends 
B and C, and both B and C extend a common D, when does the common base 
(D) get processed?

Cheers,
Martin

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book



More information about the Distutils-SIG mailing list