[Distutils] buildout.dumppickedversions and buildout_versions

Jim Fulton jim at zope.com
Thu Jan 3 16:39:11 CET 2013

On Thu, Jan 3, 2013 at 8:03 AM, Marius Gedminas <marius at pov.lt> wrote:
> On Thu, Jan 03, 2013 at 07:35:20AM -0500, Jim Fulton wrote:
>> On Sat, Dec 15, 2012 at 11:56 AM, Chris Withers <chris at simplistix.co.uk> wrote:
>> > Yeah, the stuff that buildout.dumppickedversions and buildout_versions
>> > provide should just be in the core.
>> >
>> > Jim, if I worked up a pull request would you merge it?
>> Maybe :)  I tried to look at buildout.dumppickedversions and
>> buildout_versions and I couldn't quite tell what they were for or how
>> they were different.  Are they different?  Or are they 2 versions of
>> the same thing?
> buildout-versions is a clone of buildout.dumppickedversions with a
> rather better choice of name.  I don't remember Chris's rationale for
> rewriting it instead of contributing to buildout.dumppickedversions.
> The problem is this: people tend to release new package versions to
> PyPI, and they tend to sometimes break backwards compatibility
> (intentionally or unintentionally), and then your existing
> buildouts start to fail.
> Buildout has version pinning to combat this, but you have to manage
> version pins by hand.  It's easy to miss a dependency or two when you
> add a new package to your buildout, or upgrade to a newer version that
> introduces an indirect dependency.

If you run buildout in verbose mode (-v) then you get output that's easy to
grep, sort, uniq and cut into new versions, but I agree that something easier
would be better.

> Buildout-versions solves the problem of missing version pins in two
> ways: it can warn you when your buildout uses packages without a version
> pin, and it can automatically update your version pins with those
> packages.
> It's enough to add
>   [buildout]
>   extensions = buildout-versions
> to your buildout.cfg to get a warning if any packages that are installed
> by buildout do not have a version pin.  I imagine it could be a buildout
> core feature named, e.g., 'warn-about-picked-versions = true' (a
> non-fatal version of 'allow-picked-versions = false').
> If you also add
>   buildout_versions_file = versions.cfg
> it'll append new version pins to versions.cfg with the versions that
> are picked while you run bin/buildout.  It's not smart -- you have to
> ensure the file you're writing to ends with a [versions] section, and
> that your mail buildout.cfg specifies `extends = versions.cfg` and that
> you have `[buildout] versions = versions` in either of the config files.
> You can append directly to buildout.cfg if you're careful enough to keep
> the [versions] section at the bottom, but I prefer a separate
> versions.cfg file to reduce clutter and make it easier to grep.
> As far as I can tell, buildout.dumppickedversions always overwrites the
> versions.cfg file, while buildout-versions only appends new version
> pins.  The append-only behaviour makes it easier to add meaningful
> comments like "# zc.somepackage 1.3.7 fixes this important bug about X"
> above version pins when you update them manually.
> This could be a builtin feature called, e.g., 'append-picked-versions-to
> = versions.cfg'.

Well, it sounds like this could be improved, especially based on our
experience to

I'm going to give this some thought and start another thread to discuss ideas.


Jim Fulton
Jerky is better than bacon! http://zo.pe/Kqm

More information about the Distutils-SIG mailing list