[Distutils] buildout: several fold performance increases
me at rpatterson.net
Sat Jan 21 20:05:08 CET 2012
Marius Gedminas <marius at pov.lt> writes:
> On Sat, Jan 21, 2012 at 02:19:03AM -0800, Ross Patterson wrote:
>> I moved this patch to a branch of zc.buildout:
>> Also, buildout.dumppickedversions
>> suffers the same logging hot spot as I fixed in zc.buildout r122980 and
>> before I patched it the buildout run time was 2m13s. Can someone cut a
>> release of buildout.dumppickedversions?
> There's also buildout-versions , which is a fork/reimplementation of
> buildout.dumppickedversions, and which I use for my buildouts. I wonder
> if the same speedup could be applied to it. Perhaps not: if I disable
> buildout-versions in my 41s do-nothing buildout, the do-nothing time
> stays exactly the same.
>  http://pypi.python.org/pypi/buildout-versions
Yup, it does suffer the same:
That speedup is proportional to the number of required dists, so your
do-nothing might not suffer much. In fact, I think all of these
optimizations are at least proportional to the size of your egg cache,
IOW, the number of possible dists available on your path.
> Incidentally, looking at r122980 I wonder if you got the right revision
> number ? It seems to be a bugfix for LP#697913 (Buildout doesn't
> honor exit code from scripts).
>  http://zope3.pov.lt/trac/changeset/122980
Oops, sorry about that, it's r124059. Don't know how I got that wrong:
> How can I test your branch? It doesn't seem to be enough to add
> develop += /path/to/your/branch/checkout
> zc.buildout =
> to my existing buildout.cfg.
Not sure how. I've been patching eggs to do the 'timed' tests with
real-world buildout. Maybe you could install the checkout as a develop
egg to the python installation? Ick. Maybe you could edit bin/buildout
and adjust the path for the buildout egg? Ick.
More information about the Distutils-SIG