[Distutils] buildout prepends eggs to sys.path, performance issues
Jim Fulton
jim at zope.com
Tue Apr 29 15:49:02 CEST 2014
On Tue, Apr 29, 2014 at 7:33 AM, Jan Van Hees <jakkevanhees at gmail.com> wrote:
> Hi list,
>
>
>
> A bit of context: we’re running buildout 2.0.1, building zope and plone
> applications, zope 2.13.10, combined with plone.recipe.zope2instance. This
> question only impacts the performance of the startup of an instance, the the
> performance of a running instance. But starting an instance is something
> developpers do quite often on a days development.
>
>
>
> While debugging some instance startup performance issues, I came across the
> following.
>
> The buildout Scripts prepends all the eggs to the system path, before the
> python path.
>
>
>
> In our setup this causes quite some delays, because imports from standard
> python modules, also try to find that module in every eggs directory, before
> it can find in in the default python location (because the eggs are
> prepended).
...
You make a good performance argument.
> As far as I’ve always understood, the default procedure working with paths
> should be to append, unless you have a good reason. The good reason in this
> case, that I see, could be that you want to prepend certain packages and
> that way make sure you use your version instead of what’s present in
> site-packages.
Yes, exactly.
> Now my expectation would be that that is a fairly limited set
> of packages that need to be prepended.
But buildout can't know what they are.
> If there are many, options like
> virtualenv exist to avoid taking site-packages at all (that’s what we do
> btw).
Buildout doesn't require virtualenv.
I've always prepended appliocation paths for predicability.
I'm not interesated in changing this behavior of buildout, at
least not as a default.
I'll give this some thought, but some possible ideas:
- Provide an option to install the eggs for a part in a single
directory.
collective.recipe.omlette does this, although it doesn't seem to be
used for run-time package lookup afaict from the docs. You might
try using this.
- Provide an option to append, rather than prepend eggs.
Jim
--
Jim Fulton
http://www.linkedin.com/in/jimfulton
More information about the Distutils-SIG
mailing list