[Distutils] virtualenv and buildout integration ?
gary.poster at canonical.com
Tue Sep 1 15:59:47 CEST 2009
On Aug 31, 2009, at 8:29 AM, Chris Withers wrote:
> Gary Poster wrote:
>>> Indeed. Sounds great. However, what do you do if the system
>>> installed libraries are the wrong very (eg: the buildout specifies
>>> a particular version, and the one in site-packages isn't it)?
>> Using a whitelist in your .cfg,
> What does this mean/do/look like?
First, to repeat: Jim has not yet had a chance to review (and now he's
waiting for me to do a merge with trunk, and I'm also using that to
test the branch out on the upcoming Ubuntu release because I got a bug
report), and this may change. That said...
In the simplest case:
That means you don't allow any eggs from site packages. So, this
means that if one of your direct or indirect dependencies is docutils,
and docutils is installed as an egg in your system, when the eggs for
your buildout are assembled, the system egg will be ignored, and
buildout will go on the hunt for one someplace else. The typical
result is that the egg that it finds will be earlier in the site path,
so it has precedence in scripts and so on.
Meanwhile, dependencies that are not part of your dependencies (like,
say, pyopenssl) are still available from your standard site-packages.
Alternatively, consider this.
That is, you only allow the lxml egg from site packages. All the rest
will be ignored, as per the description above.
Here are the docs:
Sometimes you need or want to control what eggs from site-
used. The allowed-eggs-from-site-packages option allows you to
whitelist of project names that may be included from site-
can use globs to specify the value. It defaults to a single
value of '*',
indicating that any package may come from site-packages.
Here's a usage example::
This option interacts with the ``include-site-packages`` option
If ``include-site-packages`` is true, then
``allowed-eggs-from-site-packages`` filters what eggs from site-
may be chosen. Therefore, if ``allowed-eggs-from-site-packages``
empty list, then no eggs from site-packages are chosen, but site-
will still be included at the end of path lists.
If ``include-site-packages`` is false, the value of
``allowed-eggs-from-site-packages`` is irrelevant.
The same pattern holds true for the ``include-site-packages-for-
option, except only the bin/buildout script is affected by that
>> some or all of the eggs in site-packages are simply ignored when
>> buildout assembles dependencies (even if they are or claim to be
>> the right versions); and then the eggs that buildout chooses have
>> precedence over site-packages (they come before site-packages in
> Okay, but what if I don't know (or more accurately: don't want to
> have to care ;-) ) what's in site-packages? What happens if there's
> no whitelist and either buildout can't figure out what version of
> the package is installed or it's the wrong version?
Simplest answer: if you are not using any new features, this usually
fails, as before. I'm loathe to significantly change the default
behavior in 1.x release, and someone may be doing virtualenv/buildout
dances that rely on the current behavior with site-packages.
The next simplest answer is to use the new "include-site-packages =
false" option. That's a tiny bit like using a virtualenv for your
buildout, in that you are using the system executable but a clean site-
The last answer is to leave "include-site-packages = true" (the
default), and use the "allowed-eggs-from-site-packages" whitelist I
describe above. We currently use an empty whitelist, so we get bits
like pyopenssl from the system, but buildout manages all of the
application eggs. It's a compromise that is working well for us so far.
More information about the Distutils-SIG