[Distutils] [buildout] weird behaviour with system-installed packages and cross-buildout contamination

[Chris Withers]

Hi Jim,

This has had me tearing my hair out somewhat. If you want to play along 
at home, here's the code:

https://github.com/Simplistix/testfixtures/tree/py3k

Background: I'm on Mac OS X, Python2.6 is compiled from source, 
Python2.7 is an EPD install, so has *lot* packages in it (this is a good 
thing, and means I don't have to compile numpy and friends).

So, porting testfixtures to Python 3, I got all the tests passing on 
3.3, but now Jenkins tells me there are problems on 2.7. Okay, bootstrap 
there:

buzzkill:testfixtures chris$ python2.7 bootstrap.py
Creating directory '/Users/chris/LocalGIT/testfixtures/bin'.
Creating directory '/Users/chris/LocalGIT/testfixtures/parts'.
Creating directory '/Users/chris/LocalGIT/testfixtures/develop-eggs'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/buildout'.
buzzkill:testfixtures chris$ bin/buildout
Develop: '/Users/chris/LocalGIT/testfixtures/.'
Installing py.
...
Generated script '/Users/chris/LocalGIT/testfixtures/bin/tox'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/virtualenv'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/virtualenv-2.7'.
Generated interpreter '/Users/chris/LocalGIT/testfixtures/bin/py'.
Installing docs.
...

But wait, I don't have a bin/nosetests. Hmm, okay, well, am I right in 
guessing this is to do with Buildout 2's desire not to try and protect 
you form system packages? If so, not problem with it not installing a 
new nose, but surely it's a bug that I don't end up with bin/nosetests?

Oh well, my 2.6 will show the same problem and is clean, so lets 
bootstrap with that:

buzzkill:testfixtures chris$ python2.6 bootstrap.py
Downloading 
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.34.tar.gz
Extracting in /var/folders/m6/tsd59qsj7pd_lldh4mhwh6kh0000gn/T/tmpWmSNdS
Now working in 
/var/folders/m6/tsd59qsj7pd_lldh4mhwh6kh0000gn/T/tmpWmSNdS/distribute-0.6.34
Building a Distribute egg in 
/var/folders/m6/tsd59qsj7pd_lldh4mhwh6kh0000gn/T/tmpG09Ey9
/var/folders/m6/tsd59qsj7pd_lldh4mhwh6kh0000gn/T/tmpG09Ey9/distribute-0.6.34-py2.6.egg
Generated script '/Users/chris/LocalGIT/testfixtures/bin/buildout'.
buzzkill:testfixtures chris$ bin/buildout
Upgraded:
   distribute version 0.6.24;
restarting.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/buildout'.
Develop: '/Users/chris/LocalGIT/testfixtures/.'
Uninstalling docs.
Uninstalling py.
Installing py.
...
Generated script '/Users/chris/LocalGIT/testfixtures/bin/tox'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/virtualenv'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/virtualenv-2.6'.
Generated interpreter '/Users/chris/LocalGIT/testfixtures/bin/py'.
Installing docs.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/rst2'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/pkginfo'.
Generated interpreter '/Users/chris/LocalGIT/testfixtures/bin/docpy'.
buzzkill:testfixtures chris$ bin/nos
-bash: bin/nos: No such file or directory

Wah? Still no nose. Now things get really bizarre. The only way I can 
get nose back, with *any* python install now, is:

buzzkill:testfixtures chris$ git clean -fxd
Removing .installed.cfg
Removing bin/
Removing develop-eggs/
Removing parts/
Removing testfixtures.egg-info/
Removing testfixtures/__init__.pyc
Removing testfixtures/comparison.pyc
Removing testfixtures/compat.pyc

Now if I buildout with a non-nosed base python, I get nose back:

buzzkill:testfixtures chris$ python2.6 bootstrap.py
...
buzzkill:testfixtures chris$ bin/buildout
Develop: '/Users/chris/LocalGIT/testfixtures/.'
Installing py.
...
Generated script '/Users/chris/LocalGIT/testfixtures/bin/nosetests-2.6'.
Generated script '/Users/chris/LocalGIT/testfixtures/bin/nosetests'.
...

Which elements of the above should I file bugs for?

Suffice to say, buildout 2 has not been a very pleasant experience this 
evening :-(

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk