[Distutils] buildout/setuptools slow as it scans the whole project dir

[Reinout van Rees]

Reinout van Rees schreef op 15-04-15 om 17:01:
>
> So.... it almost seems as if there's no solution to this?
>
> Or can someone give a hint on os.walk performance relative to VMs?
Bingo! I think I've found the real problem: symlinks between filesystems.

The project is on the host file system and is mounted via vmware/vagrant 
inside the VM as /vagrant.
So my project is on /vagrant/projectname.

I have buildout set up with a download and egg cache, in 
~/.buildout/downloads and ~/.buildout/eggs respectively.

The omelette recipy makes symlinks in 
/vagrant/projectname/parts/omelette/ for all the eggs, pointing at 
~/.buildout/eggs/eggname-version/

So: the symlink is FROM the mounted host filesystem at /vagrant to the 
VM's own ubuntu filesystem.
And apparently symlinks from such a filesystem to another aren't effective.



As a test, I've set the egg-cache directory to /vagrant/.buildout/eggs/
And yes: buildout performance is great again!


I'll try to write it up a bit more eloquently and clearer later on. 
Dinner is calling now.
(Let me indulge in a quote by my brother Maurits, not unknown on this 
mailinglist: "When dinner is calling, you need a rifle.")

Reinout

-- 
Reinout van Rees                          http://reinout.vanrees.org/
reinout at vanrees.org                   http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"