[Distutils] buildout/setuptools slow as it scans the whole project dir
Reinout van Rees
reinout at vanrees.org
Wed Apr 15 19:02:51 CEST 2015
Reinout van Rees schreef op 15-04-15 om 17:01:
> So.... it almost seems as if there's no solution to this?
> Or can someone give a hint on os.walk performance relative to VMs?
Bingo! I think I've found the real problem: symlinks between filesystems.
The project is on the host file system and is mounted via vmware/vagrant
inside the VM as /vagrant.
So my project is on /vagrant/projectname.
I have buildout set up with a download and egg cache, in
~/.buildout/downloads and ~/.buildout/eggs respectively.
The omelette recipy makes symlinks in
/vagrant/projectname/parts/omelette/ for all the eggs, pointing at
So: the symlink is FROM the mounted host filesystem at /vagrant to the
VM's own ubuntu filesystem.
And apparently symlinks from such a filesystem to another aren't effective.
As a test, I've set the egg-cache directory to /vagrant/.buildout/eggs/
And yes: buildout performance is great again!
I'll try to write it up a bit more eloquently and clearer later on.
Dinner is calling now.
(Let me indulge in a quote by my brother Maurits, not unknown on this
mailinglist: "When dinner is calling, you need a rifle.")
Reinout van Rees http://reinout.vanrees.org/
reinout at vanrees.org http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"
More information about the Distutils-SIG