[Distutils] buildout/setuptools slow as it scans the whole project dir

[Reinout van Rees]

Jim Fulton schreef op 15-04-15 om 16:14:
> I wonder if the culprit is _dir_hash in zc.buildout.buildlout.
>
> Buildout reinstalls a part if it has changed.  It considers a part to have
> changed if it's arguments have changed or it's recipe has changed.
> If a recipe is a develop egg, then it computes a hash for the recipe
> based on it's
> package contents.
>
> A quick thing to try might be to hack _dir_hash to be a noop (e.g. add
> ``return 42`` at
> the top and see if it makes the delay go away.)
No, that's not it. I tried the quick hack and it is still slow.

It should be somewhere in setuptools itself, as buildout executes a temp 
copy of the setup.py by calling
/usr/bin/python /tmp/tmpEMi6Xe -v develop -mxN -d 
/vagrant/sso/develop-eggs/tmpfmeAuobuild

The "/tmp/tmpEMi6Xe" is a copy of the setup.py, though I must check, and 
the directory at the end is presumably a symlink to the project directory.



Setuptools seems to run all "egg_info.writers" entry points it can find 
in the piece of code where the slowness occurs. So any of the entry 
points could be the culprit. And perhaps even an entry point outside of 
setuptools, due to the way entry points work.

I'll try to debug further.


Reinout

-- 
Reinout van Rees                          http://reinout.vanrees.org/
reinout at vanrees.org                   http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"