On Sat, 3 Jul 2004 23:41:38 -0700 (PDT), Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
--- Chad Austin <caustin@gmail.com> wrote:
p.s. Have you tried always using the --implicit-cache option? That usually works wonders, and very rarely results in an incorrect build.
Here are the timings if I configure our build system to compile "everything" (maybe 150k lines + Boost) and do a "no-op" build (everything up to date) with the full implicit dependency scan enabled, and all files on a local disk:
% time libtbx.scons . [snip] 57.200u 6.450s 1:03.74 99.8% 0+0k 0+0io 1394pf+0w % time libtbx.scons --implicit-cache . [snip] 57.600u 6.480s 1:04.21 99.7% 0+0k 0+0io 1394pf+0w
Did you try running with --implicit-cache again? The first time you use it, it caches the dependency information of headers. The second time, it uses that information to speed up the build. In short, you could describe --implicit-cache as adding a test: "if signature of header has not changed, then don't scan it: use the previous headers found." That gets rid of the scanner pass (although MD5 is still run).
And here the same but with the implicit dependency scan disabled for the Boost subdirectory, using the CPPFLAGS trick:
% time libtbx.scons . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.540u 1.340s 0:17.41 96.9% 0+0k 0+0io 1402pf+0w % time libtbx.scons --implicit-cache . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.350u 1.330s 0:16.88 98.8% 0+0k 0+0io 1402pf+0w
Four times faster! Cool?
Nice. Try running with --implicit-cache a few times in a row... then we can compare it with your patch. Of course, with both, you might get an even bigger speedup. :)
--implicit-cache doesn't seem to do anything for me. Am I using this option incorrectly?
[snip]
That sounds like a great idea! Why didn't I think of that? :-) It is not quite as global as I had in mind initially; one still has to be careful to define the paths in all Environment instances. However, our usage pattern has evolved over time is to use env.Copy() anyway to get common settings from a basic Environment instance. I.e. your idea appears to be a perfect fit. I'd just have to add the SCAN_CPP_IGNORE_PATH to our basic environment.
That sounds good. I've noticed that nearly every SCons build creates an initial environment and uses env.Copy() to customize it, so it should be okay.
Do you have an idea how much is involved in adding the SCAN_CPP_IGNORE_PATH feature? Would it be a local patch in the scanner, or does the general framework need to be patched in several places to provide the necessary support?
I'm pretty sure you would only have to patch the scanner. Fortunately, the scanner code is pretty straightforward. Take a look at Scanner/__init__.py. Should be easy to implement using an environmental variable. :) Cheers, Chad