[C++-sig] Python, Boost.Python, and SCons

[Chad Austin]

On Sat, 3 Jul 2004 23:41:38 -0700 (PDT), Ralf W. Grosse-Kunstleve
<rwgk at yahoo.com> wrote:
> --- Chad Austin <caustin at gmail.com> wrote:
> > p.s.  Have you tried always using the --implicit-cache option?  That
> > usually works wonders, and very rarely results in an incorrect build.
> 
> Here are the timings if I configure our build system to compile "everything"
> (maybe 150k lines + Boost) and do a "no-op" build (everything up to date) with
> the full implicit dependency scan enabled, and all files on a local disk:
> 
> % time libtbx.scons .
[snip]
> 57.200u 6.450s 1:03.74 99.8%    0+0k 0+0io 1394pf+0w
> % time libtbx.scons --implicit-cache .
[snip]
> 57.600u 6.480s 1:04.21 99.7%    0+0k 0+0io 1394pf+0w

Did you try running with --implicit-cache again?  The first time you
use it, it caches the dependency information of headers.  The second
time, it uses that information to speed up the build.  In short, you
could describe --implicit-cache as adding a test:  "if signature of
header has not changed, then don't scan it: use the previous headers
found."  That gets rid of the scanner pass (although MD5 is still
run).

> And here the same but with the implicit dependency scan disabled for the Boost
> subdirectory, using the CPPFLAGS trick:
> 
> % time libtbx.scons .
> scons: Reading SConscript files ...
> libtbx.scons: implicit dependency scan disabled for directory
> /net/worm/scratch1/rwgk/dist/boost
> Compiling distributed version of phaser
> scons: done reading SConscript files.
> scons: Building targets ...
> scons: `.' is up to date.
> scons: done building targets.
> 15.540u 1.340s 0:17.41 96.9%    0+0k 0+0io 1402pf+0w
> % time libtbx.scons --implicit-cache .
> scons: Reading SConscript files ...
> libtbx.scons: implicit dependency scan disabled for directory
> /net/worm/scratch1/rwgk/dist/boost
> Compiling distributed version of phaser
> scons: done reading SConscript files.
> scons: Building targets ...
> scons: `.' is up to date.
> scons: done building targets.
> 15.350u 1.330s 0:16.88 98.8%    0+0k 0+0io 1402pf+0w
> 
> Four times faster! Cool?

Nice.  Try running with --implicit-cache a few times in a row...  then
we can compare it with your patch.  Of course, with both, you might
get an even bigger speedup.  :)

> --implicit-cache doesn't seem to do anything for me. Am I using this option
> incorrectly?
> 
[snip]
>
> That sounds like a great idea! Why didn't I think of that? :-) It is not quite
> as global as I had in mind initially; one still has to be careful to define the
> paths in all Environment instances. However, our usage pattern has evolved over
> time is to use env.Copy() anyway to get common settings from a basic
> Environment instance. I.e. your idea appears to be a perfect fit. I'd just have
> to add the SCAN_CPP_IGNORE_PATH to our basic environment.

That sounds good.  I've noticed that nearly every SCons build creates
an initial environment and uses env.Copy() to customize it, so it
should be okay.

> Do you have an idea how much is involved in adding the SCAN_CPP_IGNORE_PATH
> feature? Would it be a local patch in the scanner, or does the general
> framework need to be patched in several places to provide the necessary
> support?

I'm pretty sure you would only have to patch the scanner. 
Fortunately, the scanner code is pretty straightforward.  Take a look
at Scanner/__init__.py.  Should be easy to implement using an
environmental variable.  :)

Cheers,
Chad