[C++-sig] Python, Boost.Python, and SCons

[Ralf W. Grosse-Kunstleve]

--- Chad Austin <caustin at gmail.com> wrote:
> In my experience, the dependency scan for boost.python hasn't been
> that bad...

If you just build libboost_python.so and maybe a couple of extensions it is not
bad at all... but see below.

> but maybe that's because my primary machine is a dual
> xeon 2.8.

I have the same machine here, with RH8.

> p.s.  Have you tried always using the --implicit-cache option?  That
> usually works wonders, and very rarely results in an incorrect build.

Here are the timings if I configure our build system to compile "everything"
(maybe 150k lines + Boost) and do a "no-op" build (everything up to date) with
the full implicit dependency scan enabled, and all files on a local disk:

% time libtbx.scons .
scons: Reading SConscript files ...
Compiling distributed version of phaser
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
57.200u 6.450s 1:03.74 99.8%    0+0k 0+0io 1394pf+0w
% time libtbx.scons --implicit-cache .
scons: Reading SConscript files ...
Compiling distributed version of phaser
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
57.600u 6.480s 1:04.21 99.7%    0+0k 0+0io 1394pf+0w

And here the same but with the implicit dependency scan disabled for the Boost
subdirectory, using the CPPFLAGS trick:

% time libtbx.scons .
scons: Reading SConscript files ...
libtbx.scons: implicit dependency scan disabled for directory
/net/worm/scratch1/rwgk/dist/boost
Compiling distributed version of phaser
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
15.540u 1.340s 0:17.41 96.9%    0+0k 0+0io 1402pf+0w
% time libtbx.scons --implicit-cache .
scons: Reading SConscript files ...
libtbx.scons: implicit dependency scan disabled for directory
/net/worm/scratch1/rwgk/dist/boost
Compiling distributed version of phaser
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
15.350u 1.330s 0:16.88 98.8%    0+0k 0+0io 1402pf+0w

Four times faster! Cool?

--implicit-cache doesn't seem to do anything for me. Am I using this option
incorrectly?

> I remember your patch, but I don't remember why we disagreed with it...

My initial patch/suggestion was to introduce a global registry that the scanner
uses to decide where to prune the search for implicit dependencies. The
registry had switches to turn the scan on/off.

> I could see them buying into your patch if you
> were persistent enough and had a strong enough argument.  I don't
> remember how you implemented it, but maybe a good way would be to use
> an environment variable [SCAN_CPP_IGNORE_PATH?] that the CScanner uses
> to filter the directories it scans it.

That sounds like a great idea! Why didn't I think of that? :-) It is not quite
as global as I had in mind initially; one still has to be careful to define the
paths in all Environment instances. However, our usage pattern has evolved over
time is to use env.Copy() anyway to get common settings from a basic
Environment instance. I.e. your idea appears to be a perfect fit. I'd just have
to add the SCAN_CPP_IGNORE_PATH to our basic environment.

Do you have an idea how much is involved in adding the SCAN_CPP_IGNORE_PATH
feature? Would it be a local patch in the scanner, or does the general
framework need to be patched in several places to provide the necessary
support?

Ralf



		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail