[Python-Dev] addressing distutils inability to track file dependencies

[Jeremy Hylton]

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

  SM> I still don't quite understand what everyone's aversion to make
  SM> is (yes, I realize it's not platform-independent, but then
  SM> neither are C compilers or linkers and we manage to live with
  SM> that), but I will let that slide.

You didn't let it slide.  You brought it up again.  Many people have
offered many reasons not to use make.  You haven't offered any
rebuttal to their arguments, which comes across as rather cavalier.

  SM> Instead, I see a potentially different approach.  Write an scons
  SM> build file (typically named SConstruct) and deliver that in the
  SM> Modules directory.

I don't care much about the Modules directory actually.  I want this
for third-party extensions that use distutils for distribution,
particularly for my own third-party extensions :-).  

It sounds like you're proposing to drop distutils in favor of SCons,
but not saying so explicitly.  Is that right?  If so, we'd need to
strong case for dumping distutils than automatic dependency tracking.
If that isn't right, I don't understand how SCons and distutils meet
in the middle.  Would extension writers need to learn distutils and
SCons?

It seems like the primary benefit of SCons is that it does the
dependency analysis for us, while only gcc and MSVC seem to offer
something similar as a compiler builtin.  Since those two compilers
cover all the platforms I ever use, it isn't something that interests
me a lot.

  SM> The benefits as I see them are

  SM> * SCons implements portable automatic dependency analysis
  SM>       already

That's good.

  SM> * Dependencies are based upon file checksums instead of
  SM>       timestamps (worthwhile in highly networked development
  SM>       environments)

That's good, too, although we could do the same for distutils.  Not
too much work, but not my first priority.

  SM> * Clearer separation between build/install and edit/compile/test
  SM>       types of tasks.

I don't know what you mean.  I use the current Python make file for
both tasks, and haven't had much problem.

  SM> I was able to create a simple SConstruct file over the weekend
  SM> that builds many of the extension modules.  I stalled a bit on
  SM> library/include file discovery, but hopefully that barrier will
  SM> be passed soon.

That's cool.

  SM> I realize in the short-term there are also several disadvantages
  SM> to this idea:

  SM> * There will initially be a lot of overlap between setup.py and
  SM>       SCons.

Won't there be a lot of overlap for all time unless Python adopts
SCons as the one true way to build extension modules?  It's not like
setup.py is going to be replaced. 

  SM> * SCons doesn't yet implement a VPATH-like capability so the
  SM>       source and build directories can't easily be separated.
  SM>       One is in the works though, planned for initial release in
  SM>       0.09.  The current version is 0.07.

Absolute requirement for me :-(.  I've got three CVS checkouts of
Python and probably 10 total build directories that I use on a regular
basis -- normal builds, debug builds, profiled builds, etc.

Jeremy