[Distutils] Improving distutils vs redesigning it (was people want CPAN)

David Cournapeau cournape at gmail.com
Thu Nov 12 15:06:11 CET 2009

On Thu, Nov 12, 2009 at 10:36 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> On Thu, Nov 12, 2009 at 1:57 PM, David Cournapeau <cournape at gmail.com> wrote:
> [...]
>> The signature does not really matter I think. Since those parameters
>> depends on user customization (through config files or command line
>> flags), it could be something like:
>> class FinalizedOptions:
>>    def get_build_dir_library(self):
>>        pass
>>    def get_build_dir_src(self):
>>        pass
>>    def get_prefix(self):
>>    ...
>> And a function like
>> def get_finalized_options():
>>    # compute the finalized options
>>    # Cache it if expensive
>>    ...
>>    return FinalizedOptions()
>> could be used to obtain those options.
> I was not referring to the finalized options of any commands, but to
> the build paths you are trying to get.

But those are linked. Build directories are customizable through
pretty much any build_command (I see at least build_clib, build_ext
and build on python 2.6).

So :

> get_build_paths(scheme, vars)

This should return different value depending on whether the in place
build is set up, or the develop command is called, etc...

> If at some point you need to look to some of its finalized options in
> another commandB, it means that
> - either commandA is doing more than it is supposed to do: it sets an
> *environement* that is reused by others
> - either commandB should be a subcommand of commandA
> - either commandA doesn't let you do what you want, and you are
> building a competiting command

Those differences do not make much sense. If distutils wants command
foo to be a subcommand of bar, and we need a different ordering, how
can we do it ?

Concerning different options from different commands, I quickly looked
at build, install and build_ext commands options (the ones from
straight distutils). Very few options concerned only the command it
belongs to. For example, the install command list is:

  --prefix            installation prefix
  --exec-prefix       (Unix only) prefix for platform-specific files
  --home              (Unix only) home directory to install under
  --user              install in user site-package
  --install-base      base installation directory (instead of --prefix or --
  --install-platbase  base installation directory for platform-specific files
                      (instead of --exec-prefix or --home)
  --root              install everything relative to this alternate root
  --install-purelib   installation directory for pure Python module
  --install-platlib   installation directory for non-pure module distributions
  --install-lib       installation directory for all module distributions
                      (overrides --install-purelib and --install-platlib)
  --install-headers   installation directory for C/C++ headers
  --install-scripts   installation directory for Python scripts
  --install-data      installation directory for data files
  --compile (-c)      compile .py to .pyc [default]
  --no-compile        don't compile .py files
  --optimize (-O)     also compile with optimization: -O1 for "python -O", -O2
                      for "python -OO", and -O0 to disable [default: -O0]
  --force (-f)        force installation (overwrite any existing files)
  --skip-build        skip rebuilding everything (for testing/debugging)
  --record            filename in which to record list of installed files

Maybe --record is not needed, but I can see a usage for all the other
ones. For example, if I have a scons builder to compile/optimize
python files, I need to know it in my scons command.

> The schemes currently, go as far as Python 2.2.


>> Ideally, it would solve a fair shair amount of issues if this was done
>> at the UI level as well, that is something like
>> python setup.py build --compiler=mingw -> use mingw compiler for every
>> compiled code
> As a matter of fact, having a "compiler" option in build, that would be reused
> by build_ext and other build_ commands, where discussed lately in an issue

When I was considering this example, I had in mind that any option
which can be customized in any build_ command should be customizable
in build. Compiler, compiler options, in-place vs non in-place, etc...
Pretty much any option.

> Why that ? How this would break someone's workflow, since all the changes
> we have discussed so far is moving code out of commands to have new APIs
> (and have those commands call it), and modifying some command options.

One of the problem is dealing with inconsistencies between
subcommands. Modifying their behavior has many corner cases. One of
the big win with numscons was that build was handled by only one
command, so the inconsistencies disappear.

> Now there's another pattern emerging in this discussion : paths
> options like --prefix can obviously
> be shared by several commands.
> So, based on the change that makes it possible to get some install and
> build schemes,
> what about having global options that will be used to built a global
> environment dictionary containing paths:
> $ python setup.py --var1 --var2 --var3 cmd1 cmd2 cmd3
> then, var1, and var2  etc.. are put in a "vars" dict, that is passed
> to get_paths(scheme, vars).

As mentioned above, please keep in mind that many of those options see
their behavior modified (directly or indirectly) through commands
options. Think about inplace option, develop command, etc...


More information about the Distutils-SIG mailing list