[Distutils] moving things forward

[Greg Ewing]

Paul Moore wrote:

> Do you expect that
> projects ... should (somehow) contain simplified instructions on how to
> build the various C/Fortran extensions supplied in the bundle as
> source code?

Essentially, yes. I'm not sure how achievable it would
be, but ideally that's what I'd like.

> would the user need to put all the various details of his system into
> a configuration file somewhere (and update that file whenever he
> installs a new library, updates his compiler, or whatever)?

On a unix system, most of the time they would all be in
well-known locations. If I install something in an unusual
place or in an unusual way, I'm going to have to tell
something about it anyway. I don't see how an executable
setup file provided by the package author is going to
magically figure out all the weird stuff I've done.

I don't know if there are conventions for such things on
Windows. I suspect not, in which case manual input is
going to be needed one way or another.

> How would
> this cope with (for example) projects on Windows that *have* to be
> compiled with mingw, and not with MSVC?

An option to specify which compiler to use would be a fairly
obvious thing that the config format should provide. As
would a way to include different option settings for
different platforms. Combine them in the obvious way.

> This sounds to me more like an attempt to replace the "build" part of
> distutils/setuptools with a more declarative system.

Not all of it, only the parts that strictly have to be
performed as part of the build step of the install
process, to use your terminology. That's a fairly
restricted subset of the whole problem of compiling
software.

(Ideally, I would make it *very* restricted, such as
only compiling C code (and possibly C++, not sure).
For anything else you would have to write a C wrapper
or use a tool that generates C. But that might be
a step too far.)

Could a purely declarative config file be flexible
enough to handle this? I don't know. The distutils
API is actually pretty declarative already if you
use it in a straightforward way.

The trouble is that there's nothing to prevent, or
even discourage, writing a setup.py that works in
non-straightforward ways. I've seen some pretty
convoluted setup.py code that worked great when
your system happened to match one of the possibilites
that the author had in mind, but was very difficult
to deal with otherwise. If I'd been able to just
set a few compile options and library paths directly,
it would have been a lot easier.

There's also nothing to prevent the setup.py from
doing things that don't strictly need to be done at
install time. Running Pyrex to generate .c files
from .pyx files is one that I've encountered.
(I encouraged that one myself by including a distutils
extension for it, which I later decided had been a
mistake.)

> the proposals currently on the table
> would allow you to ask pip to use that build system rather than
> setuptools:
 >
> [buildsystem]
> requires=gregsbuild
> build_command=gregsbuild --make-wheel

That's nice, but it wouldn't help me when I encounter
a package that *hadn't* been set up to use gregsbuild. :-(

-- 
Greg