[Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev)

[David Cournapeau]

I am moving this discussion out of python-dev, as it is outside the
thread topic, but an important discussion nonetheless:

2009/7/9 Eric Smith <eric at trueblade.com <mailto:eric at trueblade.com>>:
> P.J. Eby wrote:
>>>
>>> ISTM that the problem that it solves is uninstall in the absence of
>>> the original installer.
>>
>> Or uninstall where the installer is "setup.py install", actually.
>
> I think we need to move away from "setup.py install". It's the
antithesis of
> static metadata. setup.py needs to go away.
>
> If we have a static metadata file called, say "setup.info
<http://setup.info>", then I'd like to
> see:
> "bdist_rpm setup.info <http://setup.info>" produce a .rpm
> "bdist_msi setup.info <http://setup.info>" produce a .msi
> "bdist_deb setup.info <http://setup.info>" produce a .deb
> "bdist_egg setup.info <http://setup.info>" produce a .egg
> and so on.
>
> There might be a library (and I call dibs on the name "distlib" :) that
> provides support routines to parse setup.info <http://setup.info>, but
there's no framework
> involved. And no need for a plugin system.

I too would like to see this kind of system, and have been thinking
quite a bit about it. Looks like I am not the only one. P.J. Eby
mentioned  that designing such a thing by comity is not likely to work
well. But I think we can inspire ourselves from existing designs.

One such system is the haskell build system, cabal:

http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/

Among other things:
    - It is based on a mostly static file, parsed by the cabal package.
    - Installation of 'libraries' (equivalent of our modules and
packages) and executables (scripts) is covered.
    - Debug and normal build for compiled extensions is handled,
metadata as well. Simple conditional for different platforms support.
    - Optional haskell code ala setup.py can be executed for the
complicated cases.
    - optional integration with source control systems
    - it has command lines option compatible with autotools ->
FHS-compliant packages would be easier as well.
    - cabal is integrated with a db of packages, to form hackage: it
handles installations, dependency handling (but not uninstallation it
seems). I don't have much experience with this part of the design, though.

I think it is a good and strong baseline as a starting point. Reusing
this design in a python equivalent should be much easier than starting a
system from scratch. Under this scheme, backward compatibility with
distutils could be attempted by automatically converting setup.py files
to this system, and vice et versa. The development could be gradual,
reusing some parts of distutils during the development phase.

My own motivation comes from my experience in maintaining and
'improving' distutils extensions used in numpy and scipy. I have a
pretty good idea of the flaws and internals of distutils, and care about
every platform that numpy supports (that is most if not all the ones
supported by python).

cheers,

David