[Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev)

[David Cournapeau]

P.J. Eby wrote:
> At 06:04 PM 7/9/2009 +0900, David Cournapeau wrote:
>> I too would like to see this kind of system, and have been thinking
>> quite a bit about it. Looks like I am not the only one.
>
> For reference, here are some previous posts on this topic, suggesting
> some requirements and approaches:
>
> http://mail.python.org/pipermail/distutils-sig/2008-October/010288.html
> http://mail.python.org/pipermail/distutils-sig/2008-October/010343.html
>

Thanks, I was looking for this discussion and could not find it.
Concerning your PEP proposition:

1: what's wrong with distutils. I have written a lengthy blog post about
this

http://cournape.wordpress.com/2009/04/01/python-packaging-a-few-observations-cabal-for-a-solution

but this is from the POV of a numpy/scipy developer, that it someone
with build needs beyond what most people will ever need - so this may
not appeal much to the whole community ? I can work on making this part
(distutils limitation) more formal if you think that would be useful,
but it does not address every issue. I have spent enough hours in
distutils and numpy.distutils to fill several PEP on distutils
annoyances, though :)

I think it is important that people who come from different communities
participate to this: people who work on web development have problems
which are mostly orthogonal to mine, but I get the impression that most
people on distutils-sig fall in this category.

2: I have no idea what wsgi is, so cannot comment on that

3: I agree with every goal stated there, but that would a PEP by itself,
no ?

4: Non-goal: I agree it should not be more complicated than it is now
with distutils.  And something which requires code changes outside
setup* would not only be unproductive, but really weird. One of my pet
project, numscons, shows that it is at least possible to do complicated
things outside distutils while involving no code change. Numscons is a
build tool which uses scons to build numpy and scipy; it works quite
well alongside the currently default tool based on distutils. You can
still build a numpy egg but using numscons to do the build part, you can
still use the develop command of setuptools, etc... but with scons
goodies (dependencies tracking, parallel builds, relatively painless
compiler flags customization, etc...). Both build infrastructures live
alongside without any problem (interop with distutils was a real PITA,
though).

5: I am not sure I understand why this is important. But then, as a non
native speaker, I am not in the best position anyway.

6: Do you mean you do not want to specify the metadata format for this
PEP, or that it should not be specified at all ?

7: See below

8: this should follow from 1 to 7, hopefully.


>
> If we really want setup.py to go away, the starting point *can't* be
> replacing it with something else, otherwise all of setup.py's
> requirements will infect it.

I am not sure I understand why. The way I have been thinking about this
was totally independent of distutils.

> The BUILDS concept I proposed is a way of separating "build" features
> from "install" features in such a way that neither a standard build
> system *nor* a standard installation system are required in order to
> achieve interoperability.

I think this part is uncontroversial, at least within people who want to
get rid of distutils :) I want the format to be powerful enough so that
it can be used to drive scons/waf, but obviously, a standard
implementation should be available without any such dependencies
(reusing distutils, at least at first). And of course, build and install
have to be as independent as possible. Distutils commands, their tight
coupling and imperative nature is by far the most annoying feature for
me. That's the source of much of distutils inflexibility.

> In BUILDS, I suggested that instead offering a distutils command to
> generate the metadata is a more implementable idea, with improved
> backward compatibility.

I think both can be useful. But generating the metadata from setup.py
cannot be the only solution: I want to tag my data so that packages
become FHS-compatible, I want to tag doc, generated sources vs templated
sources, etc... I understand that those things are unneeded in most
packages, though.

My main point was that the design for a purely static metadata which
support almost everything support by distutils already exists, has been
used for several years. One of the big mistake in distutils was the NIH.

David