At 03:34 PM 9/26/2008, Ian Bicking wrote:
I'm not sure I agree with removing setup.py, though I dislike how it functions currently -- that you kind of have to poke it from the side to get basic information out of it, instead of something more declarative. In some ways it's the distutils/setuptools interface that most people actually use -- pyinstall for instance only calls setup.py in subprocesses to install the package, and never touches it directly. So anything that acts fairly similar to the current setup.py's will be compatible. This is part of why I don't think the backward compatibility issue is as difficult as Guido suggests.
Though... really if pyinstall could easily detect a new source format and only setup.py with the old source format, it wouldn't be hard to support both. There does need to be room for custom code, but mostly for the build/compilation process.
The main reason to drop setup.py as the interface is to prevent people writing their own crazy command line "parsing" (e.g. probing sys.argv directly) and from trying to install stuff or do other things unconditionally in code that shouldn't be doing anything but configuration. Having a 'configure.py' or some such would be a clear break with the past in this respect. That is, a configure.py script would be there if and only if you needed dynamic metadata or build-spec generation, and it shouldn't be designed to run as a user-run script, but instead be invoked in a restricted way by build and install tools. It should also be required to be sandboxable, in the sense that it should write to no files outside the build area. It should also be well-understood that it is not to unconditionally import dependencies, nor assume that failure to import a dependency means the dependency isn't present. (And if you don't have any dynamic metadata requirements, the package metadata and build specs should be able to be defined in non-executable configuration files, with no need for a configure.py.) IOW, the main reason (IMO) to drop backward compatibility is precisely to get rid of setup.py and thus all of the crazy hairy things people do with it that they've got no business doing (because they can't accomplish what they need any other way, without first becoming an expert at distutils internals). Meanwhile, legitimate *build* extensions can and should be restrained to *building*, rather than installing. We can define a build directory layout for installers to use for actual installation, including various platform-specific elements (such as icons/menu items, registry stuff, post-install scripts, etc.) which it's then the responsibility of individual installer programs to handle. I guess that means that we would need to spec out: 1. Source tree layout 2. Build configuration files/build processing 3. configure.py 4. sdist format & metadata 5. build tree layout (for build code to build and installers to install from) But the basic idea is that the stdlib supplies the build-and-distribute framework, package authors can have build extensions, but only install programs can actually install or uninstall anything. The build layout should be flexible enough to allow things like dumping Debian or RPM or MSI metadata in there, for "install" tools (which might technically be platform bdist tools) to pull out. This is a rather important separation of duties, because right now bdist tools are responsible for build management and tricking "install" commands in order to do their work. This makes it too hard for people to write bdist tools. (And you can't even *think* about custom install tools with the distutils now.) These elements would make it possible for the distutils to be divided up into more maintainable slices, that are owned by a much larger group of people, with much lower probability of burnout. Given a stable build layout, install tools aren't likely to change much. Given a stable source and build layout, build tools aren't likely to change much. The hairiest bits (extension/compiler stuff) will have to be in stdlib anyway, and are fairly isolated within the overall process. (I also think we should encourage the model pioneered in buildout and setuptools of keeping build extensions as separate packages from the things they're used to build, but I could be convinced that making it *possible* (but not necessarily easy) to include a build extension within a package that's using it, could *perhaps* be a reasonable idea.)