I'm working on a set of distutils enhancements called "setuptools". The "setuptools" extend the base Distribution class with the ability to do: * feature management -- you can define subsets of a distribution as "features" that can be enabled or disabled via '--with-X' and '--without-X' options to 'setup.py'. Features can (recursively) depend on other features, and be included or excluded by default. (Their inclusion or exclusion affects all commands, as the relevant packages/modules/data/scripts/whatever are added or removed just after command line arguments are parsed.) * package data installation -- you can specify filenames or globs to be searched for in package source directories, for installation alongside the package .py files. All without munging the 'install_data' command or manually copying files, and the data files are correctly included in the 'get_outputs()' manifest for all the distutils features that depend on knowing what files were copied. * 'test' command -- optionally run a user-specified or 'setup.py'-specified test suite via the 'unittest' module. * Packages and modules can be simultaneously specified as arguments to 'setuptools.setup()', unlike the standard distutils 'setup()'. The above features are all fully implemented and documented via docstrings, and the majority of them are covered by an extensive unit test suite. "setuptools" uses itself for its own 'setup.py', in order to enable the 'test' command to run its built-in unit tests. The purpose of "setuptools" is to collect commonly useful distutils enhancements for large or complex packages like Zope X3 and PEAK. It's also intended to be the starting point for developing simple installer-side dependency support, as described/proposed at: http://www.python.org/cgi-bin/moinmoin/DistutilsDependencies Finally, I also intend to add some type of general-purpose documentation build and install facilities, to replace the rather hacky documentation support in my current setup scripts. At this point, the package isn't quite ready for general release, as it doesn't include documentation beyond docstrings, and of course the dependency and documentation features aren't written yet. But I believe it is ready for feedback from interested parties such as PyCon sprinters, and people who are undertaking complex distutils projects. I'm interested in your comments, especially as they may relate to the viability of eventually merging setuptools into a "Distutils 2.0" (or maybe "1.5"). Setuptools is specifically laid out to match the package layout and naming conventions of the distutils in order to facilitate such porting. You can currently find the setuptools CVS at: http://cvs.eby-sarna.com/setuptools/ although there has been some discussion about moving it to the Zope X3 CVS in the near future to facilitate the coming sprint, and other contributions. Please follow up with discussion to the distutils-sig mailing list. Thanks.
On Sunday 07 March 2004 05:38 pm, Phillip J. Eby wrote:
* feature management -- you can define subsets of a distribution as "features" that can be enabled or disabled via '--with-X' and '--without-X' options to 'setup.py'. Features can (recursively) depend on other features, and be included or excluded by default. (Their inclusion or exclusion affects all commands, as the relevant packages/modules/data/scripts/whatever are added or removed just after command line arguments are parsed.)
This seems nice, though I'm not sure I actually have an immediate need for this. Do you expect to use this to provide sub-packages that would only be installed if they weren't installed by some other distribution?
* package data installation -- you can specify filenames or globs to be searched for in package source directories, for installation alongside the package .py files. All without munging the 'install_data' command or manually copying files, and the data files are correctly included in the 'get_outputs()' manifest for all the distutils features that depend on knowing what files were copied.
I think this should be integrated directly into distutils as soon as possible. I looked at your implementation as of Friday, and it made sense to me.
* 'test' command -- optionally run a user-specified or 'setup.py'-specified test suite via the 'unittest' module.
People have been wanting this in distutils for a while, but I don't think people agree much on how unit tests should be run (how to write them isn't very well agreed on either). I suspect it's premature to integrate this into the main distutils at this time.
* Packages and modules can be simultaneously specified as arguments to 'setuptools.setup()', unlike the standard distutils 'setup()'.
Yes! This is goodness, and has been long requested. This should go directly into distutils.
At this point, the package isn't quite ready for general release, as it doesn't include documentation beyond docstrings, and of course the dependency and documentation features aren't written yet. But I believe
If you package up the two features I care most about as a patch to distutils, I'll happily write the documentation for them. ;-) I like the idea of having an extended distutils for advanced packaging, but I don't see that such a beast would need to be integrated into distutils whole hog, or that it makes sense to do so. We should be able to pick the features that belong in the distutils core independently of what an extended package supports. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation
At 03:21 PM 3/8/04 -0500, Fred L. Drake, Jr. wrote:
On Sunday 07 March 2004 05:38 pm, Phillip J. Eby wrote:
* feature management -- you can define subsets of a distribution as "features" that can be enabled or disabled via '--with-X' and '--without-X' options to 'setup.py'. Features can (recursively) depend on other features, and be included or excluded by default. (Their inclusion or exclusion affects all commands, as the relevant packages/modules/data/scripts/whatever are added or removed just after command line arguments are parsed.)
This seems nice, though I'm not sure I actually have an immediate need for this. Do you expect to use this to provide sub-packages that would only be installed if they weren't installed by some other distribution?
Actually, I intended that facility to support Zope X3 automatic package assembly, among other things. So, you could say for example that 'zope.publisher' depends on 'zope.interface', in order to be able to automatically assemble a subset distribution. I was envisioning you using it as the output of your "scan the packages for setup metadata" tool. That is, you'd create Feature instances for the packages, including their extensions and dependencies on other packages. Then, you could do e.g.: python setup.py --with-zope-publisher dist to build distributions that include zope.publisher and its dependencies from the Zope source tree. (Of course, you might also have some "logical" features that do nothing but list some "physical" features representing packages, rather than defining packages as features. Also, if you create features with the 'optional=False' argument, they do not get '--with/--without' options created, so you can use features as an internal organizing mechanism without exposing them to the user.) What I'm using features for in PEAK is to include optional things like a FastCGI module, a 'uuidgen' module that's only used on certain BSD platforms, and so on. And in PyProtocols, it's used to allow people to do: python setup.py --without-speedups install if they don't have a C compiler.
* 'test' command -- optionally run a user-specified or 'setup.py'-specified test suite via the 'unittest' module.
People have been wanting this in distutils for a while, but I don't think people agree much on how unit tests should be run (how to write them isn't very well agreed on either). I suspect it's premature to integrate this into the main distutils at this time.
Hm. What if I move the current implementation to a 'unittest' command, and make the suite argument 'unittest_suite'? Then, I could add a new 'test' command with 'unittest' as a subcommand, and the result would then be extensible for other forms of testing.
At this point, the package isn't quite ready for general release, as it doesn't include documentation beyond docstrings, and of course the dependency and documentation features aren't written yet. But I believe
If you package up the two features I care most about as a patch to distutils, I'll happily write the documentation for them. ;-)
As it happens, both of those features are implemented by the 'build_py' command, found in 'setuptools.command.build_py', and corresponding to 'distutils.command.build_py'. The only other bit you need is to add: self.package_data = {} to the beginning of 'distutils.dist.Distribution.__init__()', to allow passing 'package_data' to setup(). There are a couple of tricky bits to this, though. First, my 'build_py' makes superclass calls to the original 'build_py', so you'd need to move the old 'get_outputs()' implementation to another method name, then call that where I call the original 'build_py.get_outputs()'. I'll then have to have setuptools check whether that extra method name exists, and if so, have it use distutils' 'build_py' command instead of its own (for forward compatibility w/Python 2.4).
participants (2)
-
Fred L. Drake, Jr.
-
Phillip J. Eby