[Distutils] "setuptools" package preview

[Phillip J. Eby]

At 03:21 PM 3/8/04 -0500, Fred L. Drake, Jr. wrote:
>On Sunday 07 March 2004 05:38 pm, Phillip J. Eby wrote:
>  > * feature management -- you can define subsets of a distribution as
>  > "features" that can be enabled or disabled via '--with-X' and
>  > '--without-X' options to 'setup.py'.  Features can (recursively) depend on
>  > other features, and be included or excluded by default.  (Their inclusion
>  > or exclusion affects all commands, as the relevant
>  > packages/modules/data/scripts/whatever are added or removed just after
>  > command line arguments are parsed.)
>
>This seems nice, though I'm not sure I actually have an immediate need for
>this.  Do you expect to use this to provide sub-packages that would only be
>installed if they weren't installed by some other distribution?

Actually, I intended that facility to support Zope X3 automatic package 
assembly, among other things.  So, you could say for example that 
'zope.publisher' depends on 'zope.interface', in order to be able to 
automatically assemble a subset distribution.  I was envisioning you using 
it as the output of your "scan the packages for setup metadata" tool.  That 
is, you'd create Feature instances for the packages, including their 
extensions and dependencies on other packages.  Then, you could do e.g.:

     python setup.py --with-zope-publisher dist

to build distributions that include zope.publisher and its dependencies 
from the Zope source tree.  (Of course, you might also have some "logical" 
features that do nothing but list some "physical" features representing 
packages, rather than defining packages as features.  Also, if you create 
features with the 'optional=False' argument, they do not get 
'--with/--without' options created, so you can use features as an internal 
organizing mechanism without exposing them to the user.)

What I'm using features for in PEAK is to include optional things like a 
FastCGI module, a 'uuidgen' module that's only used on certain BSD 
platforms, and so on.  And in PyProtocols, it's used to allow people to do:

     python setup.py --without-speedups install

if they don't have a C compiler.


>  > * 'test' command -- optionally run a user-specified or
>  > 'setup.py'-specified test suite via the 'unittest' module.
>
>People have been wanting this in distutils for a while, but I don't think
>people agree much on how unit tests should be run (how to write them isn't
>very well agreed on either).  I suspect it's premature to integrate this into
>the main distutils at this time.

Hm.  What if I move the current implementation to a 'unittest' command, and 
make the suite argument 'unittest_suite'?  Then, I could add a new 'test' 
command with 'unittest' as a subcommand, and the result would then be 
extensible for other forms of testing.


>  > At this point, the package isn't quite ready for general release, as it
>  > doesn't include documentation beyond docstrings, and of course the
>  > dependency and documentation features aren't written yet.  But I believe
>
>If you package up the two features I care most about as a patch to distutils,
>I'll happily write the documentation for them.  ;-)

As it happens, both of those features are implemented by the 'build_py' 
command, found in 'setuptools.command.build_py', and corresponding to 
'distutils.command.build_py'.
The only other bit you need is to add:

     self.package_data = {}

to the beginning of 'distutils.dist.Distribution.__init__()', to allow 
passing 'package_data' to setup().

There are a couple of tricky bits to this, though.  First, my 'build_py' 
makes superclass calls to the original 'build_py', so you'd need to move 
the old 'get_outputs()' implementation to another method name, then call 
that where I call the original 'build_py.get_outputs()'.  I'll then have to 
have setuptools check whether that extra method name exists, and if so, 
have it use distutils' 'build_py' command instead of its own (for forward 
compatibility w/Python 2.4).