On Wed, May 19, 2004 at 05:14:57PM +0100, has wrote:

Also, removing metadata from setup scripts means that most of the time 'install.py' will be a completely generic script that can be automatically added to the .zip at build-time; one less thing for the module developer to have to do themselves.

You keep saying "the .zip" as if that's the only, or possibly prefered, distribution mechanism. It sounds as if your exposure to heterogenous systems is pretty limited and I think you are imposing the limitations of your view onto Distutils. Your recurring recommendations for monolithic scripts, such as build.py, install.py, whatever.py also seems to be a step backward from Distutils current capabilities. The functionality I think you intend by these scripts is already integrated in Distutils. It is accomplished by subclassing the build and install command classes or their helper commands. In an object oriented language, this is the appropriate way to re-use code; Not to have to include duplicate scripts in each package's source tree. Please review some more setup.py scripts. I agree that metadata should be provided in a consistent place (e.g. setup.py or setup.cfg but not both) but if you review most setup.py scripts, you'll find what is simply a function call that passes metadata to classes that do everything you're describing and more. In more complex cases, you'll see the same function call, referencing user-supplied subclasses of internal commands that don't quite do what the author needs. I'll also agree that how this is done is not inuitive, nor well documented, but it works none-the-less. I appreciate that your suggestions are based on your experience with Distutils. It seems to me however, that your experience is limited and some of the suggestions you are proposing will impose severe limitations on those of us that deal with more diverse environments. mwa -- Mark W. Alexander slash@dotnetslash.net The contents of this message authored by Mark W. Alexander are released under the Creative Commons Attribution-NonCommercial license. Copyright of quoted materials are retained by the original author(s). http://creativecommons.org/licenses/by-nc/1.0/

Stefan Seefeld wrote:

...

- Decoupling DU build procedures from DU installation procedures also implies delegating control of building and installing to separate scripts (e.g. 'build.py', 'install.py'), rather than having a single 'setup.py' script doing double duty.

As others already pointed out, what you suggest doesn't really change anything: 'setup.py' is just a facade to access distutils' functionality. building and installing *is* already controlled by two distinct objects ('commands').

...

Superficially this may sound like it's creating more(!) work for module developers, but bear in mind my goal of making these scripts more generic so that in most cases the module developer can simply [re]use existing ones rather than have to code new versions each time.

what do you want to reuse that you can't right now ?

The 'setup.py' [or whatever you want to call it] script. Reduce the amount of code the module developer needs to write. For simple modules and packages, this could (and should) be zero.

However, in order for 'build' and 'install' to work smoothly together, they both have to respect some conventions, for example use some common metadata format to share information about the things to be installed.

Of course. My point is that some of this data shouldn't be in the setup scripts, and at least some of the rest should be automatically determined by the system rather than specified by the user (unless they need to override the automatics): - Put metadata (by which I mean information describing the module/package itself; its name, version, author, dependencies, etc) into a standard metadata file, e.g. meta.txt, within the package. - Put data/code used to build the distro into a build.py script. Put data/code used to install the distro into a install.py script. i.e. There should be a clear distinction made between information describing the module and information used merely in building/installing it (custom paths, framework bindings, etc). These are two very different things and should be handled accordingly. [Note: any time I say 'metadata' I'm referring to the former, not the latter.] With DU1, all this data is squidged into setup.py. This is suboptimal: it's not very convenient to read/edit, and there's some unnecessary duplication of metadata occurring over the build/install process as some of this data gets duplicated into PKG-INFO (which'd be unnecessary if it was put into a meta.txt file in the first place). Often the only unique information stored in setup.py is said 'metadata', which is really a suboptimal arrangement. (Plus stuff like sub-package names, which can and should be gotten rid of in most, if not all, cases.) Maybe a practical example'd help: Current: from distutils.core import setup setup(name = 'HTMLTemplate', version = '0.4.3', description = 'HTML templating engine.', author = 'HAS', author_email = '', # see Manual.txt url='http://freespace.virgin.net/hamish.sanderson/htmltemplate.html', license = 'LGPL', platforms = ['any'], long_description = 'HTMLTemplate converts XML/HTML/XHTML templates ...', py_modules = ['HTMLTemplate'], ) Suggested: - meta.txt version 0.4.3 description HTML templating engine author HAS author email see Manual.txt url http://freespace.virgin.net/hamish.sanderson/htmltemplate.html license LGPL platforms any long description HTMLTemplate converts XML/HTML/XHTML templates ... - install.py #!/usr/bin/env python from DU2 import install install() - build.py #!/usr/bin/env python from sys import argv from DU2 import build build(argv[1], omit='\.pyc$') Eliminating unnecessary duplication means the name shouldn't need to be declared more than once (i.e. the package folder name). The py_modules value is, by default, unnecessary. PKG-INFO is obsolete. README can be auto-generated (though I kinda wonder just how useful this really is and if it could be eliminated altogether). Both install.py and build.py are generic scripts: the former can be automatically inserted into the distro, the latter run from the shell as a standard build script. It'll still scale up just as well as DU1, so that's not a concern. The aim here is to handle the simplest and [presumably] most common cases more cleanly. HTH has -- http://freespace.virgin.net/hamish.sanderson/

I wrote:

Eliminating unnecessary duplication means the name shouldn't need to be declared more than once (i.e. the package folder name). The py_modules value is, by default, unnecessary. PKG-INFO is obsolete. README can be auto-generated (though I kinda wonder just how useful this really is and if it could be eliminated altogether). Both install.py and build.py are generic scripts: the former can be automatically inserted into the distro, the latter run from the shell as a standard build script.

Oops, forgot to mention an obvious benefit for DU developers: eliminating a not insignificant quantity of code from DU. In particular, all module metadata handling (parsing/generating/verifying) duties can be handled by a much more useful general-purpose modulemetadata module in the standard library, to which DU would become just another client. HTH has -- http://freespace.virgin.net/hamish.sanderson/