This has been a very good discussion. I have been doing a bit more thinking regarding why I intended to use a packaging tool in the first place.
I am relying on the package versioning scheme to ensure that critical libraries line up across families of applications. Libraries for internal company-wide file formats are a good example of this.
I also think that it is easier for a deployers and developers to have an object or entity that must be equivalent across all applications for the application to be valid. It is easier than referencing a Subversion tag.
Given that, what would my ideal application lifecycle look like?
I would want applications to be registered to the machine, but I want libraries to be local to the execution environment. Sort of like using the 'multi-version' option, but the environment name is still in a central database. Then I could issue the following command to update every applicaton on the system:
% easy_install --list-registered | xargs -i easy_install -U -
I get: - multiple independant environments - a central database of registered environments - a database of packages on a per-environment basis by looking at the physical package names in each environment - the ability to copy elements across environments easily - I can not share elements across environments - I can use elements from the system-wide shared repository.
It would be nice to install packages as either zip, editable, or development mode on a per-environment basis. Heck, why not have the ability to install the entire environment in development mode, with one command?
I prefer the distutils installation scheme for --home or --prefix or --root lined up across operating systems. But this is contrary to distutils' general case installation scenario. (perhaps a new keyword, '--environment' or '--deploy' would make sense? Or we could use setup.cfg)
As far as general versioning and release policies, the method outlined py the py.lib project looks to be light-weight and flexible (micro-releases, API backwards compatability guaranteed through testing, etc.): http://codespeak.net/py/current/doc/why_py.html#upcoming-release-policy-api-...
This works well with the automatic upgrading of micro-releases made possible by the '1.0>=,<2.0a' version specification.
The micro-branching techniques that Ian mentioned go nicely with the previously mentioned release policy and testing guideline. http://divmod.org/trac/wiki/BranchBasedDevelopment
The overall process looks like a good method to guarantee that the trunk never gets broken (which can be a *very* annoying problem, for the reasons the Divmod guys outline).
It is also possible that micro-branching would solve the ChangeLog issues that Ian spoke of. Since each micro-branch merge is for a specific feature it would be possible to create a nice formatted commit message, and use something like svn-to-changelog on the trunk to get the whole ChangeLog automatically (even if the output format from svn2cl is ugly...): http://ch.tudelft.nl/%7Earthur/svn2cl/
It would be nice to write my code in an entirely version-agnostic manner by relying on the environment entry points to handle the package specification. Then my import statements are just normal Python code.
The idea of an environment-specific package dependancy override is very nice. Almost a necessity, in fact. We have to handle the override at resource aquisition time (before site.py is generated?)
It would be nice to hook setuptools into scons or aap or even make for the time being, so that I could handle a more customized environment setup. Things like configuration templates, log file stubs, auto-generated documentation, an environment-specific directory structure, etc. could all be there. (or maybe I could just check the entire environment template out of Subversion, and let easy_install handle filling the template with Python code.)
To support the enviroment templates we would need to somehow extract some of the package data to a standard location. I considered using eager_resources for this, but it is not quite eager enough for my requirments. I could write an extension, 'very_eager_resources', to do this instead.
I believe that everything here is possible using setuptools, easy_install, and pkg_resources in it's current form, with a few custom extensions. There is definitely a focus on using setuptools as an environment building and maintenance tool, or at least as something to glue more specialized tools together.
The challenge is to create a process that is light-weight enough to be accepted for day-to-day use, instead of re-creating something like Java's EAR and WAR specifications.