[Distutils] setuptools development cycle

[Ian Bicking]

Laurent Ploix wrote:
> Hi,
> 
> I think easy_install should try to have a look at what Maven2 did. I 
> _know_ this is never a good idea to give some java examples in a python 
> list, but maven2 has some very interesting concepts, most of them being 
> absolutely language agnostic. (note: Maven2, not Maven1). In fact, I 
> think Maven2 could be implemented in any language (except that almost 
> all examples and plugins are about java... too bad :-( ). May be there 
> is something to learn from their experience.
> 
> First, there is the POM. This is the equivalent of the setup.py, where 
> we put all the description of the project. We find the authors names, 
> the code location, the tests location, the artifacts we can create, the 
> dependencies, etc. What is really interesting is that this description 
> is really _only_ a description. There is absolutely no "action" 
> (script). The POM is really conceptually close to setup.py, without any 
> action in it. It contains the structure of the project.
> 
> Then come the plugins. I think they are the equivalent of setuptools 
> extensions. And really, what makes Maven so nice is really the set of 
> plugins. Nothing else (IMHO). A few examples that are really nice: 
> Having a fully available Eclipse project ready, with all dependencies 
> downloaded in the local cache and included in the classpath is just one 
> command away ("mvn eclipse"). Ok, it means a few other things, such as 
> an artifact repository, but it's nice anyway. Testing is "mvn test", 
> building is "mvn build", etc. (Yes I know, that's similar to some 
> setuptools commands)
> 
> So basically, I really think that setup.py should benefit from this 
> experience:
> 
> - Turn setup.py into a real description-only file (no commands), or 
> invent another file

For backward compatibility this doesn't really work.  Potentially you 
could have a setup.py:

"""
from mavon_style_lib import setup

setup(__file__)
"""

And have that function read the description-only file.

I personally would prefer, say, PKG-INFO to be the canonical source of 
information instead of setup.py.  But then, I also find it handy 
sometimes to generate long_description from a segment of a text file in 
my documentation.  That's easy currently, and would be much harder with 
a description-only file.

> - Have a local repository of eggs, _not_ included in the python path by 
> default

If you easy_install -m then they won't be on the python path.  But it's 
all very fuzzy; I don't think many people make good use of -m in 
deployments.  I know I don't.

> - Get rid of the site-package hell: find a nice way to start a python 
> with a predefined set of eggs (yes, the equivalent of naming all the 
> eggs in the python path, but in a more simple way, such as "python 
> --python_path_description=myconfig", myconfig being an alias to a set of 
> eggs)

This is what zc.buildout and virtualenv do, more or less.  zc.buildout 
includes building the initial set of eggs; in virtualenv I'll be 
suggesting poach-eggs (https://svn.openplans.org/svn/PoachEggs/trunk) to 
install a set of packages.

> - Manage the dependencies by downloading the eggs in the local 
> repository (fully independent from python itself)

In buildutils I've added a bundle command, which kind of does this -- 
collecting all the eggs for a particular package.  You do this with 
"python setup.py bundle".  Buildutils is available with "hg clone 
https://www.knowledgetap.com/hg/buildutils/" (sorry it's not fully 
released at the moment).

> - Extract everything from the core setuptools which could be in a plugin 
> (everything which does not read the description of the project and calls 
> the plugins)

Setuptools already self-provides lots of its functionality as plugins. 
I don't see any need to extract these into separate packages.

> - Create a set of plugins to deal with most common use cases, such as 
> packaging eggs, compiling libraries, handling special types of files etc...
> 
> - And create a nice wiki tutorial to explain how to create the best 
> plugins you have ever dreamed of

This needs some work; I'd like to make a paster create template in 
buildutils for creating setuptools' extensions.  It would be pretty 
small.  It's not terribly hard to write a setuptools' extension now, 
though you do have to deal with the somewhat crufty interface of 
distutils commands.  It's not really so bad, though.

-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
             : Write code, do good : http://topp.openplans.org/careers