[Distutils] setuptools development cycle

Wed Oct 3 22:05:51 CEST 2007

Hi,

I think easy_install should try to have a look at what Maven2 did. I _know_
this is never a good idea to give some java examples in a python list, but
maven2 has some very interesting concepts, most of them being absolutely
language agnostic. (note: Maven2, not Maven1). In fact, I think Maven2 could
be implemented in any language (except that almost all examples and plugins
are about java... too bad :-( ). May be there is something to learn from
their experience.

First, there is the POM. This is the equivalent of the setup.py, where we
put all the description of the project. We find the authors names, the code
location, the tests location, the artifacts we can create, the dependencies,
etc. What is really interesting is that this description is really _only_ a
description. There is absolutely no "action" (script). The POM is really
conceptually close to setup.py, without any action in it. It contains the
structure of the project.

Then come the plugins. I think they are the equivalent of setuptools
extensions. And really, what makes Maven so nice is really the set of
plugins. Nothing else (IMHO). A few examples that are really nice: Having a
fully available Eclipse project ready, with all dependencies downloaded in
the local cache and included in the classpath is just one command away ("mvn
eclipse"). Ok, it means a few other things, such as an artifact repository,
but it's nice anyway. Testing is "mvn test", building is "mvn build", etc.
(Yes I know, that's similar to some setuptools commands)

So basically, I really think that setup.py should benefit from this
experience:

- Turn setup.py into a real description-only file (no commands), or invent
another file

- Have a local repository of eggs, _not_ included in the python path by
default

- Get rid of the site-package hell: find a nice way to start a python with a
predefined set of eggs (yes, the equivalent of naming all the eggs in the
python path, but in a more simple way, such as "python
--python_path_description=myconfig", myconfig being an alias to a set of
eggs)

- Manage the dependencies by downloading the eggs in the local repository
(fully independent from python itself)

- Extract everything from the core setuptools which could be in a plugin
(everything which does not read the description of the project and calls the
plugins)

- Create a set of plugins to deal with most common use cases, such as
packaging eggs, compiling libraries, handling special types of files etc...

- And create a nice wiki tutorial to explain how to create the best plugins
you have ever dreamed of

- Let others write the plugins for you

Actually, I believe the plugin orientation is already chosen ? (
http://mail.python.org/pipermail/distutils-sig/2007-January/007113.html )

And I know that finding time is the most difficult part of it...

My 2 cents.

2007/10/3, Jim Fulton <jim at zope.com>:
>
>
> On Oct 3, 2007, at 1:06 PM, Phillip J. Eby wrote:
>
> > At 10:26 AM 10/3/2007 -0400, Jim Fulton wrote:
> >> I'm a little bit worried about setuptools development cycle.  We seem
> >> to be stalled at a 0.6 pre-release that is quite stable and widely
> >> used in production.  The next feature release, 0.7, seems to
> >> anticipate a major refactoring and major new features.  I'm a bit
> >> worried about this both from a fear of potential disruption, and a
> >> need for incremental feature development.
> >>
> >> I propose that the grand re-factoring of setuptools planned for 0.7
> >> be moved off the trunk and to a 2.x release.  0.6 should be marked
> >> final
> >
> > This is coming soon, but I want to finish some ez_setup
> > improvements first.  There will be a "known issues" list to cover
> > things like the incomplete/broken FTP support; I've given up on
> > trying to keep hacking it into 0.6.
>
> I wonder what "this" you are referring to.
>
> >
> >>  Is it on the table to remove features from setuptools?
> >
> > Yes - and most are *already* removed from the 0.7 trunk.  That was
> > the main reason for the branch.
>
> I'm confused on a number of counts.  Is the 0.7 "trunk" the same as
> the setuptools trunk? Is there also a branch for 0.7?
>
> Is there a list of removed features somewhere? Do we get any input?
>
> > Other (relatively minor) refactorings expected in 0.7 are to make
> > URL handlers for easy_install pluggable, and to add build-time
> > plugin options (so that people can implement custom build steps).
> >
> > The biggest feature additions in 0.7 would be support for
> > uninstallation, package inspection, virtual environment management,
> > and ability to easy_install eggs using "classic/
> > development" (i.e., .egg-info) installation layout.
> >
> > Other new features planned for 0.7 include "or-ed" requirements,
> > build/install of shared libraries, and "standard extras" (a way to
> > specify to easy_install a set of extras that should be installed if
> > they are defined by the targets).
> >
> > None of this is what I'd consider "grand" refactoring; they're all
> > either new features or minor refactorings to make things
> > pluggable.  The most complex of them is probably the URL handler
> > one, and only because the existing URL handling in
> > setuptools.package_index is so hosed.
> >
> > Now, if you include pkg_resources in, I would rather like to
> > refactor the way the global objects (working set and resource
> > manager) are handled, but I don't know how practical or even
> > possible it is.  There are other things in pkg_resources that could
> > use refactoring, but I'm not sure I dare.
>
> "Grand" is simply the impression I had.  The list of features  above
> is extensive though. Some of the items seem fairly big, or maybe I
> just can't tell what they are. Do you plan to try to get all of them
> into the next feature release? Or do you plan them a list of ideas
> for future releases.
>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

-- 
Laurent Ploix
http://lauploix.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20071003/ad99cd53/attachment.htm 