[Distutils] Python people want CPAN and how the latter came about

[David Lyon]

On Tue, 22 Dec 2009 07:48:55 +0900, David Cournapeau <cournape at gmail.com>
wrote:

> (cpan) ... is not much a
> feature problem as much as a fundamental implementation and "state of
> mind". 

Yes. Their state of mind is make it simple and make it work across
the board for everything for everyone.

> Reliable packaging requires explicit handling, where the whole
> python stack for packaging relies a lot on implicit behavior. 

Yep.. That's it in a nutshell.

> (distutils) .. is a design error ... , instead of fixing 
> it at this level, tools try to add another level of complexity 
> to circumvent it. 

This now begs the beer-drinking style (cpan/perl developers understand)
question of what to actually do about this in 2010. 

One thing I'm pretty certain of is that people would prefer to
accomplish something and after that drink beer together. That
was really the secret of the cpan success. They loved the
sense of accomplishment. They loved bragging too - but it
was always a welcoming type of bragging that one could easily
fall in to and feel a part of.

After reading your email David, you leave the reader with no
other conclusion other than a new explicit build tool is
needed for 2010.

Otherwise, we're all going to be sitting helplessly reading
the same old 'how can I tell distutils I want to ...?" for
the whole of the next decade.

At the same time, there won't be much of an effort to
address the design issues you talk about. Simply because
of legacy software syndrome management issues. 

What can be done, if there is good delegation of the
tasks, is for a new pypi toolset (let's not fragment
things and confuse new users with lots of different
names) to be started.

At the highest level, the toolset needs to:

 - build python packages/apps from explicit metadata

 - upload and register them

 - download and install (and the other functions)

I'd toss in a few extra things not in cpan. Like
incorporating buildout support. I don't use it myself
but so many others do.

The bigger issue is about deployment. It's about 
getting python apps/libraries out to all the machines 
out there with as little effort as possible.

The Package concept is based on a static snapshot
idea.

But how much python software is now in mercurial
or bzr repositories ? lots of times software is
slow moving. Yes, we do want that latest bug
fix. Say it didn't work.. jump back a revision
or two.

Is a 'package' really a good way to deploy? I'm
not against them, just suggesting that a
repository link in pypi metadata is an equally
good but perphaps more modern way.

What is python doing or going to do to help us
manage all these deployment issues. At some point
in time the stdlib has got to address lots of
'stuff' landing up on the end machine.

Batteries included has just got to include
built in bzr or mercurial or cvs or svn
support.

Just where I work I've written enough mini apps
for different machines deployment is getting
to be a challenge. Most have scm. But setup from
scratch is still un-optimal imo.

How does our fundamental distutils design cope
with the evolving landscape? including supercomputer
and server farm type requirements. Also just plain
commercial and scientific (workstation) type
implementations.

My list of future packaging/configuration/
deployment projects to interoperate with are
(but not limited to):

 - buildout

 - celery (http://pypi.python.org/pypi/celery/0.3.0)

 - pip/distribute/setuptools

 - bzr/svn/hg etc

Add to that the web frameworks..

Anyway, you've totally convinced me it's time for
a new pypi package/app build tool.. 

I would like to also suggest that it be built
upon some small python web framework like cherrypy
and fire up a browser for the user to work with.

I don't want another text based command line
program sorry. Maybe it is just a django app.

Anybody adventurous enough to build a package or
an app isn't going to mind installing a small
web app to run on port 8080 or whatever to do
their package build - imo.

Cheers.. drink and be merry

David