[Distutils] People want CPAN :-)

Tarek Ziadé ziade.tarek at gmail.com
Sat Nov 7 00:44:32 CET 2009

On Fri, Nov 6, 2009 at 6:53 PM, Guido van Rossum <guido at python.org> wrote:
> I just found this comment on my blog. People have told me this in
> person too, so I believe it is real pain (even if the solution may be
> elusive and the suggested solutions may not work). But I don't know
> how to improve the world. Is the work on distutils-sig going to be
> enough? Or do we need some other kind of work in addition? Do we need
> more than PyPI?

I don't think PyPI has a structural problem. I do think the biggest
issue is related to the fact that it's not really possible to
describe a distribution and its dependencies today with a plain
Distutils (==stdlib). They can use Setuptools or Distribute, third
party tools. But then, there are using different installation

Changes are being done in PEP 345 + PEP 376 to address this. The goal
is to make sure Distutils provides what is required to describe
dependencies, a standard installation format, and a way to query
installed distributions.

Now, there's one thing CPAN has and PyPI don't: a network of package
repositories and an easy way to set your own
repository to serve distributions. IOW, one can sets its own CPAN
server, or add its server to an existing network.

That's useful for example if you want to have your own private PyPI
server and combine it with the public PyPI.

In PyPI, everything is centralized, and while this has some benefits,
it also makes it a single point of failure (SPF)
when you want to download distributions. We have suffered for that
problem in some sub communities
like Plone, that gets hundreds of packages at PyPI to get built. If
PyPI is down, there's nothing we can do unless we
have a local cache of packages. CPAN is way better in that field.

One work has been started to try to fix this SPF: a mirroring
standard, see PEP 381 - http://www.python.org/dev/peps/pep-0381/

In parallel, some server-side softwares are now allowing people to
register and upload packages using Distutils, like
what they would do with PyPI. For instance, you can use the "register"
and "upload" command to push
your distribution at plone.org That was made possible after some
changes to the .pypirc configuration file
(which is now allowing people to define several servers)

The next step is to add fail-over capabilities to tools like Pip or
Distribute, and ways to merge several sources of
distributions repositories: pypi and any other server that implements
the same protocol.


More information about the Distutils-SIG mailing list