[Distutils] backwards compatibility for packaging tools
zooko at zooko.com
Sat Sep 27 17:42:52 CEST 2008
When thinking about compatibility, keep in mind the distinction
between two use cases:
1. You are using a tool (e.g. distutils or setuptools) to package
your work. You might consider switching to a new tool, either one
included in the Python Standard Library or a separately-shipped one,
to build, package, and distribute your software.
2. You are re-using other people's work that they have packaged and
distributed. You might consider switching to a new tool (either
Python Standard or separate) to re-use their work.
Backwards compatibility in the first case is not overwhelmingly
important. Some people will be willing to entirely rewrite their
setup.py scripts to use a new tool, other people will be willing to
use a new tool only if they can keep using their old setup.py
scripts, and still other people will continue to package and
distribute their software with Python 2.5 and the distutils that came
with it for the forseeable future (let's say, for the next 5 years).
We can't prevent them from doing that, but also we don't need to
persuade them to change tools in order to benefit from their work.
Compatibility in the second case is overwhelmingly important. If you
offer me a new tool for re-using other people's source code, and this
new tool does *not* give me access to the thousands and thousands of
Python packages that are already out there and the dozens of hundreds
of new ones that are appearing every month, then this new tool is
completely uninteresting to me.
So we should focus on documenting and standardizing the metadata that
allows code re-use -- the interface between the author of one package
and the author of another package -- not the interface between the
author of a package and the tool that he uses to build and distribute
his own package.
We've already got a pretty good start on this -- the distutils in
Python 2.5 emits PKG-INFO and .egg-info in a format that is
understood by all of the current crop of packaging tools
(easy_install, pyinstall, distribute, yolk, bbfreeze, etc. etc. etc.).
Also of course setuptools produces .egg-info metadata in a way that
those tools can understand.
We also got past the problem that we had for awhile that Linux
distributions like Fedora and Debian were deleting those .egg-info
files. They don't do that anymore.
The next piece that is missing, from my experience in packaging Tahoe
and its numerous Python dependencies , is for more tools to start
emitting "requires" metadata in a compatible way. We already have a
de jure standard for how spell "I require zope.interface" -- PEPs 314
and 345. However, I have never seen metadata of this format in the
wild, and for all I know there are no tools that actually produce or
consume this metadata and no packages that are actually labelled with
We also have a de facto standard that is widely used by a large and
growing library of packages and is supported by a large and growing
set of tools -- the way that setuptools spells "I require
zope.interface" in its .egg-info files.
So, I have a simple and urgent request:
Extend distutils so that when the author of package passes
install_requires=['zope.interface'] to distutils.core.setup(), then
it emits an entry named "requires" with body "zope.interface" in the
resulting .egg-info. That's all. It won't hurt, and it will
probably help quite a lot to facilitate interoperation of future
Python packaging tools.
http://allmydata.org -- Tahoe, the Least-Authority Filesystem
http://allmydata.com -- back up all your files for $5/month
More information about the Distutils-SIG