backwards compatibility for packaging tools
When thinking about compatibility, keep in mind the distinction between two use cases: 1. You are using a tool (e.g. distutils or setuptools) to package your work. You might consider switching to a new tool, either one included in the Python Standard Library or a separately-shipped one, to build, package, and distribute your software. 2. You are re-using other people's work that they have packaged and distributed. You might consider switching to a new tool (either Python Standard or separate) to re-use their work. Backwards compatibility in the first case is not overwhelmingly important. Some people will be willing to entirely rewrite their setup.py scripts to use a new tool, other people will be willing to use a new tool only if they can keep using their old setup.py scripts, and still other people will continue to package and distribute their software with Python 2.5 and the distutils that came with it for the forseeable future (let's say, for the next 5 years). We can't prevent them from doing that, but also we don't need to persuade them to change tools in order to benefit from their work. Compatibility in the second case is overwhelmingly important. If you offer me a new tool for re-using other people's source code, and this new tool does *not* give me access to the thousands and thousands of Python packages that are already out there and the dozens of hundreds of new ones that are appearing every month, then this new tool is completely uninteresting to me. So we should focus on documenting and standardizing the metadata that allows code re-use -- the interface between the author of one package and the author of another package -- not the interface between the author of a package and the tool that he uses to build and distribute his own package. We've already got a pretty good start on this -- the distutils in Python 2.5 emits PKG-INFO and .egg-info in a format that is understood by all of the current crop of packaging tools (easy_install, pyinstall, distribute, yolk, bbfreeze, etc. etc. etc.). Also of course setuptools produces .egg-info metadata in a way that those tools can understand. We also got past the problem that we had for awhile that Linux distributions like Fedora and Debian were deleting those .egg-info files. They don't do that anymore. The next piece that is missing, from my experience in packaging Tahoe and its numerous Python dependencies [1], is for more tools to start emitting "requires" metadata in a compatible way. We already have a de jure standard for how spell "I require zope.interface" -- PEPs 314 and 345. However, I have never seen metadata of this format in the wild, and for all I know there are no tools that actually produce or consume this metadata and no packages that are actually labelled with this metadata. We also have a de facto standard that is widely used by a large and growing library of packages and is supported by a large and growing set of tools -- the way that setuptools spells "I require zope.interface" in its .egg-info files. So, I have a simple and urgent request: Extend distutils so that when the author of package passes install_requires=['zope.interface'] to distutils.core.setup(), then it emits an entry named "requires" with body "zope.interface" in the resulting .egg-info. That's all. It won't hurt, and it will probably help quite a lot to facilitate interoperation of future Python packaging tools. Regards, Zooko [1] http://mail.python.org/pipermail/distutils-sig/2008-September/ 010081.html --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month
zooko wrote:
So we should focus on documenting and standardizing the metadata that allows code re-use -- the interface between the author of one package and the author of another package -- not the interface between the author of a package and the tool that he uses to build and distribute his own package.
That makes sense to me. -- Greg
zooko wrote:
When thinking about compatibility, keep in mind the distinction between two use cases:
1. You are using a tool (e.g. distutils or setuptools) to package your work. You might consider switching to a new tool, either one included in the Python Standard Library or a separately-shipped one, to build, package, and distribute your software.
2. You are re-using other people's work that they have packaged and distributed. You might consider switching to a new tool (either Python Standard or separate) to re-use their work.
Backwards compatibility in the first case is not overwhelmingly important. Some people will be willing to entirely rewrite their setup.py scripts to use a new tool, other people will be willing to use a new tool only if they can keep using their old setup.py scripts, and still other people will continue to package and distribute their software with Python 2.5 and the distutils that came with it for the forseeable future (let's say, for the next 5 years). We can't prevent them from doing that, but also we don't need to persuade them to change tools in order to benefit from their work.
From your description, I understand that you care about dependencies,
I don't think it is accurate. I have no reason to doubt it is true in your case and many other people's case, but I think you have to be careful when saying 2 is "obviously" more important than 1. My impression is that 1 is important for people who invested a lot in distutils, have a lot of distutils-based extensions. It depends on the complexity of the setup.py and their build tools around. Again: numpy.distutils is as big a distutils. For numpy's case, a new well design would make most of those obsolete, but is it true for any package which has heavily invested in distutils ? this kind of things. I know *I* don't care about that for the most part, and still I would welcome a new distutils. For me, something with a sane build tool (built around a DAG, not a sequence of commands) is what matters the most. Something which enables easier packaging by OS vendors matters. So I would kindly ask to be careful when taking into account what matters and what not. cheers, David
participants (3)
-
David Cournapeau
-
Greg Ewing
-
zooko