[Distutils] formencode as .egg in Debian ??
ianb at colorstudy.com
Mon Nov 28 18:27:37 CET 2005
Josselin Mouette wrote:
> Le dimanche 27 novembre 2005 à 16:26 -0600, Ian Bicking a écrit :
>>>>No one is forcing Debian to package any of these.
>>>Of course you are forcing Debian to package these. As long as your
>>>projects have enough users, someone will want to build a Debian package.
>>>The whole point of this thread is to make this package not suck.
>>easy_install works, right now, for these packages. There are some
>>outstanding issues, and those issues can probably be resolved in
>>easy_install, without any intervention by Debian. Maybe the simplest
>>resolution is adding an option like:
>> easy_install --already-provided=1.2.6 ElementTree
> We're not going to tell our users to use easy_install for some kind of
> packages and some other thing for another package, and so on. They
> expect to be able to install anything, just by typing "apt-get install
That's okay; I'll just tell my users to use it when Debian packages are
not available (which is the case for the majority of Python libraries,
and probably always will be unless libraries are automatically
packaged). easy_install provides a much better and safer experience
than downloading tarballs and using the normal "setup.py install"
routine; it is an improvement on the status quo. The other option would
be to allow people to easily create debs, in the same way they can
easily create RPMs right now; but I get the strong impression you don't
think that's a good idea.
>>>I'm happy you see stable releases as "counterproductive". However many
>>>of our users happen to like them, as they don't like to upgrade their
>>>software every other day. Again, eggs are a useful tool only for
>>>developers, not for regular users.
>>These are developer tools, regular users *are* developers.
>>If someone reports a bug, and I fix it, and they can't get access to
>>that because they can't install a new version of the package, then that
>>is counterproductive. It discourages feedback.
> Stable releases are not meant for developers like you, but there are
> developers who want a stable development platform, even if it has some
> bugs. No bug fixing also means no regressions. The fact you don't
> understand those people's needs - and forget them when designing your
> tools - doesn't make all stable releases "counterproductive".
I'm speaking in terms of the open source ecosystem -- gaining
contributors is far more valuable than gaining users. So from my
perspective -- which isn't out of line from Debian's, I think -- cutting
off feedback is damaging.
> Developers who want a bleeding edge development platform should be using
> Debian testing or unstable, where the new versions become available as
> soon as possible.
I don't think those really provide the same experience. If you are
really serious about the bleeding edge, you should be using a checkout
from the upstream repository. Which, incidentally, setuptools and
easy_install handle nicely. But again, I say that from the perspective
of someone who wants to gain contributors, not just users.
>>*Or*, we develop various ad hoc techniques to allow
>>people to install other versions of packages *in spite* of the Debian
> If your packages can't be installed in /usr/local, where they take
> precedence over the version in /usr, your packaging system is flawed.
They can be, with no problems; easy_install uses the standard
distutils.cfg configuration, which can indicate prefix=/usr/local. Even
without it, easy_install/setuptools is much less likely to clobber
Debian installed packages, compared to normal distutils.
>>I just don't think it is an option to say that only one version of a
>>package can be installed. In /usr/lib/pythonX.Y/site-packages, sure,
>>and there can potentially be other restrictions. But you are limiting
>>your users too much if there can never be multiple versions installed.
> Stable users will want to keep the same version, and allowing them to
> install several versions provides a way to easily break their systems.
> It also cuts them off the security fixes.
New versions don't overwrite old versions. I believe that if you have a
library in /usr/lib, and install a new version to /usr/local/lib, that
the new version won't ever be accessible until you specifically
pkg_resources.require() it (or implicitly require it due to
dependencies). *However*, to make that work nicely an easy_install'ed
package should see metadata about Debian installed packages; otherwise
duplicate versions of libraries have to be installed, because it's not
reasonable to discover at runtime what Debian packages are installed (if
only because the Debian package database is much too slow). Though if
you really wanted, setuptools/easy_install could be patched to read the
debian package database on installation, maybe even writing a .egg-info
directory then (putting it in /usr/local/lib, but it can refer back to a
library that was installed in /usr/lib). That mostly depends on the
enthusiasm of the setuptools packager.
Anyway, in spite of all the disagreement, I think we actually have
reached a sort of conclusion...
For the packages in question (ultimately leading to packaging
TurboGears) there's only one current Debian package of consequence:
ElementTree. I think it's fine to remove that package in particular
from the egg dependencies. So the timescale for changes to Debian
aren't that much of an issue because these are new packages. Most of
the libraries are developed with setuptools upstream... I think all of
them, in fact (setuptools itself, TurboGears, Kid, SQLObject,
FormEncode, probably nose as well).
There are some questions about zip files -- simplest answer: don't
install them as zips. Some of them can't be installed that way anyway.
There are some questions about the performance implications of
installing them in subdirectories of site-packages, and extending
sys.path for each installed package. You can revert to normal
installation with an .egg-info directory, and Phillip will (or already
has?) added support for putting the version number in those directory
names, so that will all work fine. I believe if you require a version
of a package from /usr/local/lib, it will do the Right Thing and change
As I think about it, not putting Debian-packaged libraries entirely in
subdirectories (the typical egg installation) seems best -- keeping
libraries in subdirectories makes package management easier (by
easy_install, nest, or whatever), but in this case we definitely *don't*
want those tools to try to manage those packages. Maybe we could even
add a flag to indicate to setuptools/easy_install when a library is
managed by some external tool? I think that would add some important
safety. I would suggest something like a
Package.egg-info/managed_by.txt file, whose existance indicates the
package is externally managed, and whose content explains what manages
it (so setuptools could provide a nice error message that directs the
user to apt-get on Debian, or some other native tool on other platforms).
A distutils.cfg file (bug 338572) will allow people to use easy_install
on Debian systems, and still follow Debian policy. But I assume that
such a change will take some time to propogate even if it is applied
immediately to sid. But even without it, easy_install won't overwrite
any Debian package files without that configuration file. I think
easy_install also gives some scary-looking warnings in that case, but
that might be possible to handle in documentation, especially if we all
agree on what steps users should be following.
Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
More information about the Distutils-SIG