[Distutils] formencode as .egg in Debian ??

[Josselin Mouette]

Le mercredi 23 novembre 2005 à 18:06 -0500, Phillip J. Eby a écrit : 
> The degree of negativity from the Debian side at the outset of this 
> conversation (virtually all of it from you) has not been conducive to 
> making this happen.  As a simple matter of practicality, I can't afford to 
> leave your comments unanswered, not because I feel any need to convince you 
> personally of anything, but because I don't want to leave anyone else with 
> the impression that your portrayal of these so-called "drawbacks" is a fair 
> one.  Otherwise, I would have just ignored your comments and focused on 
> working with the people who seem more interested in finding solutions than 
> finding ways to declare a non-existence of the problem.  As it is, I feel 
> forced to spend time replying to your comments point-by-point, that I could 
> otherwise spend on actually helping to resolve the issues.

The reason why people from Debian are complaining is that all solutions
you have proposed so far are not acceptable. Full stop. Currently, I
can't see any way to package eggs in a clean and elegant way.

> If I were to adopt your tone, I would be calling Debian a fragile and 
> broken system that is unable to deal well with simple matters like editing 
> a file upon installation,

I suggest you go get yourself a cluebat. Editing a file upon
installation is a completely stupid idea, and that's why we try to avoid
it whenever possible. The reason should be obvious: while it's easy to
add or remove files from the system, you can never guarantee a file will
be successfully edited.

Furthermore, editing a file for modifying the sys.path is an even more
stupid idea, as python already has everything you need to deal with new
modules: simply ship them in site-packages/ subdirectories.

> or having multiple versions of a package 
> installed at the same time.

Is this a joke? Installing several versions of a package is fragile,
it's a security mistake, and takes place on the hard disk for no real
use.

I'm currently fighting so that we stop supporting so many python
versions, and I hope we won't have to support several versions for each
package, each of them for several python versions! This would become a
complete nightmare.

The whole purpose of eggs seems to be able to distribute whole projects,
with their dependencies, in a simple format. Sorry, but it goes in the
opposite direction of all Linux distributions. The whole point of
inter-package dependencies is to avoid this, and believe me, there are
lots of reasons to do that; otherwise, all binaries would be statically
linked, or packed with their libraries. This is how things work in the
Windows world, but dependencies were invented exactly to avoid it.

> Sure, the limitation might exist, but is it 
> fair to call Debian fragile or broken because of it?  Not a bit!  I've 
> therefore been very careful to describe any such tradeoffs that Debian 
> makes in neutral terms rather than categorically pejorative ones.  I would 
> prefer if you would extend me the same courtesy of not describing every 
> design tradeoff I make as being a "non-standard", "drawback", "for no good 
> reason".

Refusing to edit files upon installation, for example, is not a
tradeoff. It's a deliberate decision, based on the fact this is a bad
idea.

> If Debian adds this metadata marker for its non-setuptools Python packages, 
> then the Python packages will be "eggs" in the sense that other eggs will 
> be able to discover them via the pkg_resources API, and thus TurboGears 
> users will be able to use the Debian-supplied versions of ElementTree and 
> the like.

Why would we need to modify hundreds of packages to contain empty files,
just because another package cannot import its dependencies correctly?

> >I don't see why multi-version support necessarily requires to
> >increase sys.path. In the case of eggs, version dependencies are
> >expressed explicitly in the code (through require() calls),
> 
> Actually, they're expressed in the egg metadata, and the wrappers on a 
> project's scripts execute the require() calls, so that the code doesn't 
> have to contain explicit require() calls except for more-dynamic 
> situations, such as plugins and "optional extra features" that require 
> additional projects to be present.

I'm used to remove such code checking for the version of dependencies in
python packages, as it is broken most of the time. Introducing another
broken way to check the dependencies can be worked around the same way,
but it's sadly more burden on the package maintainers. Still, I can
assure you it's less burden than adapting the system to handle eggs, as
I understand them.

> Note that setuptools is in release 0.6a8 at the moment - it is obviously 
> not a polished product, but it provides enough functionality to be quite 
> useful to many Python developers.  To this point, directly working on 
> integration with external packaging tools has not been a focus, although I 
> always have given top priority to responding to questions and requests from 
> people working on integration with those tools (e.g. the volunteers who 
> worked on easy_deb and the Gentoo stuff).  I can't reasonably learn the 
> technical details of every packaging system, so it is best to let 
> volunteers familiar with individual packaging systems tell me what they 
> need in order to effectively wrap the system.

It seems to me that you haven't thought of integrating with anything
except Windows, from the very beginning. Eggs are similar to .jar
or .xpi files, that also come from the Windows world, and believe me,
it's a real pain to maintain a package with such files. They aren't
suited at all for Unix systems and you can expect many Debian
maintainers to find ways to distribute python packages the way they
should be distributed.

> Up until now, my interactions with such volunteers have been most pleasant 
> and positive.  To my knowledge, it's not usual for packaging system 
> developers to spew FUD at a project and look for ways to exclude or break 
> the work of developers who've chosen to use it.  I'm therefore more than a 
> little surprised by some of the attitude I've received.  I hope, though, 
> that we can get past that soon, if only because it means I'll have more 
> time to work on implementing and documenting whatever the resolution is.  ;)

If there is any implementation, it will be a crude hack. You'd better
think of a way for python to completely bypass all egg metadata and
still function properly. Then, we can think of ways to fill dependencies
at the .deb level automatically, but it's really less important.

Regards,
-- 
 .''`.           Josselin Mouette        /\./\
: :' :           josselin.mouette at ens-lyon.org
`. `'                        joss at debian.org
   `-  Debian GNU/Linux -- The power of freedom