[Distutils] "just use debian"

[Toshio Kuratomi]

David Cournapeau wrote:
> Josselin Mouette wrote:
>> Indeed, and the reason is that *functions never disappear from the
>> glibc*.
> 
> Yes and no. If you remove a function, you're indeed screwed, because you
> can't handle versioning in the header. But you can handle versioning in
> libraries at the link step, and file name of the library is an
> implementation detail of this versioning.
> 
I'm not 100% certain but I think that Josselin is speaking of glibc in
particular here and you're speaking of c libraries in general.

>> I don’t think Mono causes issues, but I’m pretty sure that allowing
>> multiple versions like the GAC allows *will* causes issues. Not for
>> purely-packaged things, where you can safely ignore those directory
>> renames, but if you start mixing distributed packages and
>> user-downloaded stuff, they will run into the same issues we have with
>> setuptools.
> 
> Please read the article carefully, it is not only about the GAC. It does
> handle the two conflicting issues: API stability installed globally vs
> easiness of deployment. That's why it is an interesting read IMHO: it
> addresses both issues. I don't think there is a single chance to see
> something as strong as C for python, because it would severely undermine
> the whole idea of the language used for prototyping.
> 
Mono is absolutely horrid in this regard.  Those who care about mono
(not our most persuasive speakers, I'm afraid) have asked upstream to
stop making that a best practice for Mono applications.

I've said before that ideally a Linux distribution only wants one
version of a library.  In Fedora we're willing to have compat packages
that hold old API versions if we must but by and large we would rather
help upstream apps port their applications forward than to have
compatibility packages.  This is because upstream for the library will
always be focusing on the newer versions of the libraries, not the older
versions.  If applications stay stuck on older versions, we end up
having to support libraries by ourselves with no upstream to help us
with the old version.

As much as I'd rather not have compat packages, having private versions
of third party libraries as advocated in that Mono document is worse.

The primary problem is security.  If a distro allows application to have
their own private copies of libraries and a security flaw is discovered
we're going to hate life.  We'll have to:

1) Find what packages include that library.  Unlike when the link goes
to a system installed library, this will not cause a dependency between
packages.  So we can't just query the package metadata to find out what
packages are affected.

2) Fix all the versions in all the packages.  Because each package
includes its own version of the library, there are multiple versions of
the library in these packages.  If we're unlucky the change will be
conceptual and we'll have to fix lots of different looking code.

3) Push rebuilds of all the fixed packages out that our users have to
download.  There's PR involved here: Security fix to Library Foo vs
Security Fix to Library Foo, Application, Bar, Baz, [...] Zod.  There's
also the burden for the users to download the packages.

Compare this with having to fix a set of compat library packages that's
not included in other applications:

1) Find all the libraries in the affected set.  It will probably be
enough to look by package name since these will be like:
python-foo1-1.0, python-foo2-2.2, python-foo-3.0

2) Fix the library (probably with help from upstream) and the
compat-libraries (maybe with upstream help or maybe on our own).

3) Push rebuilds of the library packages for our users to download.

Another concern is licensing.  Anytime a package includes other, third
party modules, the licensing situation becomes more complex.  Are the
licensing terms of any of the works being violated?  How do we have to
list the licensing terms in the package?  Are licensing terms for all
the packages available?  is everything open source? (believe it or not,
we do find non-OSS stuff in third party directories when we audit these
bundled packages :-(

Another concern is not giving back to upstream.  Once a package starts
including its own, private copies of a library it becomes more and more
tempting for the package to make bug fixes and enhancements on its own
copy.  This has two serious problems: 1) It becomes harder to port the
application forward to a new version because this is no longer what
upstream has shipped at any time.  2) The changes may not get back to
upstream at all.  Those bug fixes and feature enhancements may end up
being only part of this package, even though the whole community would
benefit.

Another concern is build scripts that become tied to building
with/installing the private versions.  Distributions have policies on
inclusion of third party libraries in another application.  Sometimes
upstream has a reason to include a copy of a library for compatibility
on Windows or for customers who aren't going to get it from their
distribution.  In these cases, if the distribution can give a command to
the build scripts to not build or install the compat library, all is
still well.  However, using system libraries often bitrots in an
upstream's build scripts because they start caring more about the pegged
library version they control than about making things work with system
libraries.  This makes more work for the packager.

Another concern is shipping of prebuilt-binaries.  Just before Fedora 9
came out we had to go through our Mono packages, get rid of some, and do
extensive work on the build scripts of others.  This was because some
packagers hadn't been watching their packages very well and upstream had
started shipping prebuilt versions of the third party modules they
required.  For upstream, they were making things easier for their
end-users.  For us, we had to assure that everything was built from
source that was auditable and available if needed (for instance, if one
of those third party libraries had a security flaw).

Another case is upstream reluctance to take patches to forward port the
application.  Unfortunately, when upstreams start including their own,
known good versions of libraries inside their packages, they sometimes
become reluctant to forward port to a new version of the library.  For
them, it's moving from a version that they know about to an untested
version from upstream.  For distributions, which have a vested interest
in helping upstreams forward port so they have only one version of a
library to maintain in the distro, this is an impediment to helping
upstream by providing patches to forward port.

These are some of the reasons that packaging Mono applications is
something I personally avoid in Fedora :-)

Please, do not go down this road with python.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20081001/2463ee81/attachment.pgp>