[Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)

[Nick Coghlan]

On 21 May 2015 at 03:37, Chris Barker <chris.barker at noaa.gov> wrote:
> As such, it _could_ play the role that pip+wheel (secondarily pypi) play in
> the python ecosystem.

In practice, it can't, as conda is entirely inappropriate as an input
format for yum/apt/enstaller/zc.buildout/pypm/MSI/etc. In many ways,
the barriers that keep conda from being a viable competitor to pip
from an upstream perspective are akin to those that felled the
distutils2 project, while the compatible-with-the-existing-ecosystem
d2to1 has seen far more success.

Rather than being strictly technical, the reasons for this are mostly
political (and partially user experience related) so it's not worth
the futile effort of attempting to change them. When folks try anyway,
it mainly serves to alienate people using (or working on) other
integration platforms rather than achieving anything productive (hence
my comment about the "one package manager to rule them all" attitude
of some conda proponents, although I'll grant they haven't yet gone as
far as the NixOS folks by creating an entirely conda based Linux
distro).

The core requirement for the upstream tooling is to be able to bridge
the gap from publishers of software components implemented in Python
to integrators of software applications and development environments
(regardless of whether those integrators are themselves end users,
redistributors or both). That way, Python developers can focus on
learning one publication toolchain (anchored by pip & PyPI), while
users of integrated platforms can use the appropriate tools for their
platform.

conda doesn't bridge that gap for Python in the general case, as it is
itself an integrator tool managed independently of the PSF and
designed to consume components from *multiple* language ecosystems and
make them available to end users in a common format.

Someone designing a *new* language ecosystem today could quite
reasonably decide not to invent their own distribution infrastructure,
and instead adopt conda as their *upstream* tooling, and have it be
the publication toolchain that new contributors to that ecosystem are
taught, and that downstream integrators are expected to interoperate
with, but that's not the case for Python - Python's far too far down
the distutils->setuptools->pip path to be readily amenable to
alternatives (especially alternatives that are currently still fairly
tightly coupled to the offerings of one particular commercial
redistributor).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia