[Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)

[Nick Coghlan]

On 21 May 2015 at 08:46, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 21 May 2015 at 03:37, Chris Barker <chris.barker at noaa.gov> wrote:
>> As such, it _could_ play the role that pip+wheel (secondarily pypi) play in
>> the python ecosystem.
>
> In practice, it can't, as conda is entirely inappropriate as an input
> format for yum/apt/enstaller/zc.buildout/pypm/MSI/etc. In many ways,
> the barriers that keep conda from being a viable competitor to pip
> from an upstream perspective are akin to those that felled the
> distutils2 project, while the compatible-with-the-existing-ecosystem
> d2to1 has seen far more success.

I think I've finally figured out a short way of describing these
"packaging ideas that simply won't work": if an ecosystem-wide
packaging proposal doesn't work for entirely unmaintained PyPI
packages, it's likely a bad proposal.

This was not only the fatal flaw in the previous distribute/distutils2
approach, it's the reason we introduced so much additional complexity
into PEP 440 in order to preserve compatibility with the vast majority
of existing package versions on PyPI (over 98% of existing version
numbers were still accepted), it's one of the key benefits of
separating the PyPI-to-end-user TUF PEP from the dev-to-end-user one,
and it's the reason why the "Impact assessment" section is one of the
most important parts of the proposal in PEP 470 to migrate away from
offering the current link spidering functionality
(https://www.python.org/dev/peps/pep-0470/#id13).

Coping with this problem is also why injecting setuptools when running
vanilla distutils projects is one of the secrets of pip's success: by
upgrading setuptools, and by tweaking the way pip invokes setup.py
with it injected, we can change the way packages are built and
installed *without* needing to change the packages themselves.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia