[Distutils] Library instability on PyPI and impact on OpenStack

[Nick Coghlan]

On Tue, Mar 5, 2013 at 10:34 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Tuesday, March 5, 2013 at 7:27 AM, Mark McLoughlin wrote:
>> If Python library maintainers will see PEP426 as a license to make
>> incompatible changes more often so long as they bump their major number,
>> then that perception will change.
>
> I still don't really see how this is related to PEP426 unless PEP426 has
> gotten
> a lot larger since I last looked at it. Where in particular a distribution
> gets
> installed is left up to the installers to sort out. And making sure that the
> installed
> versions exist in sys.path is similarly out of scope for PEP426.

Mark is worried that explicitly endorsing semantic versioning in PEP
426 will encourage package developers to gleefully break backwards
compatibility whenever they want to, so the situation quickly
degenerates into a mess of incompatible version requirements as you
move higher up the stack. If such a situation occurs, then it is a
potential problem for large applications like OpenStack (with a
deep-and-broad dependency stack) and especially for full Linux
distributions that are trying to jam large fractions of PyPI (plus
their own distro-specific code) into a single coherent dependency
tree.

That's actually the opposite of the intent, though - as I see it,
while most Python developers are doing the right thing and offering
reasonable deprecation periods, there are also some cases where
backwards compatibility is broken without suitable advance notice, or
people aren't aware of the implications of depending on 0.x releases,
or, as recently happened with the requests 0.9 -> 1.0 transition,
developers bump their minimum required version of a dependency in an
incompatible way. I don't expect a sudden stampede of "Hey, there's an
official way to indicate I'm breaking backwards compatibility, I'm
going to do it when I wouldn't have before!", no matter what any PEP
says.

However, I also see dependency management as primarily an integrator's
problem, and something to be solved primarily external to the
libraries themselves, through virtual environments, *.pth files and
appropriate installation layouts. The only responsibility I see as
lying with the upstream library and application developers is to
accurately declare their dependencies (and to make them as broad as is
reasonable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia