[Distutils] Library instability on PyPI and impact on OpenStack

[Nick Coghlan]

On Tue, Mar 5, 2013 at 10:55 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Tuesday, March 5, 2013 at 7:50 AM, Mark McLoughlin wrote:
>
> I still don't really see how this is related to PEP426 unless PEP426
> has gotten
> a lot larger since I last looked at it. Where in particular a
> distribution gets
> installed is left up to the installers to sort out. And making sure
> that the installed
> versions exist in sys.path is similarly out of scope for PEP426.
>
>
> Sorry, maybe I'm being obtuse.
>
> I can see people read PEP426 and thinking "oh, awesome! Python now has a
> mechanism for handling incompatible API changes! Now I can get rid of
> that crufty backwards compat code and bump my major number!".
>
> My point is that it's (potentially) damaging to send that message to
> library maintainers before Python has the infrastructure for sanely
> dealing with parallel installs.
>
> Gotcha, you think that codifying how to version with regards to breaking
> API compatibility will lead to more people breaking backwards compatibility.
>
> That's a fair concern, and there's not much that can be done inside of
> PEP426
> to alleviate it. However I will say that PEP426 doesn't really contain much
> in the way of new ideas, but rather codifies a lot of existing practices
> within the
> Python community so that tools can get simpler and more accurate without
> having to resort to heuristics and guessing.
>
> You could also argue that this would _help_ with backwards compatibility
> because
> there is now a suggested way of declaring when 2 releases are no longer
> compatible
> by incrementing the major version number.

Yeah, I think if we can come up with a clear plan whereby distros can
create a suitable installation layout such that parallel versions can
be installed and imported, even for Python libraries that have no idea
parallel installation is possible, it should alleviate a lot of the
concerns. My main point is that most Python software assumes there
will only be one version of a given library on sys.path, so relying on
the libraries themselves to specify *at runtime* which version they
want isn't going to be workable. However, the install time metadata is
a much better candidate for doing something useful (and is also a bit
more amenable to being adjusted by the distro maintainers when it is
overly restrictive).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia