Proliferation of Python packaging formats

[Mats Wichmann]

On 11/17/21 07:44, Chris Angelico wrote:
> On Thu, Nov 18, 2021 at 12:20 AM Skip Montanaro
> <skip.montanaro at gmail.com> wrote:
>>
>> Is the proliferation of packaging formats in Python as nutzo as this author
>> believes?
>>
>> https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html
>>
>> Asking because I've never been in the business of releasing "retail" Python
>> applications or packages.
>>
> 
> It's a problem if you feel the need to release in every single
> possible way. The XKCD in that example is a showcase of what happens
> when you attempt to install in every possible way, which is usually a
> consequence of everyone releasing in just one way.
> 
> He's blaming the PSF, but that's not really fair. The PSF has never
> said "hey everyone, make sure you package for conda as well as posting
> on PyPI". Part of the proliferation comes because there are multiple
> completely independent distributions, and in the XKCD shown there, he
> has installed Python at least five times from completely different
> sources (the OS-provided one, two from Homebrew, one from Anaconda,
> and one from python.org), and maybe installed pip into one of those as
> well. Duh, of course that's going to cause problems.

There are people working on the problems, but the author doesn't seem to 
like the progress being made. Here's one jumping point for those 
interested who haven't otherwise followed this stuff:

https://discuss.python.org/t/graceful-cooperation-between-external-and-python-package-managers-pep-668/10302

Some of the problems in Python ecosystem, which categorically does *not* 
just consist of machines running a Linux distro, are in fact caused by 
the choices the Linux distros make in packaging. The model of "get 
everything you need from your distro's packages" doesn't come close to 
cutting it these days, so there has to be coexistence.  Anecdotally, I 
have a module I use in a couple of projects and I forgot that on my main 
machine I actually had that supplied by a Fedora package (silly me), and 
surprise - it's been dropped in F35.

Yes, it's true that the PyPA does not set itself up to be the final 
picker of winners, so this comment isn't just off the wall:

"P.S. PEP-517 and 518 are a start, but are very disappointing in how 
little they address distro problems. These PEPs are designed to tolerate 
the proliferation of build systems, which is exactly what needs to stop. 
Python ought to stop trying to avoid hurting anyone’s feelings and pick 
one."

That doesn't seem likely to happen, and in fact there's probably 
strength in having multiple implementations of a PEP (in other worlds, 
like W3C, multiple implementations are mandatory before a standard can 
be accepted).