[Distutils] The future of invoking pip

[Donald Stufft]

On November 11, 2015 at 9:34:41 AM, Nick Coghlan (ncoghlan at gmail.com) wrote:
> On 11 November 2015 at 15:08, Wayne Werner wrote:
> >
> > With all of the weirdness involved, it makes me wonder - could there be a
> > better way? If we waved our hands and were able to magically make Python
> > package management perfect, what would that look like?
> >
> > Would that kind of discussion even be valuable?
>  
> That's essentially what PEP 426 evolved into - an all-singing
> all-dancing wish list of what *my* dream packaging system would enable
> (especially once you include the "Deferred Features" section). In
> practice, most of that is "nice to have" rather than "absolutely
> essential" though, so we're in the midst of the process:
>  
> 1. Figuring out incremental steps that help us to get from "here" to
> "there" by way of formalising what already exists
> 2. Figuring out which parts of "there" represent needless complexity
> that can just be dropped entirely
>  
> Packaging systems are a uniquely difficult ship to steer (even moreso
> than programming language design), since interoperability is king, and
> you need to cope with legacy versions of both packaging tools *and*
> language runtimes.
>  

Right. I think PEP 426 fell into the same trap that distutils2 fell into. It attempted to boil the ocean in one step and the longer it went on the more aspirational stuff got layered onto it because it was being held up as the great hope for packaging.

I think the lessons we’ve learned is that careful [1] incremental improvements is the best way forward. It’s a lot easier to reason and handle a small change than it is to handle a massive change.

The other important lesson is that one of our ecosystem’s biggest strengths is also one of (and probably *the) biggest things holding us back from a large re-envisioning. We have a massive number of available packages that have accumulated over like two decades. There are half a million individual installable package files on PyPI and who knows how many in private repositories all around the world. The most ideal system in the world isn’t actually useful if it requires throwing out the entire existing ecosystem.

[1] Ones which don’t back us into corners as far as what path we are forced to go down into and which don’t add unneeded things because they would be “cool”. The Zen of Python has a great section on this, "Now is better than never, Although never is often better than *right* now.”.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA