[Python-Dev] Standard library vs Standard distribution?

[Steve Dower]

On 29Nov2018 1330, Nathaniel Smith wrote:
> On Thu, Nov 29, 2018 at 10:22 AM Antoine Pitrou <antoine at python.org> wrote:
>>
>> Le 29/11/2018 à 19:07, Steve Dower a écrit :
>>> On 29Nov2018 0923, Antoine Pitrou wrote:
>>>> I think the whole argument amounts to hand waving anyway.  You are
>>>> inventing an extended distribution which doesn't exist (except as
>>>> Anaconda) to justify that we shouldn't accept more modules in the
>>>> stdlib.  But obviously maintaining an extended distribution is a lot
>>>> more work than accepting a single module in the stdlib, and that's why
>>>> you don't see anyone doing it, even though people have been floating the
>>>> idea for years.
>>>
>>> https://anaconda.com/
>>> https://www.activestate.com/products/activepython/
>>> http://winpython.github.io/
>>> http://python-xy.github.io/
>>> https://www.enthought.com/product/canopy/
>>> https://software.intel.com/en-us/distribution-for-python
>>> http://every-linux-distro-ever.example.com
>>>
>>> Do I need to keep going?
>>
>> I'm sure you could.  So what?  The point is that it's a lot of work to
>> maintain if you want to do it seriously and with quality standards that
>> would actually _satisfy_ the people for whom PyPI is not an option.
> 
> Yeah, I draw two conclusions from the list above:
> 
> - Paul expressed uncertainty about how many people are in his position
> of needing a single download with all the batteries included, but
> obviously he's not alone. So many people want a
> single-box-of-batteries that whole businesses are being built on
> fulfilling that need.
> 
> - Currently, our single-box-of-batteries is doing such a lousy job of
> solving Paul's problem, that people are building whole businesses on
> our failure.

I agree with these two conclusions, and for what it's worth, I don't 
think we would do anything to put these out of business - they each have 
their own value that we wouldn't reproduce.

Python's box of batteries these days are really the building blocks 
needed to ensure the libraries in these bigger distributions can 
communicate with each other. For example, asyncio has its place in the 
stdlib for this reason, as does pathlib (though perhaps the __fspath__ 
protocol makes the latter one a little less compelling). If the stdlib 
was to grow a fundamental data type somehow shared by 
numpy/scipy/pandas/etc. then I'd consider that a very good candidate for 
the stdlib to help those libraries share data, even if none of those 
packages were in the standard distro.

At the same time, why carry that data type into your embedded app if it 
won't be used? Why design it in such a way that it can't be used with 
earlier versions of Python even if it reasonably could be? We already do 
backports for many new stdlib modules, so why aren't we just installing 
the backport module in the newer version by default and running it on 
its own release schedule? I forget which one, but there was one backport 
that ended up being recommended in place of its stdlib counterpart for a 
while because it got critical bugfixes sooner. Installing pip in this 
manner hasn't broken anyone's world (that I'm aware of).

There's plenty of space for businesses to be built on being "the best 
Python distro for <X>", and if the stdlib is the best layer for enabling 
third-party libraries to agree on how to work with each other, then we 
make the entire ecosystem stronger.

(That simple agreement got longer than I intended :) )

Cheers,
Steve