[Python-Dev] Breaking up the stdlib (Was: release cadence)

[Petr Viktorin]

(sorry if you get this twice, I dropped python-dev by mistake)

On 07/05/2016 07:02 PM, Steven D'Aprano wrote:
> On Tue, Jul 05, 2016 at 09:53:24AM +0200, Petr Viktorin wrote:
> 
>> While we're on the subject, I'd like to offer another point for
>> consideration: not all implementations of Python can provide the full
>> stdlib, and not everyone wants the full stdlib.
>>
>> For MicroPython, most of Python's batteries are too heavy. Tkinter on
>> Android is probably not useful enough for people to port it. Weakref
>> can't be emulated nicely in Javascript.
>> If packages had a way to opt-out of needing the whole standard library,
>> and instead specify the stdlib subset they need, answering questions
>> like "will this run on my phone?" and "what piece of the stdlib do we
>> want to port next?" would be easier.
> 
> I don't know that they will be easier. That seems pretty counter- 
> intuitive to me. At the moment, answering these questions are really 
> easy if you use nothing but the std lib: the answer is, if you can 
> install Python, it will work. As soon as you start using non-stdlib 
> modules, the question becomes:
> 
> - have you installed Python? have you installed module X? and module Y? 
>   and module Z? do they match the version of the interpreter? where 
>   did you get them from? are you missing dependencies?
> 
> I can't tell you how much trouble I've had trying to get tkinter working 
> on some Fedora systems because they split tkinter into a separate 
> package. Sure, if I had *known* that it was split into a separate 
> package, then just running `yum install packagename` would (probably!) 
> have worked, but how was I supposed to know? It's not documented 
> anywhere that I could find. I ended up avoiding the Fedora packages and 
> installing from source.

Ah, but successfully installing from source doesn't always give you the
full stdlib either:

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_sqlite3
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.

I have missed that message before, and wondered pretty hard why the
module wasn't there.

In the tkinter case, compiling for source is easy on a developer's
computer, but doing that on a headless server brings in devel files for
the entire graphical environment.
Are you saying Python on servers should have a way to do turtle
graphics, otherwise it's not Python?

> I think there comes a time in every successful organisation that they 
> risk losing sight of what made them successful in the first place. (And, 
> yes, I'm aware that the *other* way that successful organisations lose 
> their way is by failing to change with the times.)
> 
> Yes, we're all probably sick and tired of hearing all the Chicken Little 
> scare stories about how the GIL is killing Python, how everyone is 
> abandoning Python for Ruby/Javascript/Go/Swift, how Python 3 is killing 
> Python, etc. But sometimes the sky does fall. For many people, Python's 
> single biggest advantage until now has been "batteries included", and I 
> think that changing that is risky and shouldn't be done lightly.
> 
> It's easy to say "just use pip", but if you've ever been stuck behind a 
> corporate firewall where pip doesn't work, or where dowloading and 
> installing software is a firing offence, then you might think 
> differently. If you've had to teach a room full of 14 year olds, and you 
> spend the entire lesson just helping them to install one library, you 
> might have a different view.

That is why I'm *not* arguing for shipping an incomplete stdlib in
official Python releases. I fully agree that including batteries is
great – I'm just not a fan of welding the battery to the product.
There are people who want to cut out what they don't need – to build
self-contained applications (pyinstaller, Python for Android), or to
eliminate unnecessary dependencies (python3-tkinter). And they will do
it with CPython's blessing or without.
I don't think Python can move to the mobile world of self-contained apps
without this problem being solved, one way or another.
It would be much better for the ecosystem if CPython acknowledges this
and sets some rules (like "here's how you can do it, but don't call the
result an unqualified Python").

> The other extreme is Javascript/Node.js, where the "just use pip" (or 
> npm in this case) philosophy has been taken to such extremes that one 
> developer practically brought down the entire Node.js ecosystem by 
> withdrawing an eleven line module, left-pad, in a fit of pique.
> 
> Being open source, the damage was routed around quite quickly, but 
> still, I think it's a good cautionary example of how a technological 
> advance can transform a programming culture to the worse.

I don't understand the analogy. Should the eleven-line module have been
in Node's stdlib? Outside of stdlib, people are doing this.