<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body><div><div style="font-family: Calibri,sans-serif; font-size: 11pt;">I consider making stdlib modules "optional" like this to be completely separate from making them individually versioned - can't quite tell whether you guys do as well?<br><br>To everyone: please don't conflate these two discussions. The other is about CPython workflow and this one is about community/user expectations (I have not been proposing to remove stdlib modules at any point).<br><br>Cheers,<br>Steve<br><br>Top-posted from my Windows Phone</div></div><div dir="ltr"><hr><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">From: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;"><a href="mailto:encukou@gmail.com">Petr Viktorin</a></span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">Sent: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;">7/6/2016 2:04</span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">To: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;"><a href="mailto:steve@pearwood.info">Steven D'Aprano</a>; <a href="mailto:python-dev@python.org">Python-Dev</a></span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">Subject: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;">Re: [Python-Dev] Breaking up the stdlib (Was: release cadence)</span><br><br></div>On 07/06/2016 05:11 AM, Steven D'Aprano wrote:<br>> On Tue, Jul 05, 2016 at 08:01:43PM +0200, Petr Viktorin wrote:<br>> <br>>> In the tkinter case, compiling for source is easy on a developer's<br>>> computer, but doing that on a headless server brings in devel files for<br>>> the entire graphical environment.<br>>> Are you saying Python on servers should have a way to do turtle<br>>> graphics, otherwise it's not Python?<br>> <br>> That's a really good question.<br>> <br>> I don't think we have an exact answer to "What counts as Python?". It's <br>> not like EMCAScript (Javascript) or C where there's a standard that <br>> defines the language and standard modules. We just have some de facto <br>> guidelines:<br>> <br>> - CPython is definitely Python;<br>> - Jython is surely Python, even if it lacks the byte-code of CPython and <br>> some things behave slightly differently;<br>> - MicroPython is probably Python, because nobody expects to be able to <br>> run Tkinter GUI apps on an embedded device with 256K or RAM;<br>> <br>> but it's hard to make that judgement except on a case-by-case basis.<br>> <br>> I think though that even if there's no documented line, most people <br>> recognise that there are "core" and "non-core" standard modules. dis and <br>> tkinter are non-core: if µPython leaves out tkinter, nobody will be <br>> surprised; if Jython leaves out dis, nobody will hold it against them; <br>> but if they leave out math or htmllib that's another story.<br><br>For MicroPython, I would definitely expect htmllib to be an optional<br>add-on – it's not useful for reading data off a thermometer saving it to<br>an SD card. But I guess that's getting too deep into specifics.<br><br>> So a headless server can probably leave out tkinter; but a desktop <br>> shouldn't.<br><br>Up till recently this wasn't possible to express in terms of RPM<br>dependencies. Now, it's on the ever-growing TODO list...<br><br>Another problem here is that you don't explicitly "install Python" on<br>Fedora: when you install the system, you get a minimal set of packages<br>to make everything work, and most of Python is part of that – but<br>tkinter is not. This is in contrast to python.org releases, where you<br>explicitly ask for (all of) Python.<br>Technically it would now be possible to have to install Python to use<br>it, but we run into another "batteries included" problem: Python (or,<br>"most-of-Python") is a pretty good battery for an OS.<br><br>Maybe a good short-term solution would be to make "import tkinter" raise<br>ImportError("Run `dnf install tkinter` to install the tkinter module")<br>if not found. This would prevent confusion while keeping the status quo.<br>I'll look into that.<br><br><br>> [...]<br>>>> The other extreme is Javascript/Node.js, where the "just use pip" (or <br>>>> npm in this case) philosophy has been taken to such extremes that one <br>>>> developer practically brought down the entire Node.js ecosystem by <br>>>> withdrawing an eleven line module, left-pad, in a fit of pique.<br>>>><br>>>> Being open source, the damage was routed around quite quickly, but <br>>>> still, I think it's a good cautionary example of how a technological <br>>>> advance can transform a programming culture to the worse.<br>>><br>>> I don't understand the analogy. Should the eleven-line module have been<br>>> in Node's stdlib? Outside of stdlib, people are doing this.<br>> <br>> The point is that Javascript/Node.js is so lacking in batteries that the <br>> community culture has gravitated to an extreme version of "just use <br>> pip". I'm not suggesting that you, or anyone else, has proposed that <br>> Python do the same, only that there's a balance to be found between the <br>> extremes of "everything in the Python ecosystem should be part of the <br>> standard installation" and "next to nothing should be part of the <br>> standard installation".<br>> <br>> The hard part is deciding where that balance should be :-)<br><br>I think the balance is where it needs to be for CPython, and it's also<br>where it needs to be for Fedora. The real hard part is acknowledging<br>that it needs to be in different places for different use cases, and<br>making sure work to support the different use cases is coordinated.<br><br>So, I guess I'm starting to form a concrete proposal:<br><br>1) Document what should happen when a stdlib module is not available.<br>This should be an ImportError informative error message, something along<br>the lines of 'This build of Python does not include SQLite support.' or<br>'MicroPython does not support turtle' or 'Use `sudo<br>your-package-manager` install tkinter` to install this module'.<br><br>2) Document leaf modules (or "clusters") that can be removed from the<br>stdlib, and their dependencies. Make no guarantees about cross-version<br>compatibility of this metadata.<br><br>3) Standardize a way to query which stdlib modules are present (without<br>side effects, i.e. trying an import doesn't count)<br><br>4) Adjust pip to ignore installed stdlib modules that are present, so<br>distributions can depend on "statistics" and not "statistics if<br>python_ver<3.4". (statistics is just an example, obviously this would<br>only work for modules added after the PEP). For missing stdlib modules,<br>pip should fail with the message from 1). (Unless the "pip upgrade<br>asynciio" proposal goes through, in which case install the module if<br>it's upgradable).<br><br>5) Reserve all stdlib module names on PyPI for backports or<br>non-installable placeholders.<br><br>6) To ease smoke-testing behavior on Pythons without all of the stdlib,<br>allow pip to remove leaf stdlib modules from a venv.<br><br><br>Looks like it's time for a PEP.<br><br><br>_______________________________________________<br>Python-Dev mailing list<br>Python-Dev@python.org<br>https://mail.python.org/mailman/listinfo/python-dev<br>Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org<br></body></html>