[Python-Dev] PEP 594: Removing dead batteries from the standard library
Stephen J. Turnbull
turnbull.stephen.fw at u.tsukuba.ac.jp
Wed May 22 05:26:24 EDT 2019
Neil Schemenauer writes:
> Here is an alternative, straw man, proposal. Split the CPython repo
> into two parts:
> - core Python: minimal possible stdlib
> - everything else
I take issue with the characterization of "straw man," it's a
practical idea that turns out to be not so easy to implement.
We tried to do this with XEmacs, and found it just didn't work for us.
It requires a lot of effort to pare a distribution down to a minimal
core. (In fact, we never got close.) We gave up on that idea after
only a year, and this was an age and an environment that did not have
pip-like support for installation of 3rd-party packages (which you
deprecate as "not really a great solution"), and a lot of people were
still downloading at ISDN speeds or worse. The demand for a minimal
core was correspondingly greater than today, I think.
We did bundle most applications such as MUAs and even C/C++
development support into sumo packages (one for general use, one for
packages that didn't function at all without the "MULE" multilingual
environment which was optional in XEmacs). But we still ended up with
many packages staying in the core distribution, because other parts of
the core or the installation process used them, and as you say "If I
have a script that uses one of these modules and it gets removed, my
script breaks." That annoyed core developers as much as anyone, and
they generally had priorities that did not emphasize minimalism (to
say the least).
> When Python is released, provide installers for a Python that only
> includes the "core" part and a second installer that includes
> everything else. I realize this is more work for the release team.
> Hopefully with some scripting, it will not be too labour intensive.
Creating the distributions (core and sumo) was not particularly
labor-intensive for us because we mostly didn't distribute binaries.
A few moderately baroque scripts did the trick. Automation has
advanced, but due to the amount of it it's still fragile, especially
when you're producing turn-key binary installers.
As I mentioned above, it wasn't the release process that was the
bottleneck, it was other factors that made it hard to reduce the
core. We had an "xemacs-base" package that provided a set of "extra
tools" that some packages required, and libraries moved in and out of
it pretty much every release as some developer got tired of telling
people to install xemacs-base (back to core) or working around missing
features (move it to xemacs-base where it could be updated easily).
These are the same things that Python maintainers mention as pros and
cons of having things in stdlib or out on PyPI. We learned there is a
sweet spot. I know we didn't hit it most of the time, but it's very
clear that both extremes are hard, if not infeasible, to maintain.
(At the maximalist extreme, even GNU Emacs doesn't really try to
include everything written in Emacs Lisp any more.)
> To help the people who need 100s or 1000s of extra PyPI packages,
> we could develop a tool that creates a "sumo" Python installer,
> grabbing packages from PyPI and building a installer package.
This could be done by any third party (it wouldn't even need to be a
public distribution, it could easily be an enterprise that open
sources an internal tool or a GSoC project). And it could be hosted
on PyPI itself, since it's not the kind of thing that enterprises need
to install on 100,000 workstations.
> To install that package, you would not need network access.
AIUI, it's not the network access that is usually the problem these
days, although it can be annoying if net access is flaky or if you
have to sneakernet USBs around. Still, "sneakernet" is basically a
one-off cost. The problem is that collections of random software from
untrusted sources are verboten in the enterprise. (Ah, those Walnut
Creek CDs were wonderful! I wonder what a modern AV tool would find
in them?!) I suppose that in those situations people already have the
necessary tooling to roll the sumo or provide an intranet repository;
the problem is navigating the red tape.
> That doesn't need to happen right away. Also, maybe other Python
> distributions can fill that need if core Python devs don't want to
> build it.
I don't think this is about what core devs do or don't want to build.
I also don't feel a strong sense among the core developers that they
don't want to maintain a "batteries included" standard library. Sure,
there's Christian's PEP, but there's also deliberate activity to
promote more participation at the core (ie, committer) level. AIUI,
Christian's PEP is about transparency of what is already happening
(WONTFIX-ing or long periods of inactivity on issues with modules of
"low importance" to the active core developers), and a first step
toward making a managed process for the deprecation of modules we
can't afford to maintain given others we have committed to maintain.
More information about the Python-Dev