[Python-ideas] Defining an easily installable "Recommended baseline package set"

[Paul Moore]

On 29 October 2017 at 07:54, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 29 October 2017 at 15:16, Guido van Rossum <guido at python.org> wrote:
>>
>> Why? What's wrong with pip install?
>
> At a technical level, this would just be a really thin wrapper around 'pip
> install' (even thinner than ensurepip in general, since these libraries
> *wouldn't* be bundled for offline installation, only listed by name).

I agree with Guido, this doesn't seem to add much as stated.

(From an earlier message of Nick's)
> At the same time, it would be beneficial to have a way to offer an even
> stronger recommendation to redistributors that we think full-featured
> general purpose Python scripting environments should offer the regex
> module as an opt-in alternative to the baseline re feature set, since that
> would also help with other currently difficult cases like the requests module.

This is key to me. The target area is *scripting*. Library and
application developers have their own ways of managing the dependency
handling problem, and they generally work fine. The people who don't
currently have a good solution are those who just use the system
install - i.e., people writing adhoc scripts, people writing code that
they want to share with other members of their organisation, via
"here's this file, just run it", and not as a full application. For
those people "not in a standard install" is a significantly higher
barrier to usage than elsewhere. "Here's a Python file, just install
Python and double click on the file" is a reasonable request. Running
pip may be beyond the capabilities of the recipient.

In my view, the key problems in this area are:

1. What users writing one-off scripts can expect to be available.
2. Corporate environments where "adding extra software" is a major hurdle.
3. Offline environments, where PyPI isn't available.

The solutions to these issues aren't so much around how we let people
know that they should do "pip install" (or "--install-recommended")
but rather around initial installs. To that end I'd suggest:

1. The python.org installers gain an "install recommended 3rd party
packages" option, that is true by default, which does "pip install
<whatever we recommend>". This covers (1) and (2) above, as it makes
the recommended package set the norm, and ensures that they are
covered by an approval to "install Python".
2. For offline environments, we need to do a little more, but I'd
imagine having the installer look for wheels in the same directory as
the installer executable, and leave it to the user to put them there.
If wheels aren't available the installer should warn but continue.

For 3rd party distributions (Linux, Homebrew, Conda, ...) this gives a
clear message that Python users are entitled to expect these modules
to be available. Handling that expectation is down to those
distributions.

Things I haven't thought through:

1. System vs user site-packages. If we install (say) requests into the
system site-packages, the user could potentially need admin rights to
upgrade it. Or get into a situation where they have an older version
in site-packages and a newer one in user-site (which I don't believe
is a well-tested scenario),
2. Virtual environments. Should venv/virtualenv include the
recommended packages by default? Probably not, but that does make the
"in a virtual environment" experience different from the system Python
experience.
3. The suggestion above for offline environments isn't perfect, by any
means. Better solutions would be good here (but I don't think we want
to go down the bundling route like we did with pip...?).

Paul