[Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 10 November 2017 at 08:01, Nick Coghlan <ncoghlan at gmail.com> wrote:
> You can't have it both ways - the only way we can systematically mask
> the environmental differences between Windows, Linux and Mac OS X is
> by providing tooling that actually masks those differences, which
> means introducing that tooling becomes a prerequisite for teaching.

On Windows, which is the only platform I can reasonably comment on,
the killer issue is that the installer doesn't make the commands
"python" and "pip" available by default. Just checking my PC, both go
and rust (which I installed *ages* ago) do appear to, so they "just
work". Java sort-of also works like that (the runtime is on PATH, but
the compilers need to be added manually).

The biggest reason we don't add Python to PATH, as I understand it, is
because we need to consider the implications of people having multiple
versions of Python installed. As far as I know, no other language
allows that. But equally, most beginners wouldn't actually *have*
multiple versions installed. So maybe we should optimise for that case
(only one version of Python installed). That would mean:

1. Go back to adding Python to PATH. Because our installers don't say
"do you want to uninstall the old version", we should probably do a
check for a "python" command on PATH in the installer, and if there is
one, warn the user "You already have Python installed - if you are
upgrading you should manually uninstall the old version first,
otherwise your old installation will remain the default". We could get
more complex at this point, but that depends on what capabilities we
can include in the installer - I don't know how powerful the toolset
is.
2. Explicitly document that multiple Python interpreters on one
machine is considered "advanced", and users with that sort of setup
should be prepared to manage PATH themselves. I'd put that as
something like "It is possible to install multiple versions of Python
at once, but if you do that, you should understand the implications -
the average user should not need to do this".

We still have to deal with the fact that basically every Unix
environment is "advanced" in the above sense (the python2/python3
split). I don't have a solution for that (other than "upgrade to
Windows" ;-)).

> It doesn't completely solve the problem (as getting into and out of
> the environment is still platform specific), but it does mean that the
> ubiquitous online instructions to run "pip install package-name" and
> "python -m command" will actually work once people are inside their
> working environment.
>
> That tooling is venv:
>
> * it ensures you have "pip" on your PATH
> * it ensures you have "python" on your PATH
> * it ensures that you have the required permissions to install new packages
> * it ensures that any commands you install from PyPI will be also on your PATH
>
> When we choose not to use venv, then it becomes necessary to ensure
> each of those things individually for each potential system starting
> state

Currently, the reality is that people use virtualenv, not venv. All
higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
support). Enhancing the capabilities of venv is fine, but promoting
venv over virtualenv involves technical challenges across the whole
toolset, not just documentation/education.

But agreed, once we get people into a virtual environment (of any
form) the portability issues become significantly reduced. The main
outstanding issue is managing multiple environments, which could be
handled by having a special "default" environment that is the only one
we'd expect beginners to use/need.

> That said, we'd *like* the default to be is per-user package
> installations into the user home directory, but that creates
> additional problems:
>
> * "python" may be missing, and you'll have to use "python3" or "py" instead
> * "pip" may be missing (or mean "install for Python 2.7")
> * you have to specify "--user" on *every* call to pip, and most online
> guides won't say that
> * there's a major backwards compatibility problem with switching pip
> over to per-user package installs as the default (we still want to do
> it eventually, though)
> * on Windows, system-wide Python installs can't adjust per-user PATH
> settings, and per-user installs are subject to being broken by
> system-wide installs
> * on Windows, the distinction between a per-user install of Python,
> and per-user installs of a package is hard to teach
> * on Debian, I believe ~/.local/bin still isn't on PATH by default

Also on Windows, the per-user bin directory isn't added to PATH even
if you add the system Python to PATH in the installer.

> That said, I think there is one improvement we could feasibly make,
> which would be to introduce the notion of a "default user environment"
> into `venv`, such that there was a single "python -m venv shell"
> command that:
>
> * created a default user environment if it didn't already exist
> * launched a subshell with that environment already activated
>
> This wouldn't be a full environment manager like vex or pew - it would
> just be a way to bootstrap a single usable package management
> environment in a cross-platform way.

"How do I run venv?" is no easier to answer in a cross-platform way
than "how do I run pip?" You still need to be able to run
python/python3/py. About the only improvement is that there's no
legacy of documentation and advice on the web saying "run venv" like
there is for "run pip"...

Paul