[Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 13 November 2017 at 18:57, Chris Barker <chris.barker at noaa.gov> wrote:
> This has gotten to be a big thread, and it's a pretty intractable problem,
> but I think there are a few fairly small things that could be done to at
> least make it a bit easier:

In principle, I agree with the ideas here, but there are some
practical issues that make them somewhat less straightforward than we
might like.

> 1) Add python2.exe and python3.exe files to the Windows installers -- am I
> insane or did Windows used to have that? I really think it did -- maybe got
> removed when py.exe was added.

I don't think Windows ever had python2.exe/python3.exe, but I could be wrong.

The only possible way we'd get a python2.exe is by adding it to future
releases of Python 2.7, and we shouldn't be recommending Python 2 for
new users anyway. And I'm strongly -1 on promoting "python3.exe" as
the canonical way of getting Python 3. The python3 command is a result
of the Unix issues around switching to Python 3 as the default, and we
shouldn't perpetuate that approach on Windows, where it's unneeded.

Having said that, I don't object to including versioned executables as
a convenience (if it is one). I just dislike promoting the use of them
via standard documentation.

>   1a) alternatively, we could add a "py" executable to the standard linux
> builds, so there would be THAT one way to do it. But I think that's a "BAD
> IDEA" -- the whole "py" thing is not widely know or used, it's not going to
> show up in package install instructions for a LONG time, (actualy we could
> do both anyway)

I think that getting a "py" launcher on Unix is a lost cause at this
stage. It would be nice, and maybe if the original PEP had proposed a
cross-platform py command, it might have worked, but that's history
now, and I think the py launcher will probably always be a
Windows-only thing.

> Then "python2 -m pip install" would work everywhere (only with new
> installations, but at least with newbies, that's a bit more likely ...)

No newcomer should *ever* be getting told to use "python2 -m pip
install". Maybe "python3 -m pip install", but never Python 2 (outside
of specialised environments where training people in an out of date
version is required). And of course not all Unix distributions come
with Python 3 installed (Mac OS being an obvious example, and I think
Nick mentioned CentOS and RHEL) so python3 won't work everywhere
either...

> 2) Make adding to the PATH in Windows the default. I think there are really
> three user groups:
>
>    - newbies starting from scratch -- they want it on the PATH
>
>    - newbies with whatever left over cruft from previous installations on
> their systems -- they want it at the FRONT of their PATH. They SHOULD
> uninstall all the cruft, but if they don't this will still work with as few
> surprises a possible.
>
>    - not-newbies with a previous version of python they need to continue
> using. They can uncheck the box, or use py.exe

Unfortunately, adding Python to the *front* of PATH is not possible.
It might be the best option, but it simply cannot be done. A per-user
install of Python (the default in Python 3.5+) does not have the
privileges to add items to the front of the system PATH, so the best
we can do is add it to the front of the user PATH. But the system PATH
always goes ahead of the user PATH, and system installs (for example,
default installs of Python 2.x or 3.4 or earlier) will come before
that. (Making a system install of Python be the default won't work
either, as then pip won't work from a normal prompt and we have the
same issues on Windows that Unix users have that results in people
doing "sudo pip install" with all the issues that brings).

So for (1) I agree. For (2) it simply isn't possible. For (3) you're
getting beyond newcomer so sure, "manually handle it" is a reasonable
option.

There's also the problem that file associations (i.e., what does
double clicking on a .py/.pyw file do) don't follow the same rules, as
they go through the launcher.

The only really sensible "do what I expect" situation is a single
Python 3 installation on the user's machine. For that case, adding
Python to PATH (it doesn't matter where as there's only one) is a
sensible option. The downside is that optimising for that case makes
the behaviour for other (more experienced) users worse. But it's not
unreasonable to make that trade-off.

> 3) Make --user be be automatic for pip install. Not actually the default,
> but pip could do a user install if you don't have the permissions for a
> non-user install.

The problem here is that the user scripts directory isn't on PATH. We
could put it on, but again we hit the "goes after the system PATH"
problem (on Windows).

> This means folks might accidentally install in user mode because they forgot
> to type sudo -- but that would be a mostly-sysadmin/sophisticated user
> problem. And maybe have an environment variable of configuration key for
> "prefer admin install". If tha was set, pip would only install in user mode
> if specifically asked to. I'm can't imagine a case where a user would have
> admin permissions, but want a user install (except people following bad
> practices!)

The various options here are under discussion on the pip tracker.
Suffice it to say, it's not a clear-cut change, but it would ideally
be something we do. Nobody really objects to it, but there are
significant backward compatibility and behaviour change issues to
address first.

> Except for the pip change, these would be easy to implement and backward
> compatible. So why not?

See above. But in principle, if the issues can be solved in a way
we're all happy with, I agree with the principle that "download and
install Python" should result in an environment where

    python myscript.py
    pip install something_or_other

"just work" if the user previously had no Python on their PC. Anything
less results in a worse beginner experience for Python than for other
languages (which usually don't support having multiple versions
installed at once, and often require admin installs, so bypass most of
the problems Python has to deal with).

> NOTE: even if nothing changes with any of this we need to get py.exe better
> documented and advertised -- it doesn't show up in:
>
> https://docs.python.org/3/faq/windows.html#id2
>
> for instance.

It has a whole section at
https://docs.python.org/3/using/windows.html#python-launcher-for-windows.
And https://docs.python.org/3/using/windows.html discusses how to
handle PATH, so that's clearly documented too, even if (a) no-one
reads the docs, and (b) the defaults may not be the best choices.
Agreed that https://docs.python.org/3/faq/windows.html seems pretty
out of date, and could do with an update. I'm sure patches would be
gratefully accepted, if only to replace the out of date information
with pointers to the correct places in the main documentation.

Ultimately, though, most newcomers don't read the docs. They muddle
through, ask colleagues who may or may not know any better than they
do, read out of date articles on Stack Overflow and watch videos on
Youtube. The *only* way of stopping them doing that is to make
everything "just work" for them. And when one person can tell them to
download Python from python.org, another says "use Anaconda" and a
third suggests ActiveState, and they try all three while trying to
work out what they need, sometimes we just have to do the best we can
:-)

Paul