[Python-Dev] Python install layout and the PATH on win32 (Rationale part 2: Moving the python.exe)

[VanL]

[PART 2: Moving the python binary]

There are two proposals on the table: 1) Regularize the install layout, 
and 2) move the python binary to the binaries directory. This email 
deals with the second issue exclusively. This has been the more
contentious issue.

2) Moving the Python exe:

A regular complaint of those new to Python on windows (and new to 
programming generally) has been that one of the first things that they 
need to do is to edit the PATH to allow Python to be run. In particular, 
this is the difficult sequence:

1. Install python.
2. Open up a shell and run "python"
3. Use pip or easy_install to install regetron (a package that installs 
an executable file).
4. Run regetron.

For step #2, the python exe needs to be on the PATH. For steps 3 and 4, 
the binaries directory needs to be on the PATH. Currently, neither of 
these are true, so the path needs to be edited to include both the 
python root (where python.exe is, for step 2) and the "Scripts" 
(hopefully soon "bin") directory where "pip" and "regetron" are (for 
steps 3 and 4). You can substitute "regetron" for "nose," "cython," or 
other packages as well.

MvL asked why anyone would want to run python directly from a cmd shell 
instead of running it from the start menu. There are two immediate 
responses to that: 1) observed behavior is that people prefer to run 
"python" from the cmd shell when developing (as observed by various 
people teaching Python, including Brian Curtin in this thread), and 2) 
running python or python programs from the shell is sometimes the only 
way to get a proper traceback when developing, making it a better way to 
work.

The proposal here is to move the python.exe into the binaries directory 
(whatever it is called) and add an option to the windows installer to 
add that one directory to the PATH on install (and clean up the PATH on 
uninstall). A new registry key would be added pointing to the location 
of the python binary (wherever it is).

Brian Curtin suggested this part of the proposal and has implemented it 
in a branch. MvL suggested a gradual transition to this over a 
three-release period.

Open Issues:

The PEP 397 Installer: As pointed out by Paul Moore, it may not matter 
once PEP 397 lands if python.exe is in the PATH or not - and it may be 
better if it is not. As he put it:

"""If we do put python.exe on PATH (whether it's in bin or not), we have
to debate how to handle people having multiple versions of python on
their machine. In a post-PEP 397 world, no Python is "the machine
default" - .py files are associated with py.exe, not python.exe, so we
have to consider the following 3 commands being run from a shell
prompt:

     1. myprog.py
     2. py myprog.py
     3. python myprog.py

1 and 2 will always do the same thing. However, 3 could easily do
something completely different, if the Python in the #! line differs
from the one found on PATH. To me, this implies that it's better for
(3) to need explicit user action (setting PATH) if it's to do anything
other than give an error. But maybe that's just me. I've been hit too
often by confusion caused by *not* remembering this fact."""

One possible response here is that the moving of the python.exe binary 
and the setting of the PATH would be tied to an unchecked-by-default 
installer option, making an explicit user choice needed to invoke the 
new functionality.

Breakage of existing tools: Mark Hammond, Paul Moore, and Tim Golden 
have all expressed that they have existing tools that would break and 
would need to be adjusted to match the new location of the python.exe, 
because that location is assumed to be at the root of the python install.

A related issue is that this portion of the proposal has met with some 
resistance, but not much support here on Python-dev. The reason for that 
is selection bias: Those who are on Python-dev are much more likely to 
have tools that do advanced things with Python, such as introspect on 
the location of the binary, and are also much more likely to be 
comfortable with things like editing the PATH on windows. In contrast, 
the people that have trouble with this issue are those that are newest 
to Python and programming generally - those for whom editing the PATH is 
a challenge and whom are likely to be confused by the distinction 
between python.exe and a python program - and why, even after they add 
python to the path, the python program is not directly executable.