On 14 November 2017 at 16:47, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:

General summary so far
#######################

This debate has been very civil so far and quite productive, so thanks
to everyone involved.

Several issues and solution proposals have been mentioned, so I will
make a summary here.

Issue 1: running Python is a different command different OS when several
versions are installed
==================================================================================================

Proposal A:
-----------

Suffix Python executable on Windows like on Unix, so that people will
type pythonX.X if they want a specify version.

Pros: easy and discoverable.

Cons: you need a lot of stuff in the system path.

Con: we hope to have the problem resolved on the Linux distro side such that "python" typically means "python" by the time community support for Python 2 ends in 2020. Since Windows has gone the better part of two decades without version Python commands, adding them because we're impatient with the pace of change at the Linux distro level doesn't really make sense (especially when Linux holds such a small fraction of the non-phone client device market).

Proposal B:
------------

Provide the "py" command on Unix, so that people will type "py -x.x"

Pros: no more issues with system path. Just add this command, and
register python installations to be listed by "py".

Cons: very few people know about "py", even on windows. This would need
lots of communication. Harder to change Mac and majors Linux distros
setup than just the next windows installer.

Proposal C:

Make sure 'python' just works on a default Window user install, and treat any system where that doesn't work by default as a special case that needs troubleshooting.

Pro: a solid troubleshooting guide will be useful regardless

Con: mainstream Linux distros will still be a consistent exception for at least the next couple of years (and probably longer for the LTS variants)

Issue 2: there is no one and only one obvious way to do venv / pip
===================================================================

Because pip and venv are tied to a Python version, there are numerous
ways to call them.

Just for pip, you have "pip", "py -x.x -m pip", "pythonx.x -m pip" and
"pipx".

Proposal A:
-----------

Decide on one form, and promote it everywhere.

Pros: simple.

Cons: long and tedious process, with a huge inertia before all 3rd party
to it. Would need a "PEP 8 for Python command". Requires "issue 1" to be
solved before.

This downside doesn't apply as long as the form we standardise on is "pip install module".

That already works in a lot of contexts, and the cases where it doesn't work are typically ones where "python -m pip" may also be a problem.

Given a solid troubleshooting guide for issue 1, we may also be able to add some implicit checks to pip that provide alerts for problematic configurations, like checking:

    >>> user_scheme = "posix_user" # or "nt_user" or "osx_framework_user"
    >>> sysconfig.get_path("scripts", scheme="posix_user") in os.getenv("PATH").split(os.pathsep)
    True

The latter check returning false is a sign that the `PATH` configuration might be a problem.

Proposal B:
------------

Provide a pipenv-like behavior

pipenv is a tool created by the author of "requests" that automatically
"do the right thing" by default. The first time you install a package,
it creates a virtualenv for the current directory (but don't store it in
the current directory). It will then install all packages in there.
"pipenv shell" starts a shell in the virtualenv. Dependencies are
automatically dumped in a requirement files separating dev and prod.

Pros: bypass the multiple installers issue by prompting you what version
you wish the first time it creates a virtualenv. Solve the problem of
admin rights and system path.

Cons: lots of work. Either a new tool or breaking compat with pip. If
debian didn't install pip by default, this may not help.

This is essentially the path that conda takes, and it works well for folks that are willing to adopt conda's specific approach to managing your Python runtimes.

Issue 3: pip and virtualenv are not available everywhere
===================================================================

Proposal B:
------------

Make pip and venv part of the standard and request debian that they
provide it.

Pros: straight forward.

Cons: holy war in the making.

I don't think it's *that* bad. Debian does have support for weak dependencies, so the resolution here may be as simple as asking Matthias to add a Recommends from python3 to python3-venv so that ensurepip is present by default on typical workstation configurations, but can still be easily omitted from server configurations.

Favorite personnal combination
================================

1 - So far, I like "providing py on unix" better, but I think it's too
hard to do. So provide suffix on windows seems more achievable.

So +1 for having pythonX.X on windows.

I think this is actually similar to the situation with 'py' on *nix systems: with Windows having never had these before, adding and promoting them would be at best ineffective, and at worst outright confusing (especially if "python" starts meaning Python 3 by default on Linux as well within the next few years).

2 - Decide on one form, and promote it everywhere is again the simplest
IMO. I love pipenv but the strings to pull to get something like this
means it probably would not happen.

We've been evaluating a potential pipenv tutorial for the Python Packaging User Guide, and while we're likely going to add it soon, it will be as a new "application dependency management" tutorial alongside the existing ones, rather than as a direct replacement for the pip-based "package installation" tutorial: https://github.com/pypa/python-packaging-user-guide/issues/394#issuecomment-343343760

If we stick to suffix for issue 1, this mean we can promote:

pythonX.X -m pip
pythonX.X -m venv

Everywhere. In the docs. In the tutorials. In teaching classes.

This will help a lot to remove all confusions and make the Python start
up experience way easier.

Except it won't, because now you can't seamlessly update your class instructions to new versions of Python - you'll have to go through and change all the "X.Y" references to refer to the right version. Writing cross-platform scripts will also get even harder, as you wouldn't be able to say "I just need a version of Python, I don't really care which one" any more.

Python feature releases are already disruptive enough due to filesystem layout changes and file format changes - we don't need to make them worse by encouraging people to embed the exact target version in their documentation and helper scripts.

3 - getting angry debian devs is probably not wise. The "mock" pip/venv
command also plays very well with the 2 other solutions.

While I was a bit snarky about "python -m venv" being broken by default earlier in the thread, *please* don't take that as meaning that we can't collaborate constructively with the Debian folks. We can, and do, but the friction between language level package management and system level package management is one with a *long* history that we're all working towards resolving together (See [1] & [2] for write-ups of a couple of linux.conf.au talks I've given about the problem), as is the question of how best to handle the `/usr/bin/python` symlink.

It means we can promote everywhere:

pythonX.X -m cmd

It will work most of the time.

The combination of:

pip install package

python -m venv

already works in most cases, *except* apparently the critical one of "New Python user on Windows downloads the python.org installer and clicks through all the buttons without changing any settings".

So I think the main near term step forward would be to convince Steve Dower (as the Windows installer maintainer) to change that default behaviour yet again for 3.7, and then work towards coming up with a solid environment troubleshooting guide to include on packaging.python.org.

Cheers,

Nick.

[1] https://lwn.net/Articles/580399/

[2] https://lwn.net/Articles/711906/

Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia