On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin wrote:

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...

can't you use python3 -m pip install ..... everywhere? That's what I tell my beginner students to do, and I've never had a problem. (nce they got Python installed right in the first place) For that: The python.og installers for Windows and Mac pretty much "just work" Linux is a different story, but Linux users are more comfortable with eh whole idea of command lines and packages, etc -- so my Linux users have never been the hangup. If/when I'm teaching data analysis for scientific computing, I go straight to conda, but for basic python, and most web Development, pyton.org python and pip work great. A also DO NOT introduce virtualenv right off the bat -- it is another complication that is critical to real development, but not important for learning python. I have done it it the past -- it did not go well.... What's too bad now is that so many docs say "pip install the_package", and users with multiple python installs can get burnt. (though most don't) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 2017-11-06 23:53, Ivan Pozdeev via Python-ideas wrote:

On 07.11.2017 1:48, Chris Barker wrote:

...

On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin mailto:desmoulinmichel@gmail.com> wrote:

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...

can't you use python3 -m pip install .....

everywhere? You can't. Windows versions don't create versioned executables. Got bitten with this myself.

Once you're worked around that in the first place (lesson: "this is how you start Python"), then figuring out whether you need "python -m pip" or "python3 -m pip" shouldn't be a big deal. Still the fact that the way you call Python is different on different platforms is rather unfortunate. -- Thomas

On 07.11.2017 18:34, Stephan Houben wrote:

One (smaller) suggestion on the PATH situation on Windows:

I noticed that Visual C++ Build Tools installs a number of  "Command prompts" under its Start menu item, each of which starts a cmd.exe with appropriate PATH set to the appropriate compiler (32/64 bits or ARM cross-compiler), and assorted environment variables set to the appropriate include/library directories.

Could we do something similar for Python?

I.e., Install under the "Python 3.6" start menu an additional "Python command prompt", which will start cmd.exe with an appropriate PATH so that python and pip run without further prefix.

That way, the installer still doesn't need to mess with global PATH and you can easily have multiple versions of Python, each with their own "Python command prompt" submenu.

At least for Windows users this would simplify the situation a bit.

Stephan

This suggestion is no different from environemnt activation scripts of anaconda/pyenv/virtualenv -- without the other benefits of virtualenv. IPython solves this by creating version-specific executables (even in Windows). I see no reason why executables of Python proper cannot do the same. In fact, the reason why Windows version went Py instead of this seems to rather be to solve a completely different problem -- the filetype association (when running a script via ShellExecute, it autodetects which Python version to invoke).

2017-11-06 23:53 GMT+01:00 Ivan Pozdeev via Python-ideas mailto:python-ideas@python.org>:

On 07.11.2017 1:48, Chris Barker wrote:

...

On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin mailto:desmoulinmichel@gmail.com> wrote:

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...

can't you use python3 -m pip install .....

everywhere?

You can't. Windows versions don't create versioned executables. Got bitten with this myself.

...Maybe they should? (This is python-ideas, after all ;-) )

-- Regards, Ivan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org mailto:Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ http://python.org/psf/codeofconduct/

-- Regards, Ivan

On 06/11/17 22:53, Ivan Pozdeev via Python-ideas wrote:

You can't. Windows versions don't create versioned executables. Got bitten with this myself.

You either use py -x.y -mpip or you can directly use pip.exe, pepx.exe or pipx.y.exe.

...Maybe they should? (This is python-ideas, after all ;-) )

-- Regards, Ivan

-- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence

On Fri, Nov 10, 2017 at 07:48:35AM +0100, Michel Desmoulin wrote:

On linux you can't pip install, you need --users, admin rights or a virtualenv.

Isn't it the same on Windows? For an admin-installed Python you need --users, admin rights or a virtualenv. And a user-installed Python on Windows is equivalen to a user-compiled Python on Linux -- pip installs packages to the user-owned site-packages directory. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On 7 November 2017 at 03:52, Michel Desmoulin wrote:

Le 06/11/2017 à 09:47, Nick Coghlan a écrit :

...

On 6 November 2017 at 16:47, Michel Desmoulin wrote:

...

I really want some people from this list to discuss here so we can find a way to either unify a bit the way we install and use pip, or find a way to express a tutorial that always works for people on the most popular platforms and spread the word so that any doc uses it.

https://docs.python.org/3/installing/#basic-usage is as close as we've been able to get to that for the time being.

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...

Which is why we advise getting into a virtual environment ASAP, such that the only platform specific thing folks necessarily need to learn to get started is how to get to that first working virtual environment.

- install pip manually on linux

s/Linux/Ubuntu/ Other distros (like Fedora) provide pip by default.

- make sure the system path is correctly set

Recent python.org Windows installers do this automatically, but there are unfortunately still lots of ways for things to go wrong.

Stuff that they will forget on the next install, or miss when changing plateform.

Yes, because computers are awful, and incredibly user hostile. We don't have a magic wand to wave to fix that.

And assume that stuff in any tutorial you make they know this stuff.

This is a strong barrier or entry IMO.

Sure, but it's not one we can readily fix - the user hostility of command line environments and the compromises we have to make to abide by platform conventions are in the hands of operating system vendors, and there's only so much we can do to paper over those distinctions when user lock-in and putting barriers in the way of cross-device portability is a core part of commercial OS vendors' business models. This is a big part of why mobile client devices with cloud backends are so popular, even for development purposes: they allow for a much simpler developer experience that avoids decades of accumulated cruft in the desktop operating system command line experience. Even there though, you're faced with the fact that once you choose a provider, whatever you do there will probably be locked into that provider and not transferable elsewhere. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 7 November 2017 at 13:06, אלעזר wrote:

On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan wrote:

...

On 7 November 2017 at 03:52, Michel Desmoulin wrote:

...

And assume that stuff in any tutorial you make they know this stuff.

This is a strong barrier or entry IMO.

Sure, but it's not one we can readily fix - the user hostility of command line environments and the compromises we have to make to abide by platform conventions are in the hands of operating system vendors, and there's only so much we can do to paper over those distinctions when user lock-in and putting barriers in the way of cross-device portability is a core part of commercial OS vendors' business models.

I don't know if you are referring to Microsoft Windows here, but I want to note that from my personal experience the Windows subsystem for Linux ("Bash on Ubuntu on Windows") is easy to work with, so making Windows feel (CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy it is for students to set up, but it is an option at least.

It is, but like any such approach (Cygwin is similar, in principle if not in execution) that makes one OS "look like" another, whether it's appropriate is very dependent on circumstances. Training potential Windows developers in a bash/Ubuntu style environment leaves them at a disadvantage when they need to develop for actual Windows environments. It's certainly an option, but so is "train them on Linux". Paul

Le 07/11/2017 à 14:06, אלעזר a écrit :

On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan mailto:ncoghlan@gmail.com> wrote:

On 7 November 2017 at 03:52, Michel Desmoulin mailto:desmoulinmichel@gmail.com> wrote:

> And assume that stuff in any tutorial you make they know this stuff. > > This is a strong barrier or entry IMO.

Sure, but it's not one we can readily fix - the user hostility of command line environments and the compromises we have to make to abide by platform conventions are in the hands of operating system vendors, and there's only so much we can do to paper over those distinctions when user lock-in and putting barriers in the way of cross-device portability is a core part of commercial OS vendors' business models.

I don't know if you are referring to Microsoft Windows here, but I want to note that from my personal experience the Windows subsystem for Linux ("Bash on Ubuntu on Windows") is easy to work with, so making Windows feel (CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy it is for students to set up, but it is an option at least.

It doesn't solve anything: - requires windows 10 - requires having the several Go install first - mix windows and linux, making the whole thing confusing to beginers - we still have the heterogenous setup between the 2 os being a problem. I love WSL, but it's not a solution to that particular problem.

Le 07/11/2017 à 14:08, Paul Moore a écrit :

On 7 November 2017 at 12:44, Nick Coghlan wrote:

...

...

- make sure the system path is correctly set

Recent python.org Windows installers do this automatically, but there are unfortunately still lots of ways for things to go wrong.

I believe the latest installers switch it off again, because one of the ways things can go wrong is that stuff put at the start of the user path is still lower priority than stuff in the system path, and we now default to user installs. This is an actual problem with mixed python.org and Anaconda installations, for example - Anaconda adds itself to the system PATH, and overrides a default user install of python.org Python. So you can't prioritise python.org over Anaconda without manual path hacking. (This hit me when I installed Visual Studio and selected "include Python (Anaconda)" - I can't recall the exact option, but it broke my python.org install).

So what you are saying is that even if you do the right things, something the user may do could possibly reverse it. I don't see that as a reason not do to the right thing.

On 10 November 2017 at 17:05, Michel Desmoulin wrote:

...

Which is why we advise getting into a virtual environment ASAP, such that the only platform specific thing folks necessarily need to learn to get started is how to get to that first working virtual environment.

You can't start by teaching virtualenv. I tried. It doesn't work. And it's a terrible prerequisit if you write docs, tutorial, teaching materials, etc.

You can't have it both ways - the only way we can systematically mask the environmental differences between Windows, Linux and Mac OS X is by providing tooling that actually masks those differences, which means introducing that tooling becomes a prerequisite for teaching. It doesn't completely solve the problem (as getting into and out of the environment is still platform specific), but it does mean that the ubiquitous online instructions to run "pip install package-name" and "python -m command" will actually work once people are inside their working environment. That tooling is venv: * it ensures you have "pip" on your PATH * it ensures you have "python" on your PATH * it ensures that you have the required permissions to install new packages * it ensures that any commands you install from PyPI will be also on your PATH When we choose not to use venv, then it becomes necessary to ensure each of those things individually for each potential system starting state That said, we'd *like* the default to be is per-user package installations into the user home directory, but that creates additional problems: * "python" may be missing, and you'll have to use "python3" or "py" instead * "pip" may be missing (or mean "install for Python 2.7") * you have to specify "--user" on *every* call to pip, and most online guides won't say that * there's a major backwards compatibility problem with switching pip over to per-user package installs as the default (we still want to do it eventually, though) * on Windows, system-wide Python installs can't adjust per-user PATH settings, and per-user installs are subject to being broken by system-wide installs * on Windows, the distinction between a per-user install of Python, and per-user installs of a package is hard to teach * on Debian, I believe ~/.local/bin still isn't on PATH by default That said, I think there is one improvement we could feasibly make, which would be to introduce the notion of a "default user environment" into `venv`, such that there was a single "python -m venv shell" command that: * created a default user environment if it didn't already exist * launched a subshell with that environment already activated This wouldn't be a full environment manager like vex or pew - it would just be a way to bootstrap a single usable package management environment in a cross-platform way. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 10 November 2017 at 19:50, Paul Moore wrote:

On 10 November 2017 at 08:01, Nick Coghlan wrote:

...

That tooling is venv:

* it ensures you have "pip" on your PATH * it ensures you have "python" on your PATH * it ensures that you have the required permissions to install new packages * it ensures that any commands you install from PyPI will be also on your PATH

When we choose not to use venv, then it becomes necessary to ensure each of those things individually for each potential system starting state

Currently, the reality is that people use virtualenv, not venv. All higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7 support). Enhancing the capabilities of venv is fine, but promoting venv over virtualenv involves technical challenges across the whole toolset, not just documentation/education.

We already assume there will be a step in understanding from "working with the latest Python 3.x locally" to "dealing with multiple Python versions". Switching from venv to virtualenv just becomes part of that process (and will often be hidden behind a higher level tool like pipenv, pew, or vex anyway). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Fri, Nov 10, 2017 at 09:50:22AM +0000, Paul Moore wrote:

The biggest reason we don't add Python to PATH, as I understand it, is because we need to consider the implications of people having multiple versions of Python installed.

Why not fix that the same way as on Unix -- by having versioned executables: python27.exe, python35.exe? Then python.exe in PATH will be from the most recent installed Python.

Paul

Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

On Windows, which is the only platform I can reasonably comment on, the killer issue is that the installer doesn't make the commands "python" and "pip" available by default. Just checking my PC, both go and rust (which I installed *ages* ago) do appear to, so they "just work". Java sort-of also works like that (the runtime is on PATH, but the compilers need to be added manually).

The biggest reason we don't add Python to PATH, as I understand it, is because we need to consider the implications of people having multiple versions of Python installed. As far as I know, no other language allows that. But equally, most beginners wouldn't actually *have* multiple versions installed. So maybe we should optimise for that case (only one version of Python installed). That would mean:

That seems reasonable. Especially since somebody with several versions install is more likely to know how to deal with system path issues than a total beginner installing python for the first time.

1. Go back to adding Python to PATH. Because our installers don't say "do you want to uninstall the old version", we should probably do a check for a "python" command on PATH in the installer, and if there is one, warn the user "You already have Python installed - if you are upgrading you should manually uninstall the old version first, otherwise your old installation will remain the default". We could get more complex at this point, but that depends on what capabilities we can include in the installer - I don't know how powerful the toolset is.

You don't even have to do that. We can detect it and prompt : "which python version do you want to be available by default ?", and then list command to run the alternative versions.

2. Explicitly document that multiple Python interpreters on one machine is considered "advanced", and users with that sort of setup should be prepared to manage PATH themselves. I'd put that as something like "It is possible to install multiple versions of Python at once, but if you do that, you should understand the implications - the average user should not need to do this"

We still have to deal with the fact that basically every Unix environment is "advanced" in the above sense (the python2/python3 split). I don't have a solution for that (other than "upgrade to Windows" ;-)).

Provide the "py" command on linux and mac. And make it the default recommended way in the documentation.

...

It doesn't completely solve the problem (as getting into and out of the environment is still platform specific), but it does mean that the ubiquitous online instructions to run "pip install package-name" and "python -m command" will actually work once people are inside their working environment.

That tooling is venv:

* it ensures you have "pip" on your PATH * it ensures you have "python" on your PATH * it ensures that you have the required permissions to install new packages * it ensures that any commands you install from PyPI will be also on your PATH

When we choose not to use venv, then it becomes necessary to ensure each of those things individually for each potential system starting state

Currently, the reality is that people use virtualenv, not venv. All higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7 support). Enhancing the capabilities of venv is fine, but promoting venv over virtualenv involves technical challenges across the whole toolset, not just documentation/education.

But agreed, once we get people into a virtual environment (of any form) the portability issues become significantly reduced. The main outstanding issue is managing multiple environments, which could be handled by having a special "default" environment that is the only one we'd expect beginners to use/need.

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian. And then virtualenv have their own issues: - it's a contextual mode that you need to activate AND be aware of at all time - you need to config tooling to use it (IDE, builders, etc) - the location of virtualenv is very important - you should not commit it to git - you can't relocate it easily - install jupyter / mymy / flake8 outside a virtualenv and the command will still seems to work inside a virtualenv it's not installed in. With terrible consequences. Virtualenvs are a hard tool to use for beginners. A lot of people on this list have forgotten their early years it seems.

Also on Windows, the per-user bin directory isn't added to PATH even if you add the system Python to PATH in the installer.

...

That said, I think there is one improvement we could feasibly make, which would be to introduce the notion of a "default user environment" into `venv`, such that there was a single "python -m venv shell" command that:

* created a default user environment if it didn't already exist * launched a subshell with that environment already activated

This wouldn't be a full environment manager like vex or pew - it would just be a way to bootstrap a single usable package management environment in a cross-platform way.

"How do I run venv?" is no easier to answer in a cross-platform way than "how do I run pip?" You still need to be able to run python/python3/py. About the only improvement is that there's no legacy of documentation and advice on the web saying "run venv" like there is for "run pip"...

Paul

Exactly.

On 12 November 2017 at 22:20, Paul Moore wrote:

On 12 November 2017 at 06:19, Michel Desmoulin wrote:

...

...

We still have to deal with the fact that basically every Unix environment is "advanced" in the above sense (the python2/python3 split). I don't have a solution for that (other than "upgrade to Windows" ;-)).

Provide the "py" command on linux and mac. And make it the default recommended way in the documentation.

+1 from me.

The current Python launcher is Window specific, and doesn't work on POSIX systems. So the first barrier here is "Write a py command that presents the same CLI on POSIX that the Python launcher does on Windows". The second problem is that even though we can add a "py" command to the python.org Mac OS X installers and have that take effect for new python.org downloads, there are a *lot* of people on Mac OS X and Linux that don't actually obtain their pre-built Python from python.org. While some of those would probably adapt and ship a new command fairly quickly (the cross-platform commercial redistributors, rolling release and 6-month cadence Linux distros, community redistributors like pythonz, pyenv, and homebrew), other important platforms wouldn't (most notably, LTS Linux distros). The fact it wouldn't actually solve the bootstrapping problem due to the update policies of various popular platforms then greatly reduces the incentive to work on the first part of the problem (i.e. actually creating a POSIX-compatible version of that CLI).

...

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

Yup. And RHEL/CentOS don't provide Python 3.x by default at all - you need to grab it via other means.

...

Virtualenvs are a hard tool to use for beginners.

Agreed. Genuine beginners just install Python, then use it. If they install extra packages, they want them to be available to all of their scripts.

Also agreed, but without them (or a functional equivalent, like `conda env` or `pyenv`), it's incredibly hard to get an arbitrary user on an arbitrary system to the point where all the following assumptions are satisfied: * the user is working at a command prompt (whether inside their IDE, or in an actual system shell) * the command `python` will run the desired version of Python * the commands `pip install` and `python -m pip install` will be essentially equivalent (aside from the differences when upgrading pip itself on Windows) and install packages into the desired version of Python * any executables installed that way will be executable from that command prompt * any Python import packages installed that way will be importable from a Python shell started in that environment (List of assumptions taken from https://github.com/pypa/python-packaging-user-guide/issues/396) Trying to drop any of those assumptions is problematic: * non-command-shell learning environments vary even more than command shells do * the `python` command has a much long history than any of the alternatives (including both `py` and `python3`), and third party guides reflect that * the `pip install` option really is nicer looking than `python -m pip install`, and it only has actual problems in the presence of multiple Python versions and when upgrading pip itself on Windows (plus: lots of third party guides recommend it, as do pypi.org project pages) * if `pip install devtool` doesn't let you run `command-from-devtool` in your command shell, that's thoroughly unhelpful * if `pip install dist-package` doesn't let you run `import library_from_dist_package` in your Python code, that's even less helpful It *would* be much nicer if a single user install of the Windows python.org installer met these criteria by default, though - while the `py` launcher is a good way of handling multiple versions (and nicely fills the role of handling file associations for Python file extensions), it isn't a particularly nice alternative to a properly configured user PATH on a machine with only one Python installation.

...

A lot of people on this list have forgotten their early years it seems.

Maybe. But the default beginner approach *does* have its problems, and guiding beginners to a better approach is a good idea.

We haven't forgotten our early years - we've just spent years (both individually and collectively) working on the problem of helping people get started with software development, and thus have a very good idea as to what *doesn't* work, as well as what *does* work. The latter list is currently fairly short: * having a friend or colleague walk them through it * "Install Parties", like those Django Girls runs (i.e. running a pre-tutorial event, specifically focused on getting a working environment set up) * highly prescriptive learning environments, whether online ones (like Grok Learning, trinket.io, PythonAnywhere, etc), or locally installed ones (like PyCharm Educational Edition, the Anaconda distribution, etc) Without the kinds of constraints suggested in the last option, there are too many potential starting points, and it isn't even possible to ask potential learners to self-assess what their starting point actually is, since it's a tech support problem where the first task is "assess the current state of the user's system" (hence the first two options). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 2017-11-12 05:18, Nick Coghlan wrote:

* the `pip install` option really is nicer looking than `python -m pip install`, and it only has actual problems in the presence of multiple Python versions and when upgrading pip itself on Windows (plus: lots of third party guides recommend it, as do pypi.org project pages)

Is there any *advantage* to using `pip install` instead of `python -m install`? If not, could we at least change everything under Python/pip control (e.g., pip documentation) to never recommend `pip` and always recommend `python -m pip` instead, and encourage all third-party documentation to always use `python -m pip` and never use `pip`? Obviously this isn't a full solution, but in the end there's no way we change external third-party documentation, which will always eventually become outdated. Absent that, it seems worthwhile to regularize existing official documentation. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Nick Coghlan Sent: Sunday, November 12, 2017 6:19 PM To: Brendan Barnwell Cc: python-ideas@python.org Subject: Re: [Python-ideas] Looking for input to help with the pip situation

On 13 November 2017 at 02:59, Brendan Barnwell wrote:

...

On 2017-11-12 05:18, Nick Coghlan wrote:

...

* the `pip install` option really is nicer looking than `python -m pip install`, and it only has actual problems in the presence of multiple Python versions and when upgrading pip itself on Windows (plus: lots of third party guides recommend it, as do pypi.org project pages)

Is there any *advantage* to using `pip install` instead of `python -m install`? If not, could we at least change everything under Python/pip control (e.g., pip documentation) to never recommend `pip` and always recommend `python -m pip` instead, and encourage all third-party documentation to always use `python -m pip` and never use `pip`?

We've already changed most of them (pypi.org itself is the main one that we haven't changed it yet).

However, there are still per-project READMEs out there that suggest "easy_install project" and direct invocation of "python setup.py install", so it really isn't appealing to layer yet another mandatory change in the recommended spelling of the installation command and create yet another point of confusion - it will be much nicer overall if we can retroactively make the existing "pip install" instructions correct for most users, and leave "python -m pip install" to the "Multiple versions of Python" and "Self-upgrading pip on Windows".

Cheers, Nick.

P.S. As a user, it's also genuinely irritating to have to always type the "python -m " prefix when inside an active virtual environment, as in that case, there isn't any ambiguity about which environment pip should be manipulating or which version of Python it should be using.

You kinda should be typing `env/bin/python -m ...` (or env\Scripts\python.exe -m ...) but saying that proves your previous point.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

This has gotten to be a big thread, and it's a pretty intractable problem, but I think there are a few fairly small things that could be done to at least make it a bit easier: 1) Add python2.exe and python3.exe files to the Windows installers -- am I insane or did Windows used to have that? I really think it did -- maybe got removed when py.exe was added. 1a) alternatively, we could add a "py" executable to the standard linux builds, so there would be THAT one way to do it. But I think that's a "BAD IDEA" -- the whole "py" thing is not widely know or used, it's not going to show up in package install instructions for a LONG time, (actualy we could do both anyway) Then "python2 -m pip install" would work everywhere (only with new installations, but at least with newbies, that's a bit more likely ...) 2) Make adding to the PATH in Windows the default. I think there are really three user groups: - newbies starting from scratch -- they want it on the PATH - newbies with whatever left over cruft from previous installations on their systems -- they want it at the FRONT of their PATH. They SHOULD uninstall all the cruft, but if they don't this will still work with as few surprises a possible. - not-newbies with a previous version of python they need to continue using. They can uncheck the box, or use py.exe 3) Make --user be be automatic for pip install. Not actually the default, but pip could do a user install if you don't have the permissions for a non-user install. This means folks might accidentally install in user mode because they forgot to type sudo -- but that would be a mostly-sysadmin/sophisticated user problem. And maybe have an environment variable of configuration key for "prefer admin install". If tha was set, pip would only install in user mode if specifically asked to. I'm can't imagine a case where a user would have admin permissions, but want a user install (except people following bad practices!) Except for the pip change, these would be easy to implement and backward compatible. So why not? *NOTE:* even if nothing changes with any of this we need to get py.exe better documented and advertised -- it doesn't show up in: https://docs.python.org/3/faq/windows.html#id2 for instance. In fact, I knew about py.exe (from this discussion), and was writing up notes about how to run a Python file (without access to a Windows box) , and it took a LONG time to find ANY documentation of it (adding "py" to a google search about something python does not get far...). We can do that better, but frankly this may be a lesson on why we can't rely on anything "new" to help solve this problem, when maybe we could make the "old way" work better and more cross platform. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 13 November 2017 at 18:57, Chris Barker wrote:

This has gotten to be a big thread, and it's a pretty intractable problem, but I think there are a few fairly small things that could be done to at least make it a bit easier:

In principle, I agree with the ideas here, but there are some practical issues that make them somewhat less straightforward than we might like.

1) Add python2.exe and python3.exe files to the Windows installers -- am I insane or did Windows used to have that? I really think it did -- maybe got removed when py.exe was added.

I don't think Windows ever had python2.exe/python3.exe, but I could be wrong. The only possible way we'd get a python2.exe is by adding it to future releases of Python 2.7, and we shouldn't be recommending Python 2 for new users anyway. And I'm strongly -1 on promoting "python3.exe" as the canonical way of getting Python 3. The python3 command is a result of the Unix issues around switching to Python 3 as the default, and we shouldn't perpetuate that approach on Windows, where it's unneeded. Having said that, I don't object to including versioned executables as a convenience (if it is one). I just dislike promoting the use of them via standard documentation.

1a) alternatively, we could add a "py" executable to the standard linux builds, so there would be THAT one way to do it. But I think that's a "BAD IDEA" -- the whole "py" thing is not widely know or used, it's not going to show up in package install instructions for a LONG time, (actualy we could do both anyway)

I think that getting a "py" launcher on Unix is a lost cause at this stage. It would be nice, and maybe if the original PEP had proposed a cross-platform py command, it might have worked, but that's history now, and I think the py launcher will probably always be a Windows-only thing.

Then "python2 -m pip install" would work everywhere (only with new installations, but at least with newbies, that's a bit more likely ...)

No newcomer should *ever* be getting told to use "python2 -m pip install". Maybe "python3 -m pip install", but never Python 2 (outside of specialised environments where training people in an out of date version is required). And of course not all Unix distributions come with Python 3 installed (Mac OS being an obvious example, and I think Nick mentioned CentOS and RHEL) so python3 won't work everywhere either...

2) Make adding to the PATH in Windows the default. I think there are really three user groups:

- newbies starting from scratch -- they want it on the PATH

- newbies with whatever left over cruft from previous installations on their systems -- they want it at the FRONT of their PATH. They SHOULD uninstall all the cruft, but if they don't this will still work with as few surprises a possible.

- not-newbies with a previous version of python they need to continue using. They can uncheck the box, or use py.exe

Unfortunately, adding Python to the *front* of PATH is not possible. It might be the best option, but it simply cannot be done. A per-user install of Python (the default in Python 3.5+) does not have the privileges to add items to the front of the system PATH, so the best we can do is add it to the front of the user PATH. But the system PATH always goes ahead of the user PATH, and system installs (for example, default installs of Python 2.x or 3.4 or earlier) will come before that. (Making a system install of Python be the default won't work either, as then pip won't work from a normal prompt and we have the same issues on Windows that Unix users have that results in people doing "sudo pip install" with all the issues that brings). So for (1) I agree. For (2) it simply isn't possible. For (3) you're getting beyond newcomer so sure, "manually handle it" is a reasonable option. There's also the problem that file associations (i.e., what does double clicking on a .py/.pyw file do) don't follow the same rules, as they go through the launcher. The only really sensible "do what I expect" situation is a single Python 3 installation on the user's machine. For that case, adding Python to PATH (it doesn't matter where as there's only one) is a sensible option. The downside is that optimising for that case makes the behaviour for other (more experienced) users worse. But it's not unreasonable to make that trade-off.

3) Make --user be be automatic for pip install. Not actually the default, but pip could do a user install if you don't have the permissions for a non-user install.

The problem here is that the user scripts directory isn't on PATH. We could put it on, but again we hit the "goes after the system PATH" problem (on Windows).

This means folks might accidentally install in user mode because they forgot to type sudo -- but that would be a mostly-sysadmin/sophisticated user problem. And maybe have an environment variable of configuration key for "prefer admin install". If tha was set, pip would only install in user mode if specifically asked to. I'm can't imagine a case where a user would have admin permissions, but want a user install (except people following bad practices!)

The various options here are under discussion on the pip tracker. Suffice it to say, it's not a clear-cut change, but it would ideally be something we do. Nobody really objects to it, but there are significant backward compatibility and behaviour change issues to address first.

Except for the pip change, these would be easy to implement and backward compatible. So why not?

See above. But in principle, if the issues can be solved in a way we're all happy with, I agree with the principle that "download and install Python" should result in an environment where python myscript.py pip install something_or_other "just work" if the user previously had no Python on their PC. Anything less results in a worse beginner experience for Python than for other languages (which usually don't support having multiple versions installed at once, and often require admin installs, so bypass most of the problems Python has to deal with).

NOTE: even if nothing changes with any of this we need to get py.exe better documented and advertised -- it doesn't show up in:

https://docs.python.org/3/faq/windows.html#id2

for instance.

It has a whole section at https://docs.python.org/3/using/windows.html#python-launcher-for-windows. And https://docs.python.org/3/using/windows.html discusses how to handle PATH, so that's clearly documented too, even if (a) no-one reads the docs, and (b) the defaults may not be the best choices. Agreed that https://docs.python.org/3/faq/windows.html seems pretty out of date, and could do with an update. I'm sure patches would be gratefully accepted, if only to replace the out of date information with pointers to the correct places in the main documentation. Ultimately, though, most newcomers don't read the docs. They muddle through, ask colleagues who may or may not know any better than they do, read out of date articles on Stack Overflow and watch videos on Youtube. The *only* way of stopping them doing that is to make everything "just work" for them. And when one person can tell them to download Python from python.org, another says "use Anaconda" and a third suggests ActiveState, and they try all three while trying to work out what they need, sometimes we just have to do the best we can :-) Paul

On Tue, 14 Nov 2017 12:14:50 +1000 Nick Coghlan wrote:

...

I don't think Windows ever had python2.exe/python3.exe, but I could be wrong.

Not that I'm aware of in the python.org installers, and I don't think ActivePython does either. I'm less sure about Enthought or Anaconda (since I've never used either of them on Windows).

Looking at my Windows VM... * from a Miniconda install (equivalent, I think, to a bare-bones Anaconda with the minimal package set required for a function "python" and "conda"): c:\>where python C:\Miniconda3\python.exe c:\>where python3 INFO: Could not find files for the given pattern(s). * from a user-created conda environment: c:\>activate da36 (da36) c:\>where python C:\Miniconda3\envs\da36\python.exe C:\Miniconda3\python.exe (da36) c:\>where python3 INFO: Could not find files for the given pattern(s). (no clue about Enthought, sorry)

Yeah, myself, Barry Warsaw, Matthias Klose, and a number of other folks on linux-sig have poked at this idea multiple times since Geoff Thomas first posted about https://github.com/geofft/pythonmux, but our conclusion each time has been that it wouldn't help enough to justify the effort involved in implementing and promoting it. After 20+ years of usage in the Linux ecosystem, `/usr/bin/python` and `/usr/bin/env python` are simply too entrenched in both people's habits and existing code.

+1. As a matter of fact, Anaconda also doesn't seem to expose "py" on Windows: (da36) c:\>py 'py' is not recognized as an internal or external command, operable program or batch file.

...

...

3) Make --user be be automatic for pip install. Not actually the default, but pip could do a user install if you don't have the permissions for a non-user install.

The problem here is that the user scripts directory isn't on PATH. We could put it on, but again we hit the "goes after the system PATH" problem (on Windows).

We should still optimise the defaults for the desired system configuration (i.e. user-mode installs as the cross-platform default outside a venv), and then work on platform-specific troubleshooting guides for the common ways that things can go wrong.

Hmm... I liked --user at some point, but if you start using it a lot it becomes problematic, for example if you want to install two different scripts/applications with incompatible dependencies. That said, it *is* much better than system-wide installs of user packages :-) Regards Antoine.

On Nov 13, 2017, at 1:53 PM, Paul Moore wrote: In principle, I agree with the ideas here, but there are some practical issues that make them somewhat less straightforward than we might like. And practically beats principally ;-) But yeah. 1) Add python2.exe and python3.exe files to the Windows installers The only possible way we'd get a python2.exe is by adding it to future releases of Python 2.7, and we shouldn't be recommending Python 2 for new users anyway. No, but it’s going to be in use for a while yet (2020, yes?). And no, new users should not install py3, but new users may work in an org that is using py2. And I'm strongly -1 on promoting "python3.exe" as the canonical way of getting Python 3. So what is canonical? Not the py executable, for sure. *maybe* we are at a point where we expect py3 to be the default on most systems, but we DO need a way to tell people how to get py3 if: Typing python gets them py2. And it’s not an option to remove py2 or re-configure their systems to change the default ( see your point below about Windows PATH priorities) The python3 command is a result of the Unix issues around switching to Python 3 as the default, and we shouldn't perpetuate that approach on Windows, where it's unneeded. But the exact same issues are there in windows (except the shebang line) bit it’s pretty weird that py.exe respects “python3”, when there is no python3.exe on the system... And having multiple systems work the same way IS a laudable goal. Having said that, I don't object to including versioned executables as a convenience (if it is one). I just dislike promoting the use of them via standard documentation. Totally agree. 1a) alternatively, we could add a "py" executable to the standard linux builds, so there would be THAT one way to do it. But I think that's a "BAD IDEA" -- I think that getting a "py" launcher on Unix is a lost cause at this stage. Fine — I never liked the idea :-) Then "python2 -m pip install" would work everywhere (only with new installations, but at least with newbies, that's a bit more likely ...) No newcomer should *ever* be getting told to use "python2 -m pip install". Maybe "python3 -m pip install", Exactly — that was supposed to be an example. Both should work, but probably: Python -m pip install would be recommended, with “you can add 2 or 3 if it doesn’t do the right thing. And of course not all Unix distributions come with Python 3 installed Exactly — which is why even newbies that install just python 3 end up with both on their system. (Mac OS being an obvious example, and I think Nick mentioned CentOS and RHEL) so python3 won't work everywhere either... But it will give you an error, rather that python 2. That’s exactly what we want! 2) Make adding to the PATH in Windows the default. ... Unfortunately, adding Python to the *front* of PATH is not possible. Well, darn. But we can still make it the default and it will sometimes work. And all the more reason we need a “python3” executable. There's also the problem that file associations (i.e., what does double clicking on a .py/.pyw file do) don't follow the same rules, as they go through the launcher. I don’t see anything we can do with that. But I don’t recommend users use that anyway. The only really sensible "do what I expect" situation is a single Python 3 installation on the user's machine. Sure — but we can’t control that. All we can do is mitigate the damage. For that case, adding Python to PATH (it doesn't matter where as there's only one) is a sensible option. The downside is that optimising for that case makes the behaviour for other (more experienced) users worse. But it's not unreasonable to make that trade-off. Agreed. And not much worse. 3) Make --user be be automatic for pip install. Not actually the default, but pip could do a user install if you don't have the permissions for a non-user install. The problem here is that the user scripts directory isn't on PATH. How does that have anything to do with it? Either they can launch pip or they can’t (the above are about that) this is about what should happen when they do launch pip. The various options here are under discussion on the pip tracker. I’ll try to go there to comment. but there are significant backward compatibility and behaviour change issues to address first. Always the challenge! python myscript.py pip install something_or_other "just work" if the user previously had no Python on their PC. And as many other configurations as possible :-) I think the changes we should make should emphasize two things that perhaps haven’t been given enough priority in the past: 1) things should work as much the same as possible across platforms. 2) making it easier for newbies is more important that for more experienced folks. It has a whole section at https://docs.python.org/3/using/windows.html#python-launcher-for-windows. What I can tell you is that I knew what I was looking for, and didn’t find in more than a few google searches :-( Ultimately, though, most newcomers don't read the docs. They muddle through, ask colleagues who may or may not know any better than they do, read out of date articles on Stack Overflow ... Given this, in fact, any changes to documentation and recommended best practices won’t filter through ‘till after py2’s end of life ... :-( So really, if we can change behavior to better match what is already documented — we should. Too late to change best practices. -CHB

On 15 November 2017 at 22:46, Michel Desmoulin wrote:

Le 13/11/2017 à 19:57, Chris Barker a écrit :

...

3) Make --user be be automatic for pip install. Not actually the default, but pip could do a user install if you don't have the permissions for a non-user install.

Breaking compat ? Not sure people will accept.

This isn't actually a Python level change - it's a pip UX level one, and already on their todo list: https://github.com/pypa/pip/issues/1668 I believe the next concrete step that can be taken there is to actually add an explicit `--global` flag, so I've belatedly broken that out as its own issue: https://github.com/pypa/pip/issues/4865 =====

Should I do a PEP with a summary of all the stuff we discussed ?

I think a Windows-specific PEP covering adding PATH updates back to the default installer behaviour, and adding pythonX and pythonX.Y commands would be useful (and Guido would presumably delegate resolving that to Steve Dower as the Windows installer maintainer). The one thing I'd ask is that any such PEP *not* advocate for promoting ther variants as the preferred way of invoking Python on Windows - rather, they should be positioned as a way of making online instructions written for Linux more likely to "just work" for folks on Windows (similar to the utf-8 encoding changes in https://www.python.org/dev/peps/pep-0529/) Instead, the focus should be on ensuring the "python -m pip install" and "pip install" both work after clicking through the installer without changing any settings, and devising a troubleshooting guide to help folks that are familiar with computers and Python, but perhaps not with Windows, guide folks to a properly working environment. The update to the Windows installer would then start offering benefits as soon as Python 3.7 becomes the default download, while the troubleshooting guide would be beneficial as soon as folks start learning about it's existence. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Wed, Nov 15, 2017 at 1:07 PM, Steve Dower wrote:

My preferred solution for this is to rename "py.exe" to "python.exe" (or rather, make a copy of it with the new name), and extend (or more likely, rewrite) the launcher such that:

* if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and launch Python based on first command line argument * if argv[0] == "python<numeric tag>.exe", find the matching PythonCore/<tag> install (where tag may be a partial match - e.g. "python3.exe" finds the latest PythonCore/3.x) * else, if argv[0] == "<module><numeric tag>.exe, find the matching PythonCore/<tag> install and launch "-m <module>"

With the launcher behaving like this, we can make as many hard links as we want in its install directory (it only gets installed once, so only needs one PATH entry, and this is C:\Windows for admin installs): * python.exe * python2.exe * python3.exe * python3.6.exe * pip.exe * pip2.exe * pip3.exe

I haven't been following this thread closely, but this sounds lovely. I'm not terribly keen on cluttering up C:\Windows with this, but that's a minor issue. -- Zach

On 16 November 2017 at 05:29, Zachary Ware wrote:

On Wed, Nov 15, 2017 at 1:07 PM, Steve Dower wrote:

...

My preferred solution for this is to rename "py.exe" to "python.exe" (or rather, make a copy of it with the new name), and extend (or more likely, rewrite) the launcher such that:

* if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and launch Python based on first command line argument * if argv[0] == "python<numeric tag>.exe", find the matching PythonCore/<tag> install (where tag may be a partial match - e.g. "python3.exe" finds the latest PythonCore/3.x) * else, if argv[0] == "<module><numeric tag>.exe, find the matching PythonCore/<tag> install and launch "-m <module>"

With the launcher behaving like this, we can make as many hard links as we want in its install directory (it only gets installed once, so only needs one PATH entry, and this is C:\Windows for admin installs): * python.exe * python2.exe * python3.exe * python3.6.exe * pip.exe * pip2.exe * pip3.exe

I haven't been following this thread closely, but this sounds lovely. I'm not terribly keen on cluttering up C:\Windows with this, but that's a minor issue.

I'd missed Steve's post before writing my last one. This sounds like a really nice technical solution to me, too, especially as it will handle Python 2 as well (even for Python 2 only systems, the launcher is available as an independently installable executable). Regardless of the underlying implementation details though, a PEP would be a helpful way of writing it up so we can make sure packaging.python.org and other resources properly account for it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Would something like this be helpful? (`python -m pip` seems to work fine after `conda install pip` after a miniconda install) https://gist.github.com/westurner/80e81cd4bf9b79acb5989622f2621655 ```python #!/usr/bin/env python """ Something like this as e.g site.discover (for use as ``python -m site.discover``) could be helpful for explaining and diagnosing multiple pythons and pips """ import os import re from distutils.spawn import find_executable from subprocess import check_output, STDOUT import collections def discover(path=None, printsite=False, uniques=True): """ Find all ``python*`` and ``pip*`` executables and their version strings within ``os.environ['PATH']`` Kwargs: path (str): os.pathsep-delimited path str (defaults to ``os.environ['PATH']``) printsite (bool): call ``<python> -m site`` with each found python binary uniques (bool): print uniques according to ``os.realpath`` at the end Returns: None """ if path is None: path = os.environ['PATH'] rgx = re.compile(r'(python|pip)\-?(\d?\.?\d?)$') uniques = collections.OrderedDict() for directory in path.split(os.pathsep): paths = sorted( ((match.group(0), match.group(2)) for match in (rgx.match(name) for name in os.listdir(directory)) if match), key=lambda x: x[::-1], reverse=True) print(u'# %s' % directory) for (name, ver) in paths: pathjoined = os.path.join(directory, name) binpath = find_executable(pathjoined, directory) realpath = os.path.realpath(binpath) if binpath is None: continue if os.path.exists(pathjoined): if os.path.islink(pathjoined): print(( u"%r is a symlink to %r, which doesn't exist" % (pathjoined, binpath))) # TODO: %r exists but isnt executable verstring = check_output((binpath, '-V'), stderr=STDOUT).rstrip() uniques.setdefault(realpath, collections.OrderedDict()) \ .setdefault(binpath, verstring) print(u'%-11r %-5r %r' % (name, ver, verstring)) if printsite and name.startswith('python'): sitestring = check_output((binpath, '-m', 'site')) lines = (u'> %s' % l for l in sitestring.splitlines()) for line in lines: print(line) # TODO: check_output((binpath, '-m', 'pip')) # TODO: AND/OR (?) only print uniques at the end if uniques: print('### UNIQUES') for path in uniques: print('## %s' % path) paths = uniques.get(path) for otherpath in sorted(paths): print(otherpath) print('') if __name__ == '__main__': discover() ``` On Wednesday, November 15, 2017, Steve Dower wrote:

On 15Nov2017 0617, Nick Coghlan wrote:

...

On 15 November 2017 at 22:46, Michel Desmoulin mailto:desmoulinmichel@gmail.com> wrote: Should I do a PEP with a summary of all the stuff we discussed ? I think a Windows-specific PEP covering adding PATH updates back to the default installer behaviour, and adding pythonX and pythonX.Y commands would be useful (and Guido would presumably delegate resolving that to Steve Dower as the Windows installer maintainer).

If you write such a PEP, please also research and write up the issues with modifying PATH on Windows (they're largely scattered throughout bugs.p.o and earlier discussions on python-dev).

Once you realise the tradeoff involved in modifying these global settings, you'll either come around to my point of view or be volunteering to take *all* the support questions when they come in :)

The one thing I'd ask is that any such PEP *not* advocate for promoting

...

ther variants as the preferred way of invoking Python on Windows - rather, they should be positioned as a way of making online instructions written for Linux more likely to "just work" for folks on Windows (similar to the utf-8 encoding changes in https://www.python.org/dev/peps/pep-0529/)

Instead, the focus should be on ensuring the "python -m pip install" and "pip install" both work after clicking through the installer without changing any settings, and devising a troubleshooting guide to help folks that are familiar with computers and Python, but perhaps not with Windows, guide folks to a properly working environment.

My preferred solution for this is to rename "py.exe" to "python.exe" (or rather, make a copy of it with the new name), and extend (or more likely, rewrite) the launcher such that:

* if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and launch Python based on first command line argument * if argv[0] == "python<numeric tag>.exe", find the matching PythonCore/<tag> install (where tag may be a partial match - e.g. "python3.exe" finds the latest PythonCore/3.x) * else, if argv[0] == "<module><numeric tag>.exe, find the matching PythonCore/<tag> install and launch "-m <module>"

With the launcher behaving like this, we can make as many hard links as we want in its install directory (it only gets installed once, so only needs one PATH entry, and this is C:\Windows for admin installs): * python.exe * python2.exe * python3.exe * python3.6.exe * pip.exe * pip2.exe * pip3.exe

As well as allowing e.g. "py.exe -anaconda36-64 ..." to reliably locate and run non-Python.org installs.

It needs to be fully specced out, obviously, and we may want to move the all-users install to its own directory to reduce clutter, but part of the reason behind PEP 514 was to enable this sort of launcher. It could even extend to "you don't have this version right now, want to download and install it?"

And finally it should be fairly obvious that this doesn't have to be a core Python tool. It has no reliance on anything in core (that isn't already specified in a PEP) and could be written totally independently. I've tried (weakly) to get work time allocated to this in the past, and if it's genuinely not going to get done unless I do it then I'll try again.

Cheers, Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 17 November 2017 at 05:15, Chris Barker wrote:

On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower wrote:

...

If you write such a PEP, please also research and write up the issues with modifying PATH on Windows (they're largely scattered throughout bugs.p.o and earlier discussions on python-dev).

Is anyone proposing doing anything new with that? (other than changing the default)

My preferred solution for this is to rename "py.exe" to "python.exe"

I was going to propose that in this thread, but then thought: "there has GOT to be a reason why that reall obvious solution wan't done in the first place", and didn't have time to go back and research it.

As far as I recall, the arguments against it are: - wanting the regular executable and the launcher to be able to coexist in the same build target directory - not wanting the regular python executable to prevent access to the launcher at a shell prompt - not wanting the launcher at a shell prompt to prevent access to the regular executable at a shell prompt However, https://www.python.org/dev/peps/pep-0397/ doesn't spell those out, it just states what the launcher's name will be, and emphasises that the main purpose is to provide a sensible target for file associations after the "always use the most recently installed version" assumption broke down: https://www.python.org/dev/peps/pep-0397/#rationale Addressing them now: * as long as the extra hard links are only generated at install time, there also won't be any problems with build directory name conflicts. * the launcher will always be available a `py`, regardless of the current PATH * PATH in a venv will still put the regular python executable ahead of the launcher Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 17/11/2017 09:13, Paul Moore wrote:

On 17 November 2017 at 01:57, Nick Coghlan wrote:

...

On 17 November 2017 at 05:15, Chris Barker wrote:

...

On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower wrote:

...

If you write such a PEP, please also research and write up the issues with modifying PATH on Windows (they're largely scattered throughout bugs.p.o and earlier discussions on python-dev).

Is anyone proposing doing anything new with that? (other than changing the default)

...

My preferred solution for this is to rename "py.exe" to "python.exe"

I was going to propose that in this thread, but then thought: "there has GOT to be a reason why that reall obvious solution wan't done in the first place", and didn't have time to go back and research it.

As far as I recall, the arguments against it are:

- wanting the regular executable and the launcher to be able to coexist in the same build target directory - not wanting the regular python executable to prevent access to the launcher at a shell prompt - not wanting the launcher at a shell prompt to prevent access to the regular executable at a shell prompt

However, https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.python.org%2Fdev%2Fpeps%2Fpep-0397%2F&data=02%7C01%7Cgadgetsteve%40live.co.uk%7C18fce28e3baa4e4e70c708d52d9b80c5%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636465068254194278&sdata=4%2BAusacb7TwM5NY9xEUXx%2B%2Ff7s%2BTvV05xIhUWtn9xjg%3D&reserved=0 doesn't spell those out, it just states what the launcher's name will be, and emphasises that the main purpose is to provide a sensible target for file associations after the "always use the most recently installed version" assumption broke down: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.python.org%2Fdev%2Fpeps%2Fpep-0397%2F%23rationale&data=02%7C01%7Cgadgetsteve%40live.co.uk%7C18fce28e3baa4e4e70c708d52d9b80c5%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636465068254194278&sdata=OVs0HEVnlob8Rj5LQewOE0YoyIvIA9ydar7i0b1%2BZfY%3D&reserved=0

Addressing them now:

* as long as the extra hard links are only generated at install time, there also won't be any problems with build directory name conflicts. * the launcher will always be available a `py`, regardless of the current PATH * PATH in a venv will still put the regular python executable ahead of the launcher

Note that if I *do* get round to working on this, my primary intention will be to propose altering the launcher to allow it to be used under different names, as described by Steve. However, I won't initially be proposing that we add any additional links to the launcher by default, leaving that for users to do manually, and/or for later revisions of the PEP (or further PEPs) to propose this.

Reasons:

1. The launcher is typically in C:\Windows, which has a very high priority on PATH, so getting the environment right will be tricky. 2. I don't have any real experience with the installer. 3. Backporting this change, or dealing with older versions of Python that don't include the new launcher, is a complicated question.

I'd rather keep the initial PEP restricted to the simple behaviour change. It's possible that the behaviour change may not even need a PEP, but I think it'd be better to have one. The "how do we use the new behaviour in the installers" question is likely to be much more controversial...

Paul

PS This (particularly the replacement of python.exe) is almost certainly too late for 3.7, so it's not as if there's a rush to answer all the questions at once :-) _______________________________________________

Note that renaming py.exe to python.exe would have several problems: 1. It would only address things on Windows or if py were created for other platforms as python there would be issues with trying to ensure that it was the one that was called first in every case. 1. We would have to also rename python.exe to avoid py.exe executing itself. 1. The current behaviour of py.exe doesn't quite match what I was suggesting in that it always executes a python, (the default or specified), or in 3.7 with the -0 lists the ones that it can find, so would not really address the users installing to the wrong context. I was suggesting that pip, and possibly other tools, would simply execute the command if there was only one python in the current context or prompt the user if there were more than one. (It occurs to me that it might also be valuable to have an all option, e.g. for admins installing security updates). I do like the idea to making this sort of behaviour available, possibly as soft-links or alternate entry points, to tools other than pip as that would be very handy in some cases, so having this behaviour in a python library that pip and other tools could invoke would be ideal. (Having it in an external library also takes care of the old installations issue as with back ports & requires.txt it should have a rapid uptake). A soft-link, entry point or batch file option might run something like, (using pydisamb as a possible name), pip.exe = pydisamb -m pip or possibly examining sys.argv[0] and invoking the found or selected python with -m the same name as the invocation if we can find a way to access that, (it will have to give the invocation name for exe, bat, ps1, sh, lnk, etc. to work). -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com

​ On Mon, Nov 20, 2017 at 1:59 PM, Chris Barker wrote:

BTW -- does pip install a "pip3" on Windows?

​ In 3.6 it does:

find t:/Python36/ -iname 'pip*exe' t:/Python36/Scripts/pip.exe t:/Python36/Scripts/pip3.6.exe t:/Python36/Scripts/pip3.exe

​

On 20 November 2017 at 21:59, Chris Barker wrote:

I don't understand any of this enough to have an opinion, so while I'd like to see py.exe be renamed python.exe, let's not let "the perfect be the enemy of good enough". So, if someone can make the case that they can restructure the whole py.exe / python.exe thing nicely in a way that will work, AND write the code to do it fairly quickly, then great!

I'm happy enough to modify the py.exe code to base its behaviour on the name it's called with, as Steve suggested. I'm not sure if that would require a PEP, but I'm willing to do one if it's felt that there is a need. It should be possible to get that change in for 3.7. I don't know what would be the best approach for adding copies/links of the launcher under other names, though. And I don't have any experience with the tools we use to build the installer, so that would be for someone else. That part probably *would* need a PEP, and it may well be controversial enough that getting agreement in time for 3.7 (i.e. in the next 2 months or so) will be challenging. I think the launcher change is worthwhile in its own right. Even if we don't install any aliases by default, users will be able to manually add them. (And someone could write a 3rd party tool to manage creation of such aliases, making that capability available to users who don't have the necessary skills themselves).

Otherwise, let's at least get a python3.exe into 3.7 -- and ideally into updates to 3.5, 3.6, and (python2.exe in this case, 2.7)

It's not going to be acceptable for 3.5, as that is now in security fix only mode. I'm not certain it's even acceptable for bugfix-only releases. It's a backward incompatible change (3.6.3 has no python3.exe but python 3.6.4 does) but maybe an acceptable one - we'd need feedback from the 3.6 and 2.7 release managers on that. Also, I'm assuming here you mean "create a copy of python.exe called python3.exe". If we do that, we risk making it harder to later switch to making "python3.exe" a link to the launcher - for the same reason that making "python.exe" be an alternative name for the launcher is problematic right now.

And maybe make "add to PATH" be the default in the installer.

I'm not willing to contradict Steve on this one. There are too many not-uncommon cases that we could mess up badly here. It's the right choice for "make new users' experience as easy as possible", but if we do this at the cost of making the experience of people upgrading to 3.7 (possibly by installing 3.7 to gradually switch over, or starting their migration from 2.7 with 3.7) unpleasant, then we risk getting bad publicity for the 3.7 release. The history of how we added Python to PATH across various versions is messy and inconsistent. Add to that other distributions (Anaconda, ActiveState) making different choices and adding yet more inconsistency, means that anyone who *isn't* a brand new user probably already has a tweaked, and likely fragile, setup. We're not doing them any favours by adding another new behaviour. (And enabling "Add to PATH" in 3.7 *would* be a new behaviour - I don't think we've enabled add to path by default in any version since we switched to per-user installs as the default). We need *another* solution here. Not just variations on the existing mess. That's why I like Steve's suggestion of making the launcher into the canonical entry point. It's not easy, but at least it stands a chance of breaking out of the cycle we're currently in, of switching back and forth between "add to PATH" and "don't add to PATH".

Those are quick and simple to do, result in little disruption, and make the whole cross-platform thing more manageable.

They aren't that simple. I've already discussed "add to PATH". And if we add python3.exe, we have to consider questions like will venv be modified to include it in virtual environments? Will virtualenv? If not, will "python3" run the system Python rather than the current virtualenv? That's just as much a problem for the option of having python3 be an alias for the launcher, of course - but my point here is you have to think about questions like this even for your "simple" approach. I just don't think there *is* a quick and simple solution here. So better not to rush into a solution that isn't fully thought through, IMO. "Although never is often better than *right* now" is the relevant Zen here, not "Now is better than never" (nobody's suggesting we *never* fix this).

BTW -- does pip install a "pip3" on Windows?

No. Just "pip.exe". Paul

On Mon, Nov 20, 2017 at 3:24 PM, Paul Moore wrote:

...

I don't understand any of this enough to have an opinion, so while I'd

On 20 November 2017 at 21:59, Chris Barker wrote: like

...

to see py.exe be renamed python.exe, let's not let "the perfect be the enemy of good enough". So, if someone can make the case that they can restructure the whole py.exe / python.exe thing nicely in a way that will work, AND write the code to do it fairly quickly, then great!

I'm happy enough to modify the py.exe code to base its behaviour on the name it's called with, as Steve suggested. I'm not sure if that would require a PEP, but I'm willing to do one if it's felt that there is a need. It should be possible to get that change in for 3.7.

very cool -- thanks!

I don't know what would be the best approach for adding copies/links of the launcher under other names, though.

this is the thing -- I wonder if py.exe has been a success at all :-) I honestly don't know - it's a cool idea, and I liked the idea of replacing the #! line and all that, but the truth is that people simple don't write scripts on Windows and expect them to "just work" the same way they do on *nix systems. And if you do distutils/setuptools "scripts" or "entry points" or whatever we are calling them, right, then you can write utility scripts and install them, and use them.... So ti comes down to: does anyone USE py.exe??? The fact that it's unique to Windows makes it simply less discoverable, and the fact that Windows users aren't used to the whole #! line thing, means that as useful as it might be, it's not getting actual use. In fact, I happened to be paying a lot of attention to this conversion because at the very same moment, a group of us are developing a python curriculum, and working on the "how to run a python file" part. We are producing platform specific instructions, but decided NOT to include the py.exe option for Windows users .. just more confusing things among a lot of confusing things! So who does teach people about it??? Anyway, this whole thread has been about making the experience more platform-independent -- ideally users should see docs without Windows-specifc advice, and have it "just work" -- having a python3.exe would make that more likely, so let's do that. If we could py.exe enhanced and then use it instead of python.ex,m that would be great. Though I at least still have no idea of that's possible. But I don't see why we couldn't make a pyton3.exe NOW (in py3.7) that is a copy (Or link, still don't get linking on Windows...) of python.exe, that would help the situation, and if at some point the in the future, python.exe, python3.exe and py.exe all become the same thing, great!

I think the launcher change is worthwhile in its own right. Even if we don't install any aliases by default, users will be able to manually add them.

sure -- but no one would -- or not really no one, but it would become maybe even a worse hassle of the same thing not working everywhere. we need th=e basic advice given out to *nix users to work: if you type "python" at the command line, and get python2, then you can type "python3" to get python 3. simple, straightforward, and it CAN help.

...

Otherwise, let's at least get a python3.exe into 3.7 -- and ideally into updates to 3.5, 3.6, and (python2.exe in this case, 2.7)

It's not going to be acceptable for 3.5, as that is now in security fix only mode. I'm not certain it's even acceptable for bugfix-only releases. It's a backward incompatible change (3.6.3 has no python3.exe but python 3.6.4 does) but maybe an acceptable one - we'd need feedback from the 3.6 and 2.7 release managers on that.

Fine -- we can leave that for future discussion....

Also, I'm assuming here you mean "create a copy of python.exe called python3.exe". If we do that, we risk making it harder to later switch to making "python3.exe" a link to the launcher - for the same reason that making "python.exe" be an alternative name for the launcher is problematic right now.

I'm lost -- how does it make it any harder, rather than exactly the same ??? (except two changes, rather than one...but they are the same change)

And maybe make "add to PATH" be the default in the installer.

I'm not willing to contradict Steve on this one.

I didn't know he'd made a clear statement about it. But if people that understand these things say it's a bad idea, then it's a bad idea. But we SHOULD make sure we are considering not just what could go wrong, or how often it could go wrong, but who it is going to go wrong for... i.e. making it "do the right thing" for newbies is much more important that making sure it doesn't "do the wrong thing" for more experienced folks that can uncheck a box, or clean up their PATH after the fact.... There are too many

not-uncommon cases that we could mess up badly here. It's the right choice for "make new users' experience as easy as possible", but if we do this at the cost of making the experience of people upgrading to 3.7 (possibly by installing 3.7 to gradually switch over,

then you uncheck the box -- and if someone is installing 3.7 to test it out, they SHOULD be a experience enough to uncheck a box...

or starting their migration from 2.7 with 3.7) unpleasant,

why would that be a problem???? Though we may need a "python2.exe" executable for that to work well :-(

The history of how we added Python to PATH across various versions is messy and inconsistent. Add to that other distributions (Anaconda, ActiveState) making different choices and adding yet more inconsistency, means that anyone who *isn't* a brand new user probably already has a tweaked, and likely fragile, setup.

We're not doing them

any favours by adding another new behaviour.

I'm not trying to do them any favors -- I'm trying to do the favors for the newbies.. and YES, at the expense of people with "tweaked, and likely fragile, setup."

We need *another* solution here. Not just variations on the existing mess. That's why I like Steve's suggestion of making the launcher into the canonical entry point.

only if it's named "python" (and python3) -- what we DON'T need is a different way to do it on Windows -- that's what we tried, and I thin it's failed. It's not easy, but at least it stands a

chance of breaking out of the cycle we're currently in, of switching back and forth between "add to PATH" and "don't add to PATH".

It also seems completely orthogonal to the "add to PATH" question -- having this nifty new launcher setup, but not on a newbies PATH where they can use it, isn't going to help at all.

...

Those are quick and simple to do, result in little disruption, and make the whole cross-platform thing more manageable.

They aren't that simple. I've already discussed "add to PATH". And if we add python3.exe, we have to consider questions like will venv be modified to include it in virtual environments? Will virtualenv?

what do those systems do on *nix? do exactly the same thing on Windows. Done. Also -- I don't much care -- I want newbies that aren't using virtual environments to have a better expeience -- virtualenv pretty much solves this issue IF they are being used.

"Although never is often better than *right* now" is the relevant Zen here, not "Now is better than never" (nobody's suggesting we *never* fix this).

if we take long enough, everyone will be using python3 anyway :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

[Excuse any attribution errors, the quoting appears to have got badly messed up somehow] On 21 November 2017 at 06:21, Steve Barnes wrote:

On 21/11/2017 00:32, Chris Barker wrote:

...

I don't know what would be the best approach for adding copies/links of the launcher under other names, though.

this is the thing -- I wonder if py.exe has been a success at all :-)

A lot of people, on windows, are using py.exe and pyw.exe without realising it - the default behaviour of the recent installer is to associate py.exe with *.py and pyw.exe with *.pyw so any user that is running python code by just typing the name of the file, or double clicking it, will be using it.

I use the py launcher a lot. Often just as "py" or "py -3.6". It certainly isn't just limited to use via shebang lines. I will say that I don't see beginners using it as much. That's probably because there's so much documentation that says "use the python command to start Python". That is of course an argument that we should make the canonical command "python" rather than "py", but it doesn't mean that if people find the "py" command they won't use it. There's not a lot of evidence that isn't anecdotal either way on that latter point.

...

So ti comes down to: does anyone USE py.exe???

As above.

And me.

...

we need th=e basic advice given out to *nix users to work:

if you type "python" at the command line, and get python2, then you can type "python3" to get python 3.

Um, we need the basic advice given to *everyone* to work. Insisting that advice has to be "what Unix people do right now" is not the only option. The advice "use python to run your default Python interpreter" is too entrenched to change. The python2 and python3 aliases are a platform-specific hack to get around the fact that some Linux distributions couldn't switch the default Python to Python 3. Most of the beginners I work with have no experience on Unix and no awareness of python2/python3 commands. "If you run python at the command line and get the wrong version of Python, then..." - In that case you already have multiple versions of Python on your system, and by the definitions we've been working to here, you're an advanced user so you can fix it yourself. Obviously, that's too harsh a stance for us to take. But let's not ignore the fact that Unix users are typically *already* having to deal with multiple versions of Python, and are typically more familiar with command lines than Windows users. So there's no guarantee that a solution that works for them is usable for Windows users who've never used Python or the command line before, who downloaded Python 2.7 and installed it, were then told "no, you should be using Python 3", and installed 3.7, and now don't understand why "upgrading" didn't uninstall the old version, what they need to do to work with Python 3, etc...

One of the nice things about embedding the behaviour into the launcher is that if the user/admin has installed a python that includes it it provides the behaviour for python installations that were already present and discoverable even if they didn't have it so if you have python 2.5 then after you install something with py.exe you can, currently, use py -2.5 to run it, (or py -2 if it is the only python 2).

+1. This is the biggest benefit for me. Also the fact that all the older Python versions *don't need to be on PATH*. Having multiple versions of Python on PATH is a bad thing (because of the Scripts directory - if you want, I can explain in detail, but it's a side issue for now).

...

I'm not willing to contradict Steve on this one.

I didn't know he'd made a clear statement about it.

I am not sure that I spelt it out well enough but my personal feeling is that the installer should, (and I am reasonably sure that it should capable of this), default to add to path IF there is not a currently available python, (discoverable), and either default to don't add or prompt the installer if there is.

My apologies - I was unclear. I meant Steve Dower, who maintains the installer, and who has strongly stated that the way the Windows PATH works (separate system and user parts) makes sensible "add to PATH" behaviour impossible. The reasons have all been described multiple times, and I won't go into them again. Search the archives if you want to see the details. Suffice it to say that wishing it was easy to "just add Python to PATH" doesn't make it so...

Of course the whole add to path or not becomes moot if the py.exe launcher is present as it is installed on the path, (C:\Windows), and then tries very hard just to "do the right thing".

That's why I prefer this option. There are still issues to address (such as the Scripts directory), but it's much easier to keep the complexity manageable with this approach.

...

But if people that understand these things say it's a bad idea, then it's a bad idea. But we SHOULD make sure we are considering not just what could go wrong, or how often it could go wrong, but who it is going to go wrong for...

i.e. making it "do the right thing" for newbies is much more important that making sure it doesn't "do the wrong thing" for more experienced folks that can uncheck a box, or clean up their PATH after the fact....

There are too many not-uncommon cases that we could mess up badly here. It's the right choice for "make new users' experience as easy as possible", but if we do this at the cost of making the experience of people upgrading to 3.7 (possibly by installing 3.7 to gradually switch over,

then you uncheck the box -- and if someone is installing 3.7 to test it out, they SHOULD be a experience enough to uncheck a box...

But I (as an advanced user) don't *want* to uncheck the box. I *do* want 3.7 to be my default Python. But if I previously had 2.7 as my default, and that's in the *system* part of PATH, then checking the box *doesn't work* - Python 2.7 is still my default. OF COURSE I have enough knowledge to fix the issue. That's not my point. My point is, do we want experienced Python users' first experience with Python 3.7 to be "the installer didn't work right", or "installing 3.7 broke my system, and I had to fix it manually", or something similar? Experienced Python users on Windows have been remarkably tolerant over recent years of our continual changing of where we install Python, whether we add to PATH by default, etc. At some point we need to stop, think things through, and get it right once and for all. Or at least agree to stick with what we decide for longer than a couple of releases.

...

We're not doing them any favours by adding another new behaviour.

I'm not trying to do them any favors -- I'm trying to do the favors for the newbies.. and YES, at the expense of people with "tweaked, and likely fragile, setup."

OK. Maybe "not doing them any favours" is not as well known a phrase as I thought, sorry. What I meant was we're repeatedly penalising such users and we need to stop. We don't need to cater for their specific environments (they know how to do that themselves, and have done so). But we do need to stop breaking their setup every release.

...

We need *another* solution here. Not just variations on the existing mess. That's why I like Steve's suggestion of making the launcher into the canonical entry point.

only if it's named "python" (and python3) -- what we DON'T need is a different way to do it on Windows -- that's what we tried, and I thin it's failed.

"python" yes. Not so much "python3", which I don't believe is a solution anyone *likes* - it's just an expedient for a specific Unix issue. (If you don't believe me, there's plenty of issues on the pip tracker that boil down to Unix users getting confused over which Python version they are executing, so certainly "python3" isn't a perfect solution, or even a particularly good one IMO). Paul

On Sun, Nov 12, 2017 at 8:59 AM, Brendan Barnwell wrote:

On 2017-11-12 05:18, Nick Coghlan wrote:

...

* the `pip install` option really is nicer looking than `python -m pip install`, and it only has actual problems in the presence of multiple Python versions and when upgrading pip itself on Windows (plus: lots of third party guides recommend it, as do pypi.org project pages)

Is there any *advantage* to using `pip install` instead of `python -m install`? If not, could we at least change everything under Python/pip control (e.g., pip documentation) to never recommend `pip` and always recommend `python -m pip` instead, and encourage all third-party documentation to always use `python -m pip` and never use `pip`? Obviously this isn't a full solution, but in the end there's no way we change external third-party documentation, which will always eventually become outdated. Absent that, it seems worthwhile to regularize existing official documentation.

Can we instead make it so that 'pip' and 'python -m pip' *are* actually equivalent? I know there are all kinds of pathological things that can happen, but it seems like we can drive the frequency of this error down more. What if instead of installing a standard entry point, the pip executable was installed as #!/bin/sh exec python -m pip "$@" on Unix-likes, and a pip.bat with the equivalent contents on Windows? (Bonus: maybe this would fix the problem with upgrading pip on Windows?) -n -- Nathaniel J. Smith -- https://vorpus.org

On Nov 13, 2017 6:47 PM, "Nick Coghlan" wrote: On 14 November 2017 at 11:51, Nathaniel Smith wrote:

What if instead of installing a standard entry point, the pip executable was installed as

#!/bin/sh exec python -m pip "$@"

on Unix-likes

It would technically be enough to make the shebang line `#!/usr/bin/env python` so the interpreter used was picked up from the environment, rather than being preconfigured at install time. However, the problem is that you don't know for certain that that python will actually have `pip` installed, so it might just fail with a cryptic error instead. This would still be a massive improvement over the status quo, which in this situation would present a perfect simulacrum of downloading and installing the package you asked for, except then when you start python the import still fails. I did think of another issue: when installing into a virtualenv, we probably want to keep the current system, so explicit/path/bin/pip continues to work as expected. However, `pip` could potentially be updated with a `checkenv` subcommand that complains if `sys.executable` and `shutil.which('python')` don't match (and could presumably include other checks as well). Unfortunately by the time you know to run this command to have already understood the problem and how to fix it :-). That said, this is probably cheap enough we could do it automatically at startup. Does pip know whether it's pip, pip3, etc. that invoked it? I guess sys.argv[0] should tell us?

and a pip.bat with the equivalent contents on Windows? (Bonus: maybe this would fix the problem with upgrading pip on Windows?)

Depending on how the batch file was written, I think the answer to that is "maybe": https://stackoverflow.com/questions/2888976/how-to-make-bat- file-delete-it-self-after-completion/20333152#20333152 Sigh. -n

On 14 November 2017 at 03:08, Nathaniel Smith wrote:

On Nov 13, 2017 6:47 PM, "Nick Coghlan" wrote:

...

and a pip.bat with the equivalent contents on Windows? (Bonus: maybe this would fix the problem with upgrading pip on Windows?)

Depending on how the batch file was written, I think the answer to that is "maybe": https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-s...

Sigh.

Batch files are not suitable for this task. The wrappers have to be executables. See http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for a detailed analysis I did some time ago. Paul

On 14 November 2017 at 10:02, Nathaniel Smith wrote:

On Tue, Nov 14, 2017 at 12:56 AM, Paul Moore wrote:

...

On 14 November 2017 at 03:08, Nathaniel Smith wrote:

...

On Nov 13, 2017 6:47 PM, "Nick Coghlan" wrote:

...

...

and a pip.bat with the equivalent contents on Windows? (Bonus: maybe this would fix the problem with upgrading pip on Windows?)

Depending on how the batch file was written, I think the answer to that is "maybe": https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-s...

Sigh.

Batch files are not suitable for this task. The wrappers have to be executables. See http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for a detailed analysis I did some time ago.

Ah, interesting. My reason for suggesting it in the first place because I was hoping to avoid paying the process spawn overhead twice, but it sounds like this specific trick is misguided all around :-).

It doesn't even save the process overhead if you're running Powershell as your main shell, or running pip from a terminal window in your editor/IDE, ... I really wish Windows *did* provide a usable "shell script" implementation (of any form) but sadly it simply doesn't. Paul

On 13 November 2017 at 04:51, Antoine Pitrou wrote:

On Sun, 12 Nov 2017 23:18:26 +1000 Nick Coghlan wrote:

...

We haven't forgotten our early years - we've just spent years (both individually and collectively) working on the problem of helping people get started with software development, and thus have a very good idea as to what *doesn't* work, as well as what *does* work.

The latter list is currently fairly short:

* having a friend or colleague walk them through it * "Install Parties", like those Django Girls runs (i.e. running a pre-tutorial event, specifically focused on getting a working environment set up) * highly prescriptive learning environments, whether online ones (like Grok Learning, trinket.io, PythonAnywhere, etc), or locally installed ones (like PyCharm Educational Edition, the Anaconda distribution, etc)

Not wanting to nitpick, but I don't think the Anaconda distribution is "highly prescriptive". It's a software distribution with scientific computing as its main focus, but perfectly usable for ordinary Python programming. It's not more prescriptive than Debian, which by its philosophy is directed primarily towards sysadmin crowds but also used by some people on their personal desktops.

I meant prescriptive in the sense of "Here's how to get the binaries and manage upgrades". The non-prescriptive alternative is "Use any Python binary you like, and we'll try to adjust to that". The latter can potentially work given sufficiently experienced instructors (or a suitable mix of instructors with knowledge of different platforms), but it's not easy. (The downside of this being that prescriptive environments for particular workshops is one of the most common ways for beginners to end up with multiple versions of Python installed: whatever they found on their own, plus the version their workshop instructors told them to use).

Also, you can create custom Anaconda-like distributions to provide the desired environment to your students in an executable installer (*). I'm not sure how usable that option is, but it definitely exists, is open source and cross-platform.

(*) https://github.com/conda/constructor

Oh, cool - I didn't know about that. Yeah, that could definitely provide an interesting middle ground between "Choose your own adventure" and "Here's the entire scientific Python stack". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Le 12/11/2017 à 19:02, Stephan Houben a écrit :

2017-11-12 13:20 GMT+01:00 Paul Moore mailto:p.f.moore@gmail.com>:

On 12 November 2017 at 06:19, Michel Desmoulin mailto:desmoulinmichel@gmail.com> wrote:

> Well, not exactly. Do you do python -m venv, or py -x.x -m venv or > pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

I think that it is wrong to think of Debian's "python3" package as purporting to provide the "standard Python install".

This is really more properly though of as the Python *runtime* environment, i.e. the minimum stuff you need to run programs written in Python.

I think it's wrong to have a different setup on different plateforms. Python is a portable language. It's also a language you use for quick scripts and from a lot of non professional devs. Unifying the python experience and making sure it's easy for beginners to get started is a very important goal IMO. Python is actually quite good at it. But we can make it even better.

On Sun, 12 Nov 2017 12:20:45 +0000 Paul Moore wrote:

...

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

Frankly, I don't know where the current discussion comes from, but on two recent Debian and Ubuntu setups, I get: $ dpkg -S /usr/lib/python3.5/venv/__init__.py libpython3.5-stdlib:amd64: /usr/lib/python3.5/venv/__init__.py Which, for the uninitiated, means "the venv module is provided by the Debian/Ubuntu package named libpython3.5-stdlib". That package is, in turn, a dependency of the "python3.5" package. Regards Antoine.

Hi Antoine, The venv module is included, however the pyvenv script is in a separate package python3.5-venv . By the way, I was totally confused by the following text form the doc. https://docs.python.org/3/library/venv.html ======== Deprecated since version 3.6: pyvenv was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6 https://docs.python.org/dev/whatsnew/3.6.html#deprecated-features. Changed in version 3.5: The use of venv is now recommended for creating virtual environments. ======== So many questions: * What is the status of "pyenv" in 3.5? Apparently it is not deprecated there. * What is it replaced by? Apparently "venv", but it doesn't say so explicitly. * Is "venv" the same thing as "python -m venv" discussed earlier? Or is it a different thing? With so many things names so similarly, it is hard to tell * What does it mean for "venv" to be recommend in 3.5 if "pyvenv" is not deprecated there? Also, the link brings us to a long page of "What's New in Python 3.6" . Just searching for "venv" only gives the apparently irrelevant: "venv https://docs.python.org/dev/library/venv.html#module-venv accepts a new parameter --prompt. This parameter provides an alternative prefix for the virtual environment. (Proposed by Łukasz Balcerzak and ported to 3.6 by Stéphane Wirtel in bpo-22829 https://bugs.python.org/issue22829.) " I suppose at that point the newbie gave up and downloaded node.js ;-) Stephan On Sun, 12 Nov 2017 12:20:45 +0000

Paul Moore wrote:

...

...

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

Frankly, I don't know where the current discussion comes from, but on two recent Debian and Ubuntu setups, I get:

$ dpkg -S /usr/lib/python3.5/venv/__init__.py libpython3.5-stdlib:amd64: /usr/lib/python3.5/venv/__init__.py

Which, for the uninitiated, means "the venv module is provided by the Debian/Ubuntu package named libpython3.5-stdlib". That package is, in turn, a dependency of the "python3.5" package.

Regards

Antoine.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben wrote:

Hi Antoine,

The venv module is included, however the pyvenv script is in a separate package python3.5-venv .

By the way, I was totally confused by the following text form the doc.

https://docs.python.org/3/library/venv.html

======== Deprecated since version 3.6: pyvenv was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6.

Changed in version 3.5: The use of venv is now recommended for creating virtual environments.

========

Not sure where you're reading that. I'm seeing: """ Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. """ I think that's pretty clear. "python3 -m venv env" is the standard and recommended way to spin up a virtual environment. ChrisA

On 13 November 2017 at 09:29, Nick Coghlan wrote:

On 13 November 2017 at 07:11, Chris Angelico wrote:

...

On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben wrote:

...

Hi Antoine,

The venv module is included, however the pyvenv script is in a separate package python3.5-venv .

By the way, I was totally confused by the following text form the doc.

https://docs.python.org/3/library/venv.html

======== Deprecated since version 3.6: pyvenv was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6.

Changed in version 3.5: The use of venv is now recommended for creating virtual environments.

========

Not sure where you're reading that. I'm seeing:

""" Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. """

I think that's pretty clear. "python3 -m venv env" is the standard and recommended way to spin up a virtual environment.

It's further down in the page, under https://docs.python.org/3/library/venv.html#creating-virtual-environments

I think the deprecation notice for pyvenv should just be deleted, since it renders like the *module* is deprecated.

That is, the confusing one starting with "Deprecated since version 3.6: ...". The note Chris quoted is fine, and should be kept. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi all, Related to this text on https://docs.python.org/3/library/venv.html : ============ Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. ============ It's clearer than the text below to which I originally referred. However, this text has also problems in that it is too unix-specific. In particular: * Most seriously, it refers to "python3" which doesn't work with the python.org Windows installer. * Less seriously, it refers to "pyenv" as a "script" which is unix jargon and moreover technically incorrect on Windows. (Also, needlessly specific, it should just be "the pyenv command", how it is implemented is irrelevant for this section). Stephan 2017-11-13 0:32 GMT+01:00 Chris Angelico :

On Mon, Nov 13, 2017 at 10:29 AM, Nick Coghlan wrote:

...

On 13 November 2017 at 07:11, Chris Angelico wrote:

...

On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben wrote:

...

Hi Antoine,

The venv module is included, however the pyvenv script is in a separate package python3.5-venv .

By the way, I was totally confused by the following text form the doc.

https://docs.python.org/3/library/venv.html

======== Deprecated since version 3.6: pyvenv was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6.

Changed in version 3.5: The use of venv is now recommended for creating virtual environments.

========

Not sure where you're reading that. I'm seeing:

""" Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. """

I think that's pretty clear. "python3 -m venv env" is the standard and recommended way to spin up a virtual environment.

It's further down in the page, under https://docs.python.org/3/library/venv.html#creating- virtual-environments

I think the deprecation notice for pyvenv should just be deleted, since it renders like the *module* is deprecated.

Ah, I see it now, thanks.

Agreed; or maybe downgrade it to a parenthetical comment. Focus on "this is how to do the obvious thing", and only as an afterthought mention "it used to be done differently" in case someone greps for pyvenv.

ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi Brett, The current documentation *cannot* be fixed, since fixing it would entail adding an initial two-page essay on "how to start Python on various platforms/systems" (it is really NOT as simple as Windows=python, Linux=python3) and such a PR will certainly by rejected. In my opinion, the only alternatives are 1. either harmonize the invocation of python across platforms (and *then* adapt the docs to follow). Which was pretty much the whole topic of this thread so far. 2. or just use "python" consistently across all docs (since that is the *only* command which is at least consistent among python.org installers), and add weasel-wording to "consult documentation of third-party installers" 3. or leave the docs broken for at least some people some of the time. Stephan 2017-11-14 2:31 GMT+01:00 Brett Cannon :

On Mon, Nov 13, 2017, 00:01 Stephan Houben, wrote:

...

Hi all,

Related to this text on https://docs.python.org/3/library/venv.html :

============

Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. ============

It's clearer than the text below to which I originally referred.

However, this text has also problems in that it is too unix-specific. In particular: * Most seriously, it refers to "python3" which doesn't work with the python.org Windows installer.

It can, but it's opt-in. It's just one of those things that's easy to forget.

* Less seriously, it refers to "pyenv" as a "script" which is unix jargon

...

and moreover technically incorrect on Windows. (Also, needlessly specific, it should just be "the pyenv command", how it is implemented is irrelevant for this section).

I disagree with this as Python refers to .Py files that you execute directly as "scripts", so I don't think this requires clarification.

Anyway, a pull request with suggested wording to address your concerns would be the best way to try and rectify the issue.

-brett

...

Stephan

2017-11-13 0:32 GMT+01:00 Chris Angelico :

...

On Mon, Nov 13, 2017 at 10:29 AM, Nick Coghlan wrote:

...

On 13 November 2017 at 07:11, Chris Angelico wrote:

...

On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben wrote:

...

Hi Antoine,

The venv module is included, however the pyvenv script is in a separate package python3.5-venv .

By the way, I was totally confused by the following text form the doc.

https://docs.python.org/3/library/venv.html

======== Deprecated since version 3.6: pyvenv was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6.

Changed in version 3.5: The use of venv is now recommended for creating virtual environments.

========

Not sure where you're reading that. I'm seeing:

""" Note The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on. """

I think that's pretty clear. "python3 -m venv env" is the standard and recommended way to spin up a virtual environment.

It's further down in the page, under https://docs.python.org/3/library/venv.html#creating- virtual-environments

I think the deprecation notice for pyvenv should just be deleted, since it renders like the *module* is deprecated.

Ah, I see it now, thanks.

Agreed; or maybe downgrade it to a parenthetical comment. Focus on "this is how to do the obvious thing", and only as an afterthought mention "it used to be done differently" in case someone greps for pyvenv.

ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 13 November 2017 at 04:38, Antoine Pitrou wrote:

On Sun, 12 Nov 2017 12:20:45 +0000 Paul Moore wrote:

...

...

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

Frankly, I don't know where the current discussion comes from, but on two recent Debian and Ubuntu setups, I get:

$ dpkg -S /usr/lib/python3.5/venv/__init__.py libpython3.5-stdlib:amd64: /usr/lib/python3.5/venv/__init__.py

The discussion comes from the fact that even though the code is present it doesn't actually work: $ docker run --rm -it ncoghlan/debian-python bash root@e9c0aa482aeb:/# python3 -m venv test_venv Error: Command '['/test_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1 (I just refreshed my Docker image to 9.2, so that's the current default behaviour after "apt-get install -y python3") So technically it's ensurepip that's broken by default, but that translates to venv also being broken by default. I haven't worked out what the actual steps needed to fix it are (but I do know that installing "python3-pip" isn't enough the way it is on Fedora - we reconcile ensurepip with our security management policies by having a Recommends dep from python3 to python3-pip, and then patching ensurepip to use rewheel to inject a copy of the system pip into freshly created virtual environments. At least in theory, Debian should be able to do something similar with dirtbike, but whether or not they actually will would be a question for Matthias Klose as the Debian Python maintainer). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 13 November 2017 at 17:46, Stephan Houben wrote:

2017-11-13 3:32 GMT+01:00 Nick Coghlan :

...

So technically it's ensurepip that's broken by default, but that translates to venv also being broken by default.

I haven't worked out what the actual steps needed to fix it are

On Debian, ensurepip is in the python3.5-venv package.

https://packages.debian.org/stretch/amd64/python3.5-venv/filelist

Thanks! I've tweaked the ncoghlan/debian-python Dockerfile to install that in addition to the base Python package (as well as finally enabling automatic rebuilds of that image whenever the base Debian image updates). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 11/13/2017 11:05 AM, Antoine Pitrou wrote:

On Mon, 13 Nov 2017 12:32:06 +1000 Nick Coghlan wrote:

...

On 13 November 2017 at 04:38, Antoine Pitrou wrote:

...

On Sun, 12 Nov 2017 12:20:45 +0000 Paul Moore wrote:

...

...

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install? That's just broken.

Frankly, I don't know where the current discussion comes from, but on two recent Debian and Ubuntu setups, I get:

$ dpkg -S /usr/lib/python3.5/venv/__init__.py libpython3.5-stdlib:amd64: /usr/lib/python3.5/venv/__init__.py

The discussion comes from the fact that even though the code is present it doesn't actually work: [...]

Wow. I had forgotten Debian could be such a user-hostile distribution. I'm not sure what the reason would be to use it as a basis for a training course in Python programming, then (other than the teacher having their own ideological preferences).

For us, it's the *student's* preference. I believe it's better to let students use the machine and environment they're used to, even if it means extra trouble for the instructors. So, we get a healthy mix of Windows, Mac, Debian, Fedora, and sometimes some surprises.

On 13 November 2017 at 20:43, Antoine Pitrou wrote:

On Mon, 13 Nov 2017 11:17:54 +0100 Petr Viktorin wrote:

...

...

Wow. I had forgotten Debian could be such a user-hostile distribution. I'm not sure what the reason would be to use it as a basis for a training course in Python programming, then (other than the teacher having their own ideological preferences).

For us, it's the *student's* preference. I believe it's better to let students use the machine and environment they're used to, even if it means extra trouble for the instructors. So, we get a healthy mix of Windows, Mac, Debian, Fedora, and sometimes some surprises.

In that case the student must be ready to deal with the perils of their own preferences.

It's also currently missing from https://packaging.python.org/guides/installing-using-linux-tools/, but we can amend that now that we know what the required fix is. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

<snip>

...

...

We still have to deal with the fact that basically every Unix environment is "advanced" in the above sense (the python2/python3 split). I don't have a solution for that (other than "upgrade to Windows" ;-)).

Provide the "py" command on linux and mac. And make it the default recommended way in the documentation.

+1 from me.

<snip> How about update pip so that, from the command line it can accept pretty much the same syntax as the py launcher: - "python -m pip operation ..." all use the same python as is running the pip module. - "pip -X[.Y][-32|-64] operation ..." tries to find a python matching -X[.Y][-32|-64] and if it succeeds executes "python -m pip operation ..." with that python, (if it doesn't find a matching python is should fail with a sensible error message and possibly list the valid python specifiers). "pip operation ..." searches the available python contexts and, if there is only one, (or in a venv), calls "python -m pip operation ..." using it but, if there is more than one candidate, (or in not in a venv), tells the user that they have more than one python and need to specify which, listing the options. "pip -0" searches the available pythons and lists the available contexts. Ideally the list contexts operations would differentiate between user space and system contexts and display them suitably based on the current user privileges. This would: - maintain the current behaviour as much as possible, - not break all of the existing examples in the wild & literature - avoid trying to introduce a new launcher command on Mac/Linux, - propagate reasonably quickly, as people are already fairly used to being prompted to update pip, - Automatically propagate to the default pip as long as it was back ported to at least 2.7, possibly further. - Allow users who do need to support multiple python installations to do so, (I am one of those). - Does not require updates to python itself. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com

On 15 November 2017 at 08:22, Nick Coghlan wrote:

On 15 November 2017 at 16:13, Steve Barnes wrote:

...

- "pip -X[.Y][-32|-64] operation ..." tries to find a python matching -X[.Y][-32|-64] and if it succeeds executes "python -m pip operation ..." with that python, (if it doesn't find a matching python is should fail with a sensible error message and possibly list the valid python specifiers).

This is a genuinely interesting option, especially as `pipenv` has already implemented a feature somewhat akin to this: https://docs.pipenv.org/basics.html#specifying-versions-of-python

`pipenv` also allows `pipenv --two` and `pipenv --three` when setting up your initial virtual environment.

This is an interesting idea for *any* tool that's about "working with Python environments" as opposed to "writing Python code". So pip, virtualenv, tox, pipenv, ... Many of these tools are variously reinventing "tell me which Python environment to work on" options. Having a common way to do this would be really useful. I'm not sure how likely it would be for pip to be able to use it (pip introspects sys.executable to get site-packages, etc), but it's certainly a possibility. Having a standardised library/wrapper that handles the "select a Python environment" process would be a huge plus. There's https://github.com/zooba/pep514tools which is a start on handling the Windows registry scanning logic, and building on that to include Unix and anything else we needed would be great. Paul

On Nov 12, 2017 14:21, "Paul Moore" wrote: [....]

Virtualenvs are a hard tool to use for beginners.

From personal experience, ​I do have to say that for my day-to-day normal

Agreed. Genuine beginners just install Python, then use it. If they install extra packages, they want them to be available to all of their scripts. Agreed. python *use*, I try to use a single environment (the default miniconda one) for everything. Most of the time, that's enough. I suppose that makes me a genuine beginner :-). Being able to conveniently installing a package into the current environment from within any interactive interpreter would definitely improve the beginner experience, I would say. ––Koos [..]

On 12 November 2017 at 16:20, Michel Desmoulin wrote:

Le 10/11/2017 à 09:01, Nick Coghlan a écrit :

...

On 10 November 2017 at 17:05, Michel Desmoulin wrote: When we choose not to use venv, then it becomes necessary to ensure each of those things individually for each potential system starting state

The way we do things is not the only way. Take JS: they don't have this problem, you npm install the same way everywhere. You don't have virtualenv but local node_modules. And you have transpilers to help with the langage version differences.

And they have browser manufacturers pouring millions of dollars a year into their tooling ecosystem into order to influence whose ad networks get clicked on most often. Python doesn't have that level of investment, but we do have fine folks volunteering to work on projects like `pip`, `virtualenv`, PyPI, and the packaging.python.org documentation project, as well as backports of standard library modules to earlier Python versions. When someone attempts to explain to you the practical challenges that limit both python-dev's and PyPA's ability to reliably control the starting experience of new Python users, the appropriate response is to *start listening*, not harangue them for failing to immediately follow your peremptory orders to make things simpler for you. "Do it because I said so" is bad management style even in an actual company - it's even worse in a peer production environment like an open source community.

Now I'm not advocating we do it the JS way. I'm just saying that you are very keen to defend a statu quo instead of offering ideas to solve the problem.

I already opened https://github.com/pypa/python-packaging-user-guide/issues/396 to track possible areas of concrete near term improvement (I would have tagged you on the issue, but I couldn't find a GitHub account under your name).

Besides, using venv have the same issues. It's not installed on linux by defaut.

That depends greatly on which Linux you install - the choice on whether to deliberately cripple "python3 -m venv" or not is made by the maintainers for that distribution, and Python-friendly distros make sure Python's native tooling works properly by default.

And on windows you'll have to do py -x.x -m but on mac pythonx -m.

CPython updates take years to reliably roll out to end user systems, so if you're looking to have an impact in a shorter time frame than that, the differences in cross-platform invocation are a constraint you're going to have to learn to live with. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 6 November 2017 at 18:50, Stephan Houben wrote:

Hi Michel,

That's exactly why I proposed a `pip` function available from the Python prompt. I suppose you could still tell your students to copy/paste the following into their Python interpreter.

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split()) print("Please restart Python now to use installed or upgraded packages.")

Depending on where and how Python is installed, this may still not work (while I'll grant that Linux users are more likely to already be familiar with the command line than Windows and Mac OS X users, it's still a non-trivial step from there to realising why "sudo pip install" is almost always a bad idea)

I suppose an alternative is to set up jupyterhub

https://jupyterhub.readthedocs.io/en/latest/

and let all your students just access that from a webbrowser.

Yep, and lots of teachers use services like PythonAnywhere, trinket.io, and similar, precisely because the only thing the learners need locally is a web browser. The main downside is that learning that way isn't quite as transferable a skill, since it completely hides the code packaging and distribution step. However, it's a good option when the primary aim is to teach computational skills, and Python is just the vehicle used for that purpose. Software Carpentry starts out with the Anaconda distribution, as it not only improves the cross-platform UX consistent situation, it also deals with the external binary dependency problem (at least for the core set of packages provided either natively or via conda-forge). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Software Carpentry starts out with the Anaconda distribution, as it not only improves the cross-platform UX consistent situation, it also deals with the external binary dependency problem (at least for the core set of packages provided either natively or via conda-forge).

Yeah but the knowledge is not transposable. You can't understand a lot of tutorials that uses pip. Plus it has licences issues. And if somebody starts mixing conda and pip, it can results in strange things. I do use anaconda, but you can't rely solely on it.

On 2017-11-06 19:04, Michel Desmoulin wrote:

...

I don't see anything particularly bogging here. It's always like this when you have multiple versions of the same software on the system. There's only one PATH, after all.

Heck, *the mere fact that Python allows to work like this is already a huge leap forward.* **Doing this cross-platform with exactly the same steps is something few could believe was even possible a couple of years ago!**

I agree.

...

To simplify things with Python, i do the following: * use the system's Python whenever possible

So python 2.7 on mac and some linux or none for windows...

...

* if using something else, only install the one version/environment that I'm using day-to-day and add it to PATH (system's if safe & convenient, personal otherwise) * prefer the system's/environment's package manager to pip to install 3rd-party modules, too, if there is one.

We can't solve the situation perfectly, but we can unify a bit. E.G:

- provide the "py" command on all OSes to avoid the various naming and aliases of python - promote it in documentation - promote the use of py -x.x -m pip instead of the myriads of alternatives

I'm sure there's a reason why these acrobatics are required on Windows, but on other OS's separate "python3", "python3.6", "python2", "python2.7" and "python" executables/symlinks all in the PATH work perfectly fine. As Ivan said earlier, perhaps the Windows installers should provide a "python3" executable, so "python3 -m pip" works everywhere. I normally use "pip2" and "pip3", but those aren't available everywhere, and may be hard to support properly.

- provide an empty pip and venv module. If they are not here, py -m pip says "your plateform doesn't provide pip by default, please do xxxxx" to install it. With "xxxx" being plateform specific.

Isn't pip installed by default on Windows and OSX? On Linux some distribution will probably do something unexpected and mess up your grand plan.

- check "add python executable to system path" on the windows installer once by defaut

I like this, but I have a suspicion that this is rather unorthodox. -- Thomas Jollans

On 7 November 2017 at 20:38, Chris Barker wrote:

On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans wrote:

...

As Ivan said earlier, perhaps the Windows installers should provide a "python3" executable, so "python3 -m pip" works everywhere.

absolutely! I really, really thought it did!!!! (I'm amazed I never heard from a single student getting bit by that...)

On Windows, use py -X.Y to select the exact version of Python you want. Maybe Unix should have a launcher like this, too? It doesn't really need to be any more complex than exec pythonX.Y $@ relying on the existence of versioned executables on Unix. (Excuse my garbled don't-really-know-how-to-do-it shell scripting...) Paul

Le 07/11/2017 à 22:26, Paul Moore a écrit :

On 7 November 2017 at 20:38, Chris Barker wrote:

...

On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans wrote:

...

As Ivan said earlier, perhaps the Windows installers should provide a "python3" executable, so "python3 -m pip" works everywhere.

absolutely! I really, really thought it did!!!! (I'm amazed I never heard from a single student getting bit by that...)

On Windows, use py -X.Y to select the exact version of Python you want. Maybe Unix should have a launcher like this, too? It doesn't really need to be any more complex than

exec pythonX.Y $@

That's my whole point. We should find a way to unify each of those issues.

there can be multiple python2 or 3s too...

at least with:

python? -m pip install

you will get the pip that matches the python you use...

Not on windows. You have the py command. My all point is that each of the specific issues I mentioned are A and B but not C propositions. It's not unified.

with ensurepip, having pip no installed in a python is getting less common, so maybe this isn't needed anymore, but....

ensurepip may depend of youself having an internet connection when you install it. And without a proxy. And it's not used on debian flavours.

-CHB  

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R            (206) 526-6959 tel:(206)%20526-6959   voice 7600 Sand Point Way NE   (206) 526-6329 tel:(206)%20526-6329   fax Seattle, WA  98115       (206) 526-6317 tel:(206)%20526-6317   main reception

Chris.Barker@noaa.gov mailto:Chris.Barker@noaa.gov

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Isn't pip installed by default on Windows and OSX? On Linux some distribution will probably do something unexpected and mess up your grand plan.

Some installers download pip, which won't work if you don't have an internet connection or a proxy setup. But my point is, all this should be unified. All the points I mentioned worked in a A and B but not C, or a combination of those, different for each point. This is very unfortunate to write docs, tutorials, or any teaching material.

Do you know about "fades"?

https://fades.readthedocs.io/en/release_6_0/

So when they read any tutorial online they don't have reusable knowledge ?