On 3/25/20, Steve Barnes <GadgetSteve@live.co.uk> wrote:
That is not consistent with Unix. env is supposed to search PATH for the command. However, the launcher does not search PATH for a versioned command such as "python3". Instead it uses the highest version that's registered for 3.x or 2.x, respectively, or the version set by PY_PYTHON3 or PY_PYTHON2 if defined, respectively.
#!/usr/bin/env python # Launch with the latest/preferred version of python 2 unless PY_PYTHON=3[.n[-64/32]] is set or py.ini has the same in.
In this case "env" first searches PATH before falling back on registered installations and PY_PYTHON, which is correct -- at least for the PATH search. I would prefer that "env" never checks registered installations. For the registry fallback, it should instead check the user and system "App Paths" key, like what ShellExecuteExW does.
I think the reasoning is that the whole point of the py.exe is to avoid having users edit their PATH on Windows. And further the thinking goes that you do not need the alternatively named python programs. Is that your understanding as well Eryk? Barry
On Wed, 25 Mar 2020 at 11:43, Barry Scott <barry@barrys-emacs.org> wrote:
The alternatively named programs have never been needed on Windows because the issue that prompted needing them on Unix (system scripts that referred to "/usr/bin/python" expecting to get Python 2) didn't exist on Windows. The py launcher was introduced to support file associations for .py[w] files, and to support shebangs. It became common as a command line utility because it's convenient to not need to edit PATH. The relevant PEPs for the history are PEP 397 for the launcher and PEP 394 for the background on versioned names on Unix (although I'm not sure all distros actually follow PEP 394...) The only reason anyone has ever suggested versioned executables on Windows is for Unix compatibility - the reasons they are needed on Unix simply don't apply on Windows (at least not in my experience - it's possible that some peoplehave workflows that need versioned executables, rather than simply using absolute paths or the launcher). Paul
On Mar 25, 2020, at 05:02, Paul Moore <p.f.moore@gmail.com> wrote:
The obvious exception is exactly the one the OP has: they work primarily in Cygwin, but use native Windows rather than Cygwin Python, so it’s Cygwin bash scripts (and Linux-familiar users at the Cygwin shell) launching Python. In that case, the reasons they’re needed on Unix do apply. However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
Either a bash alias or a bash script called python3 that runs py.exe with the right args is surely the fix? alias python3='/mnt/c/WINDOWS/py -3' I tested that in cygwin and it works fine. Only odd thing I noticed is that to get a REPL I have to call as "py -3 -i" from cygwin.
However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
Not if you need to get a Windows APIs and services it is not an answer. I think solutions are available without changes to python for the OPs problem. Barry
On 3/25/20, Barry Scott <barry@barrys-emacs.org> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead, but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment. I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
Nope, I agree with your point about that running 2.x on Unix. IMO, the /usr/bin/python* and /usr/bin/env aliases are for Unix compatibility and so should prioritise that - but because Windows python doesn't include versioned executables, it's not always easy to do that.
In principle I agree, but I'm not sure how you see that working in practice. If you have an activated venv, `#!/usr/bin/env python3` should use the venv Python or the system Python depending on whether the venv is Python 3 or not. But the only way of finding that out in the absence of versioned executables is running `python -V`. That's a performance hit that I'd rather we avoided. [see below about pyvenv.cfg - tl;dr is that there's no *documented* version info in there]
I'd agree with that, if we had versioned executables. Without them, I honestly don't know what answer would cause the fewest issues, so I tend to assume that sticking with what we have is good enough, because we're not getting lots of complaints (we get some, but not that many). Following on from my comment above about needing to run `python -V`, I'd completely forgotten that pyvenv.cfg held the version statically nowadays. But virtualenv 20.0 and later writes a pyvenv.cfg with different information, and virtualenv < 20.0 doesn't write pyvenv.cfg at all. I'm OK with discounting old versions of virtualenv, but realistically, I believe that not interoperating with current versions of virtualenv is impractical. I think that if we want to rely on pyvenv.cfg, we should document/standardise its contents. At the moment, the docs only mention `home` and `include-system-site-packages`, and to be honest they sound more informational than normative anyway. Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment. So for now, at least, it's difficult to see how the launcher can behave as you want (I assume it's obvious that we don't want to hard-code implementation details of virtualenv into the launcher). I'd support a standardisation effort, but I don't intend to champion such an effort myself. Just to reiterate, I'd like a better, more uniform solution here. But I think there's more complexity than people are assuming, at least if we want to interoperate with 3rd party tools (which I view as necessary, although I guess others may take a more hard-line stance). Paul
On Thu, 26 Mar 2020 at 15:52, Barry Scott <barry@barrys-emacs.org> wrote:
... which is probably more evidence in favour of some sort of point, but at this stage I have no idea what, unless it's "nothing is consistent and you need to be prepared to deal with cross-platform differences" :-) Paul
On Wed, Mar 25, 2020 at 9:57 AM Paul Moore <p.f.moore@gmail.com> wrote:
3.9 adds "prompt" to record any custom prompt that was specified.
Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment.
I have interest. :)
Up to this point the only thing the Launcher has is support for when the VIRTUAL_ENV environment variable is defined (my hope to standardize on environment naming seems to have stalled out).
I don't either ATM, but maybe someday.
I think the reasoning is that the whole point of the py.exe is to avoid having users edit their PATH on Windows. And further the thinking goes that you do not need the alternatively named python programs. Is that your understanding as well Eryk? Barry
On Wed, 25 Mar 2020 at 11:43, Barry Scott <barry@barrys-emacs.org> wrote:
The alternatively named programs have never been needed on Windows because the issue that prompted needing them on Unix (system scripts that referred to "/usr/bin/python" expecting to get Python 2) didn't exist on Windows. The py launcher was introduced to support file associations for .py[w] files, and to support shebangs. It became common as a command line utility because it's convenient to not need to edit PATH. The relevant PEPs for the history are PEP 397 for the launcher and PEP 394 for the background on versioned names on Unix (although I'm not sure all distros actually follow PEP 394...) The only reason anyone has ever suggested versioned executables on Windows is for Unix compatibility - the reasons they are needed on Unix simply don't apply on Windows (at least not in my experience - it's possible that some peoplehave workflows that need versioned executables, rather than simply using absolute paths or the launcher). Paul
On Mar 25, 2020, at 05:02, Paul Moore <p.f.moore@gmail.com> wrote:
The obvious exception is exactly the one the OP has: they work primarily in Cygwin, but use native Windows rather than Cygwin Python, so it’s Cygwin bash scripts (and Linux-familiar users at the Cygwin shell) launching Python. In that case, the reasons they’re needed on Unix do apply. However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
Either a bash alias or a bash script called python3 that runs py.exe with the right args is surely the fix? alias python3='/mnt/c/WINDOWS/py -3' I tested that in cygwin and it works fine. Only odd thing I noticed is that to get a REPL I have to call as "py -3 -i" from cygwin.
However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
Not if you need to get a Windows APIs and services it is not an answer. I think solutions are available without changes to python for the OPs problem. Barry
On 3/25/20, Barry Scott <barry@barrys-emacs.org> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead, but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment. I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
Nope, I agree with your point about that running 2.x on Unix. IMO, the /usr/bin/python* and /usr/bin/env aliases are for Unix compatibility and so should prioritise that - but because Windows python doesn't include versioned executables, it's not always easy to do that.
In principle I agree, but I'm not sure how you see that working in practice. If you have an activated venv, `#!/usr/bin/env python3` should use the venv Python or the system Python depending on whether the venv is Python 3 or not. But the only way of finding that out in the absence of versioned executables is running `python -V`. That's a performance hit that I'd rather we avoided. [see below about pyvenv.cfg - tl;dr is that there's no *documented* version info in there]
I'd agree with that, if we had versioned executables. Without them, I honestly don't know what answer would cause the fewest issues, so I tend to assume that sticking with what we have is good enough, because we're not getting lots of complaints (we get some, but not that many). Following on from my comment above about needing to run `python -V`, I'd completely forgotten that pyvenv.cfg held the version statically nowadays. But virtualenv 20.0 and later writes a pyvenv.cfg with different information, and virtualenv < 20.0 doesn't write pyvenv.cfg at all. I'm OK with discounting old versions of virtualenv, but realistically, I believe that not interoperating with current versions of virtualenv is impractical. I think that if we want to rely on pyvenv.cfg, we should document/standardise its contents. At the moment, the docs only mention `home` and `include-system-site-packages`, and to be honest they sound more informational than normative anyway. Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment. So for now, at least, it's difficult to see how the launcher can behave as you want (I assume it's obvious that we don't want to hard-code implementation details of virtualenv into the launcher). I'd support a standardisation effort, but I don't intend to champion such an effort myself. Just to reiterate, I'd like a better, more uniform solution here. But I think there's more complexity than people are assuming, at least if we want to interoperate with 3rd party tools (which I view as necessary, although I guess others may take a more hard-line stance). Paul
On Thu, 26 Mar 2020 at 15:52, Barry Scott <barry@barrys-emacs.org> wrote:
... which is probably more evidence in favour of some sort of point, but at this stage I have no idea what, unless it's "nothing is consistent and you need to be prepared to deal with cross-platform differences" :-) Paul
On Wed, Mar 25, 2020 at 9:57 AM Paul Moore <p.f.moore@gmail.com> wrote:
3.9 adds "prompt" to record any custom prompt that was specified.
Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment.
I have interest. :)
Up to this point the only thing the Launcher has is support for when the VIRTUAL_ENV environment variable is defined (my hope to standardize on environment naming seems to have stalled out).
I don't either ATM, but maybe someday.
participants (5)
-
Andrew Barnert -
Barry Scott -
Brett Cannon -
Eryk Sun -
Paul Moore