On 3/25/20, Steve Barnes <GadgetSteve@live.co.uk> wrote:
Except it's not necessarily what the original post wants. The OP wants the shebang "#!/usr/bin/env python3" to "work everywhere by default", for which I assume it's implied that it should work consistently everywhere. I'd prefer for the launcher's env search to also support versioned "pythonX[.Y][-32|-64]" commands such as "python3".
The windows launcher already does support this with shebangs of: #!/usr/bin/env python3 # Launch with the latest/preferred version of python3 #!/usr/bin/env python2 # Launch with the latest/preferred version of python2
That is not consistent with Unix. env is supposed to search PATH for the command. However, the launcher does not search PATH for a versioned command such as "python3". Instead it uses the highest version that's registered for 3.x or 2.x, respectively, or the version set by PY_PYTHON3 or PY_PYTHON2 if defined, respectively.
#!/usr/bin/env python # Launch with the latest/preferred version of python 2 unless PY_PYTHON=3[.n[-64/32]] is set or py.ini has the same in.
In this case "env" first searches PATH before falling back on registered installations and PY_PYTHON, which is correct -- at least for the PATH search. I would prefer that "env" never checks registered installations. For the registry fallback, it should instead check the user and system "App Paths" key, like what ShellExecuteExW does.
On 25 Mar 2020, at 09:15, Eryk Sun <eryksun@gmail.com> wrote:
On 3/25/20, Steve Barnes <GadgetSteve@live.co.uk> wrote:
Except it's not necessarily what the original post wants. The OP wants the shebang "#!/usr/bin/env python3" to "work everywhere by default", for which I assume it's implied that it should work consistently everywhere. I'd prefer for the launcher's env search to also support versioned "pythonX[.Y][-32|-64]" commands such as "python3".
The windows launcher already does support this with shebangs of: #!/usr/bin/env python3 # Launch with the latest/preferred version of python3 #!/usr/bin/env python2 # Launch with the latest/preferred version of python2
That is not consistent with Unix. env is supposed to search PATH for the command. However, the launcher does not search PATH for a versioned command such as "python3". Instead it uses the highest version that's registered for 3.x or 2.x, respectively, or the version set by PY_PYTHON3 or PY_PYTHON2 if defined, respectively.
I think the reasoning is that the whole point of the py.exe is to avoid having users edit their PATH on Windows. And further the thinking goes that you do not need the alternatively named python programs. Is that your understanding as well Eryk? Barry
#!/usr/bin/env python # Launch with the latest/preferred version of python 2 unless PY_PYTHON=3[.n[-64/32]] is set or py.ini has the same in.
In this case "env" first searches PATH before falling back on registered installations and PY_PYTHON, which is correct -- at least for the PATH search. I would prefer that "env" never checks registered installations. For the registry fallback, it should instead check the user and system "App Paths" key, like what ShellExecuteExW does. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YX4DOI... Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, 25 Mar 2020 at 11:43, Barry Scott <barry@barrys-emacs.org> wrote:
I think the reasoning is that the whole point of the py.exe is to avoid having users edit their PATH on Windows. And further the thinking goes that you do not need the alternatively named python programs.
The alternatively named programs have never been needed on Windows because the issue that prompted needing them on Unix (system scripts that referred to "/usr/bin/python" expecting to get Python 2) didn't exist on Windows. The py launcher was introduced to support file associations for .py[w] files, and to support shebangs. It became common as a command line utility because it's convenient to not need to edit PATH. The relevant PEPs for the history are PEP 397 for the launcher and PEP 394 for the background on versioned names on Unix (although I'm not sure all distros actually follow PEP 394...) The only reason anyone has ever suggested versioned executables on Windows is for Unix compatibility - the reasons they are needed on Unix simply don't apply on Windows (at least not in my experience - it's possible that some peoplehave workflows that need versioned executables, rather than simply using absolute paths or the launcher). Paul
On Mar 25, 2020, at 05:02, Paul Moore <p.f.moore@gmail.com> wrote:
The only reason anyone has ever suggested versioned executables on Windows is for Unix compatibility - the reasons they are needed on Unix simply don't apply on Windows (at least not in my experience - it's possible that some peoplehave workflows that need versioned executables, rather than simply using absolute paths or the launcher).
The obvious exception is exactly the one the OP has: they work primarily in Cygwin, but use native Windows rather than Cygwin Python, so it’s Cygwin bash scripts (and Linux-familiar users at the Cygwin shell) launching Python. In that case, the reasons they’re needed on Unix do apply. However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
On 25 Mar 2020, at 17:02, Andrew Barnert <abarnert@yahoo.com> wrote:
On Mar 25, 2020, at 05:02, Paul Moore <p.f.moore@gmail.com> wrote:
The only reason anyone has ever suggested versioned executables on Windows is for Unix compatibility - the reasons they are needed on Unix simply don't apply on Windows (at least not in my experience - it's possible that some peoplehave workflows that need versioned executables, rather than simply using absolute paths or the launcher).
The obvious exception is exactly the one the OP has: they work primarily in Cygwin, but use native Windows rather than Cygwin Python, so it’s Cygwin bash scripts (and Linux-familiar users at the Cygwin shell) launching Python. In that case, the reasons they’re needed on Unix do apply.
Either a bash alias or a bash script called python3 that runs py.exe with the right args is surely the fix? alias python3='/mnt/c/WINDOWS/py -3' I tested that in cygwin and it works fine. Only odd thing I noticed is that to get a REPL I have to call as "py -3 -i" from cygwin.
However, I’m pretty sure the traditional answer for that use case has always been to use Cygwin Python, which I’d assume (it’s been nearly a decade since I’ve dealt with this…) follows the Python-on-Unix naming PEP as well as whichever linux distro inspires its packaging.
Not if you need to get a Windows APIs and services it is not an answer. I think solutions are available without changes to python for the OPs problem. Barry
On 3/25/20, Barry Scott <barry@barrys-emacs.org> wrote:
On 25 Mar 2020, at 09:15, Eryk Sun <eryksun@gmail.com> wrote:
That is not consistent with Unix. env is supposed to search PATH for the command. However, the launcher does not search PATH for a versioned command such as "python3". Instead it uses the highest version that's registered for 3.x or 2.x, respectively, or the version set by PY_PYTHON3 or PY_PYTHON2 if defined, respectively.
I think the reasoning is that the whole point of the py.exe is to avoid having users edit their PATH on Windows. And further the thinking goes that you do not need the alternatively named python programs.
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead, but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment. I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead,
Nope, I agree with your point about that running 2.x on Unix. IMO, the /usr/bin/python* and /usr/bin/env aliases are for Unix compatibility and so should prioritise that - but because Windows python doesn't include versioned executables, it's not always easy to do that.
but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment.
In principle I agree, but I'm not sure how you see that working in practice. If you have an activated venv, `#!/usr/bin/env python3` should use the venv Python or the system Python depending on whether the venv is Python 3 or not. But the only way of finding that out in the absence of versioned executables is running `python -V`. That's a performance hit that I'd rather we avoided. [see below about pyvenv.cfg - tl;dr is that there's no *documented* version info in there]
I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
I'd agree with that, if we had versioned executables. Without them, I honestly don't know what answer would cause the fewest issues, so I tend to assume that sticking with what we have is good enough, because we're not getting lots of complaints (we get some, but not that many). Following on from my comment above about needing to run `python -V`, I'd completely forgotten that pyvenv.cfg held the version statically nowadays. But virtualenv 20.0 and later writes a pyvenv.cfg with different information, and virtualenv < 20.0 doesn't write pyvenv.cfg at all. I'm OK with discounting old versions of virtualenv, but realistically, I believe that not interoperating with current versions of virtualenv is impractical. I think that if we want to rely on pyvenv.cfg, we should document/standardise its contents. At the moment, the docs only mention `home` and `include-system-site-packages`, and to be honest they sound more informational than normative anyway. Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment. So for now, at least, it's difficult to see how the launcher can behave as you want (I assume it's obvious that we don't want to hard-code implementation details of virtualenv into the launcher). I'd support a standardisation effort, but I don't intend to champion such an effort myself. Just to reiterate, I'd like a better, more uniform solution here. But I think there's more complexity than people are assuming, at least if we want to interoperate with 3rd party tools (which I view as necessary, although I guess others may take a more hard-line stance). Paul
On 25 Mar 2020, at 16:53, Paul Moore <p.f.moore@gmail.com> wrote:
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead,
Nope, I agree with your point about that running 2.x on Unix.
At least on Fedora python is python3 now. On Centos 8 there is no python until you use the alternatives mechanism to set it as python2 or python3. Barry
IMO, the /usr/bin/python* and /usr/bin/env aliases are for Unix compatibility and so should prioritise that - but because Windows python doesn't include versioned executables, it's not always easy to do that.
but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment.
In principle I agree, but I'm not sure how you see that working in practice. If you have an activated venv, `#!/usr/bin/env python3` should use the venv Python or the system Python depending on whether the venv is Python 3 or not. But the only way of finding that out in the absence of versioned executables is running `python -V`. That's a performance hit that I'd rather we avoided.
[see below about pyvenv.cfg - tl;dr is that there's no *documented* version info in there]
I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
I'd agree with that, if we had versioned executables. Without them, I honestly don't know what answer would cause the fewest issues, so I tend to assume that sticking with what we have is good enough, because we're not getting lots of complaints (we get some, but not that many).
Following on from my comment above about needing to run `python -V`, I'd completely forgotten that pyvenv.cfg held the version statically nowadays.
But virtualenv 20.0 and later writes a pyvenv.cfg with different information, and virtualenv < 20.0 doesn't write pyvenv.cfg at all. I'm OK with discounting old versions of virtualenv, but realistically, I believe that not interoperating with current versions of virtualenv is impractical. I think that if we want to rely on pyvenv.cfg, we should document/standardise its contents. At the moment, the docs only mention `home` and `include-system-site-packages`, and to be honest they sound more informational than normative anyway.
Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment. So for now, at least, it's difficult to see how the launcher can behave as you want (I assume it's obvious that we don't want to hard-code implementation details of virtualenv into the launcher). I'd support a standardisation effort, but I don't intend to champion such an effort myself.
Just to reiterate, I'd like a better, more uniform solution here. But I think there's more complexity than people are assuming, at least if we want to interoperate with 3rd party tools (which I view as necessary, although I guess others may take a more hard-line stance).
Paul _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ITB4WD... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, 26 Mar 2020 at 15:52, Barry Scott <barry@barrys-emacs.org> wrote:
On 25 Mar 2020, at 16:53, Paul Moore <p.f.moore@gmail.com> wrote:
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead,
Nope, I agree with your point about that running 2.x on Unix.
At least on Fedora python is python3 now.
On Centos 8 there is no python until you use the alternatives mechanism to set it as python2 or python3.
... which is probably more evidence in favour of some sort of point, but at this stage I have no idea what, unless it's "nothing is consistent and you need to be prepared to deal with cross-platform differences" :-) Paul
On Wed, Mar 25, 2020 at 9:57 AM Paul Moore <p.f.moore@gmail.com> wrote:
On Wed, 25 Mar 2020 at 16:10, Eryk Sun <eryksun@gmail.com> wrote:
The py launcher's "env" command searches PATH for anything from "python" to "notepad" -- but not for a versioned Python command such as "python3" or "python2". It always uses a registered installation in this case, which is at the very least problematic when using "#!/usr/bin/env python3" in an active virtual environment. Paul Moore will probably suggest that the script should use "#!/usr/bin/env python" instead,
Nope, I agree with your point about that running 2.x on Unix.
IMO, the /usr/bin/python* and /usr/bin/env aliases are for Unix compatibility and so should prioritise that - but because Windows python doesn't include versioned executables, it's not always easy to do that.
but that will run 2.x in most Unix systems unless a 3.x environment is active. We can assume that such a script requires 3.x and is meant to run flexibly, in or out of an active environment.
In principle I agree, but I'm not sure how you see that working in practice. If you have an activated venv, `#!/usr/bin/env python3` should use the venv Python or the system Python depending on whether the venv is Python 3 or not. But the only way of finding that out in the absence of versioned executables is running `python -V`. That's a performance hit that I'd rather we avoided.
[see below about pyvenv.cfg - tl;dr is that there's no *documented* version info in there]
I'd prefer a consistent implementation of the "env" command that doesn't special case versioned "pythonX[.Y]" commands compared to plain "python". But another option that will at least make virtual-environment users happy would be for "env" to check for an active VIRTUAL_ENV and read its Python version from "pyvenv.cfg".
I'd agree with that, if we had versioned executables. Without them, I honestly don't know what answer would cause the fewest issues, so I tend to assume that sticking with what we have is good enough, because we're not getting lots of complaints (we get some, but not that many).
Following on from my comment above about needing to run `python -V`, I'd completely forgotten that pyvenv.cfg held the version statically nowadays.
But virtualenv 20.0 and later writes a pyvenv.cfg with different information, and virtualenv < 20.0 doesn't write pyvenv.cfg at all. I'm OK with discounting old versions of virtualenv, but realistically, I believe that not interoperating with current versions of virtualenv is impractical. I think that if we want to rely on pyvenv.cfg, we should document/standardise its contents. At the moment, the docs only mention `home` and `include-system-site-packages`, and to be honest they sound more informational than normative anyway.
3.9 adds "prompt" to record any custom prompt that was specified.
Unfortunately, there doesn't seem to be any real interest in standardising virtual environments at the moment.
I have interest. :)
So for now, at least, it's difficult to see how the launcher can behave as you want (I assume it's obvious that we don't want to hard-code implementation details of virtualenv into the launcher).
Up to this point the only thing the Launcher has is support for when the VIRTUAL_ENV environment variable is defined (my hope to standardize on environment naming seems to have stalled out).
I'd support a standardisation effort, but I don't intend to champion such an effort myself.
I don't either ATM, but maybe someday.
Just to reiterate, I'd like a better, more uniform solution here. But I think there's more complexity than people are assuming, at least if we want to interoperate with 3rd party tools (which I view as necessary, although I guess others may take a more hard-line stance).
participants (5)
-
Andrew Barnert -
Barry Scott -
Brett Cannon -
Eryk Sun -
Paul Moore