Hello,
In Fedora, I found that PEP 394's strict recommendation that python
points to python2 is holding us back. From discussions on Zulip and
elsewhere it's clear that this recommendation is not changing any time
soon, but I would like to officially relax it in several cases.
The problems are:
- For developers that are not following the language's development, the
fact that
pythoninvokespython2sends a strong signal that 2 is somehow the preferred version, and it's OK to start new projects in it. - Users and sysadmins that do want to “live in the future” are
switching the symlink to
python3themselves. We would like to give them a supported, documented way to do so -- and make surer they're aware of the caveats. - The
pythoncommand is still not available out-of-the box on macOS, so it didn't completely live up to the expectation of being the cross-platform way to launch 2/3 source compatile scripts. pythonin the shebang line can mean either that a script is carefully written to be 2/3 compatible, or that the author/maintainer is lazy or unaware of the recommendations. While Fedora guidelines have long banned the unversioned command, we feel that the only way to enforce that guidelines is to provide environments where thepythoncommand does not work (unless explicitly installed).
To help solve these, I would like to relax recommendations on the Unix
python -> python2 symlink in these cases:
- Users and administrators can, by a deliberate action, change
pythonto invoke Python 3. (Activating a venv counts as such an action, but so would e.g. using alternates, installing a non-default overriding package, or replacing /usr/bin/python.) - In controlled environments where being explicit is valued more than
user experience (test environments, build systems, etc.), distributions
can omit the
pythoncommand even whenpython2is installed.
I have filed these changes as a pull request here:
https://github.com/python/peps/pull/630
The PR also spells out several other things, which I felt were hidden between the lines -- but correct me if you disagree with my reading.
Petr Viktorin encukou@gmail.com writes:
In Fedora, I found that PEP 394's strict
recommendation that python
points to python2 is holding us back.
I have read the message, but I don't see how you draw the link that PEP 394 is holding you back.
The problems are:
- For developers that are not following the language's development,
the fact that
pythoninvokespython2sends a strong signal that 2 is somehow the preferred version, and it's OK to start new projects in it.
I agree with the statement you make later in the message:
[…] we feel that the only way to enforce
that guidelines is to
provide environments where the python command does not work (unless
explicitly installed).
Yes. The ‘python’ command is confusing, for the reasons you say. There should be ‘python2’ and ‘python3’ commands for Python 2 and Python 3 respectively, and no ‘python’ command should be installed by the operating system.
The fact that ‘/usr/bin/python’ exists is an historical accident, and I agree with the proposal you state: the best way to correct the confusion is to bar the confusing command from being installed by packages.
- Users and sysadmins that do want to “live in the future” are
switching the symlink to
python3themselves. We would like to give them a supported, documented way to do so -- and make surer they're aware of the caveats.
The supported, documented way to add a command pointing to a different command already exists, and there is no need to make a Python-specific special case.
Users who want to make a ‘python’ alias can do so in their shell; this is supported and documented.
Users who want to add a new command file can add a suitable directory (e.g. ‘$HOME/bin’) to their ‘PATH’ variable, and put a symlink in there named ‘python’. This is supported and documented.
Sysadmins who want to create a system-wide command ‘python’ can put a symlink at ‘/usr/local/bin/python’. This is supported and documented.
I disagree with making some special-case extra way; that would be both cunfusing and superfluous.
- The
pythoncommand is still not available out-of-the box on macOS, so it didn't completely live up to the expectation of being the cross-platform way to launch 2/3 source compatile scripts.
That is one of the minor ways which macOS fails to conform to community-agreed conventions. We should not let that intransigence distort our discussion of best practices.
To help solve these, I would like to relax
recommendations on the Unix
python -> python2 symlink in these cases:
For the above reasons, I disagree that PEP 394 is limiting what you want to do on free-software operating systems.
For non-free operating systems, I don't think the already-discussed PEP 394 should be weakened if the operating system vendor fails to conform.
- Users and administrators can, by a deliberate action, change
pythonto invoke Python 3.
Yes. That is well-known and long-standardised on Unix operating systems, and is much more broadly understood than any Python-specific special case would be. So I don't see how anyone is being held back.
I trust that PEP 394 will not be weakened in its effect, and I wish you well with using the already-supported, already-documented, PEP-394 compatible means to add local customisations for a ‘python’ command.
-- \ “Pity the meek, for they shall inherit the earth.” —Donald | `\ Robert Perry Marquis | _o__) | Ben Finney
Ben Finney ben+python@benfinney.id.au writes:
Petr Viktorin encukou@gmail.com writes:
[…] we feel that the only way to
enforce that guidelines is to
provide environments where the python command does not work
(unless explicitly installed).
Yes. The ‘python’ command is confusing, for the reasons you say. There should be ‘python2’ and ‘python3’ commands for Python 2 and Python 3 respectively, and no ‘python’ command should be installed by the operating system.
The fact that ‘/usr/bin/python’ exists is an historical accident, and I agree with the proposal you state: the best way to correct the confusion is to bar the confusing command from being installed by packages.
Because the above is ambiguous, I'll clarify: I am not calling for, and PEP 394 does not call for, the banishment of the ‘python’ command.
What I'm saying is that muddying the rules further on what ‘python’ may or may not mean is worse than banishing the ‘python’ command entirely.
So, short of banishing ‘python’ entirely, I think PEP 394 is already a good clear way to address the issue. Existing, documented and supported means to locally modify a ‘python’ command already exist and should be sufficient.
I trust that PEP 394 will not be weakened in its effect, and I wish you well with using the already-supported, already-documented, PEP-394 compatible means to add local customisations for a ‘python’ command.
-- \ “Try to learn something about everything and everything about | `\ something.” —Thomas Henry Huxley | _o__) | Ben Finney
On 04/27/18 02:03, Ben Finney wrote:
Ben Finney ben+python@benfinney.id.au writes:
Petr Viktorin encukou@gmail.com writes:
[…] we feel that the only way to
enforce that guidelines is to
provide environments where the python command does not work
(unless explicitly installed).
Yes. The ‘python’ command is confusing, for the reasons you say. There should be ‘python2’ and ‘python3’ commands for Python 2 and Python 3 respectively, and no ‘python’ command should be installed by the operating system.
The fact that ‘/usr/bin/python’ exists is an historical accident, and I agree with the proposal you state: the best way to correct the confusion is to bar the confusing command from being installed by packages.
Because the above is ambiguous, I'll clarify: I am not calling for, and PEP 394 does not call for, the banishment of the ‘python’ command.
Well, Guido is calling for it :) It would break too many things, but after discussions on the PR, it's clear that we want a future where the "python" doesn't exist. But while it's available, it should point to Python 2.
What I'm saying is that muddying the rules further on what ‘python’ may or may not mean is worse than banishing the ‘python’ command entirely.
That's also consistent with the PR discussion. (But not that much with
the original PEP, which said python is expected to eventually mean
python3.)
So, short of banishing ‘python’ entirely, I think PEP 394 is already a good clear way to address the issue. Existing, documented and supported means to locally modify a ‘python’ command already exist and should be sufficient.
I trust that PEP 394 will not be weakened in its effect, and I wish you well with using the already-supported, already-documented, PEP-394 compatible means to add local customisations for a ‘python’ command.
Right. But some already-supported, already-documented mechanisms like
Debian Alternatives or alternate package repos, are not compatible with
PEP 394.
And as a PEP 394 compliant distro, we also won't be promoting the
/usr/local or $HOME/bin ways to change python (which makes me a bit
sad, because that documentation might have included a link to the
caveats listed in the PEP).
Hello, After discussion on the Pull Request, my update to PEP 394 changed scope somewhat. The new major changes are:
- The
pythoncommand may not exist at all in some cases (see the PEP for details) - The paragraph about the anticipated future where python points to
Python 3 is removed. (We'd rather see a future where
pythondoesn't exist and one always has to specifypython2orpython3.) - The PEP now explicitly says that in an active venv, python means that venv's interpreter. (Some view this as a design mistake, but one not worth reverting now.)
There are also other edits and clarifications.
Thanks for everyone involved, especially Guido for pitching in with the intended direction -- which was not clear from (or is genuinely different from) the 7-year-old PEP!
I'll keep the PR open for a day or so, in case someone still wants to comment.
On 04/26/18 19:21, Ben Finney wrote:
Petr Viktorin encukou@gmail.com writes:
In Fedora, I found that PEP 394's strict
recommendation that python
points to python2 is holding us back.
I have read the message, but I don't see how you draw the link that PEP 394 is holding you back.
The problems are:
- For developers that are not following the language's development,
the fact that
pythoninvokespython2sends a strong signal that 2 is somehow the preferred version, and it's OK to start new projects in it.
I agree with the statement you make later in the message:
[…] we feel that the only way to
enforce that guidelines is to
provide environments where the python command does not work (unless
explicitly installed).
Yes. The ‘python’ command is confusing, for the reasons you say. There should be ‘python2’ and ‘python3’ commands for Python 2 and Python 3 respectively, and no ‘python’ command should be installed by the operating system.
The fact that ‘/usr/bin/python’ exists is an historical accident, and I agree with the proposal you state: the best way to correct the confusion is to bar the confusing command from being installed by packages.
- Users and sysadmins that do want to “live in the future” are
switching the symlink to
python3themselves. We would like to give them a supported, documented way to do so -- and make surer they're aware of the caveats.
The supported, documented way to add a command pointing to a different command already exists, and there is no need to make a Python-specific special case.
Users who want to make a ‘python’ alias can do so in their shell; this is supported and documented.
Users who want to add a new command file can add a suitable directory (e.g. ‘$HOME/bin’) to their ‘PATH’ variable, and put a symlink in there named ‘python’. This is supported and documented.
Sysadmins who want to create a system-wide command ‘python’ can put a symlink at ‘/usr/local/bin/python’. This is supported and documented.
I disagree with making some special-case extra way; that would be both cunfusing and superfluous.
- The
pythoncommand is still not available out-of-the box on macOS, so it didn't completely live up to the expectation of being the cross-platform way to launch 2/3 source compatile scripts.
That is one of the minor ways which macOS fails to conform to community-agreed conventions. We should not let that intransigence distort our discussion of best practices.
To help solve these, I would like to relax
recommendations on the Unix
python -> python2 symlink in these cases:
For the above reasons, I disagree that PEP 394 is limiting what you want to do on free-software operating systems.
For non-free operating systems, I don't think the already-discussed PEP 394 should be weakened if the operating system vendor fails to conform.
- Users and administrators can, by a deliberate action, change
pythonto invoke Python 3.
Yes. That is well-known and long-standardised on Unix operating systems, and is much more broadly understood than any Python-specific special case would be. So I don't see how anyone is being held back.
I trust that PEP 394 will not be weakened in its effect, and I wish you well with using the already-supported, already-documented, PEP-394 compatible means to add local customisations for a ‘python’ command.
On 28 April 2018 at 01:37, Petr Viktorin encukou@gmail.com wrote:
Hello, After discussion on the [Pull Request], my update to PEP 394 changed scope somewhat. The new major changes are:
- The
pythoncommand may not exist at all in some cases (see the PEP for details) - The paragraph about the anticipated future where python points to Python
3 is removed. (We'd rather see a future where
pythondoesn't exist and one always has to specifypython2orpython3.)
That isn't currently my desired future, as I don't want to see a python3 to python4 naming transition at any point, I want a transition from python3 back to an unqualified name to accurately reflect the change in version management philosophy resulting from the extended Python 3 migration. (And then "python3" would linger solely as a backwards compatibility remnant, even if it referred to python 4+, or a switch to some kind of CalVer based versioning scheme)
However, it may be possible to solve two problems at once here, as one of the issues we have in writing cross-platform instructions for both Windows and nix systems is that "python" isn't guaranteed to be on the Windows path, but the "py" launcher is*. This means the most robust tooling invocations on Windows are of the form "py -m pip ...", since they don't care about how your system PATH is configured. Getting not only "python", but also scripts installed with "py -m pip install --user..." to be reliably on the PATH is an arcane problem with assorted complications stemming from the differences between per-machine and per-user installations of Windows applications, and the "py -m module ..." approach avoids that mess entirely (as long as the tools of interest correctly support invocation via "-m").
The downside of those instructions is that it means we have a platform split where, depending on how your system is setup, which version of Python you're using, and whether or not you're running in a virtual environment, you need to invoke Python from the command line as one of:
- python
- python3
- py
(more details at https://github.com/pypa/python-packaging-user-guide/issues/396 )
Thanks for everyone involved, especially Guido for pitching in with the
intended direction -- which was not clear from (or is genuinely different from) the 7-year-old PEP!
It's a change, and not one that's been explicitly discussed with the current authors of the PEP yet (although I do see that Barry has chimed in on the PR).
I think it's a reasonable way for us to go, if it's accompanied by an explicit decision that we want to drive cross-platform convergence around "py" as the convention for unqualified version independent access to a Python interpreter (regardless of whether or not a virtual environment is active or not)
The key missing piece for doing that would be to define how we'd want a
py launcher to work on *nix systems, and then provide that as part of
CPython 3.8+ (and potentially backport it to a 3.7x maintenance release).
Cheers, Nick.
P.S. Note that on Windows, py always refers specifically to the launcher
(it doesn't get shadowed in virtual environments), but the launcher itself
is virtual environment aware (see https://www.python.org/dev/peps/pep-0486/
for details).
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Um, the PEP has "Unix-Like Systems" in its heading, so discussing the Windows situation seems out of scope to me.
You're one of its authors, so if you really want to keep the paragraph about the anticipated unified future we can keep it (though preferably this should be discussed in the issue, https://github.com/python/peps/pull/630). But I think this PEP is strongest in its guidelines for what distros and sysadmins should do today; I feel that that paragraph encourages hopes about a future that's farther away than most people care to plan, and not at all certain.
On Fri, Apr 27, 2018 at 6:23 PM, Nick Coghlan ncoghlan@gmail.com wrote:
On 28 April 2018 at 01:37, Petr Viktorin encukou@gmail.com wrote:
Hello, After discussion on the [Pull Request], my update to PEP 394 changed scope somewhat. The new major changes are:
- The
pythoncommand may not exist at all in some cases (see the PEP for details) - The paragraph about the anticipated future where python points to
Python 3 is removed. (We'd rather see a future where
pythondoesn't exist and one always has to specifypython2orpython3.)
That isn't currently my desired future, as I don't want to see a python3 to python4 naming transition at any point, I want a transition from python3 back to an unqualified name to accurately reflect the change in version management philosophy resulting from the extended Python 3 migration. (And then "python3" would linger solely as a backwards compatibility remnant, even if it referred to python 4+, or a switch to some kind of CalVer based versioning scheme)
However, it may be possible to solve two problems at once here, as one of the issues we have in writing cross-platform instructions for both Windows and nix systems is that "python" isn't guaranteed to be on the Windows path, but the "py" launcher is*. This means the most robust tooling invocations on Windows are of the form "py -m pip ...", since they don't care about how your system PATH is configured. Getting not only "python", but also scripts installed with "py -m pip install --user..." to be reliably on the PATH is an arcane problem with assorted complications stemming from the differences between per-machine and per-user installations of Windows applications, and the "py -m module ..." approach avoids that mess entirely (as long as the tools of interest correctly support invocation via "-m").
The downside of those instructions is that it means we have a platform split where, depending on how your system is setup, which version of Python you're using, and whether or not you're running in a virtual environment, you need to invoke Python from the command line as one of:
- python
- python3
- py
(more details at https://github.com/pypa/python-packaging-user-guide/ issues/396 )
Thanks for everyone involved, especially Guido for pitching in with the
intended direction -- which was not clear from (or is genuinely different from) the 7-year-old PEP!
It's a change, and not one that's been explicitly discussed with the current authors of the PEP yet (although I do see that Barry has chimed in on the PR).
I think it's a reasonable way for us to go, if it's accompanied by an explicit decision that we want to drive cross-platform convergence around "py" as the convention for unqualified version independent access to a Python interpreter (regardless of whether or not a virtual environment is active or not)
The key missing piece for doing that would be to define how we'd want a
py launcher to work on *nix systems, and then provide that as part of
CPython 3.8+ (and potentially backport it to a 3.7x maintenance release).
Cheers, Nick.
P.S. Note that on Windows, py always refers specifically to the launcher
(it doesn't get shadowed in virtual environments), but the launcher itself
is virtual environment aware (see https://www.python.org/dev/
peps/pep-0486/ for details).
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On 28 April 2018 at 12:34, Guido van Rossum guido@python.org wrote:
Um, the PEP has "Unix-Like Systems" in its heading, so discussing the Windows situation seems out of scope to me.
Sorry, I conflated two issues there - while PEP 394 itself is specific to Unix-like systems, my thoughts on where I'd like to take it in the future are mainly informed by my experiences helping to maintain the Python Packaging User Guide these days, where the current platform dependence of "How does a user run Python, pip, and pip-installed Python tools from the command line?" is a frequent source of problems for folks just starting out. (We really don't want to be maintaining separate "Windows instructions" and "everywhere else instructions" indefinitely, but that's where we are currently)
You're one of its authors, so if you really want to keep the paragraph about the anticipated unified future we can keep it (though preferably this should be discussed in the issue, https://github.com/python/peps/pull/630). But I think this PEP is strongest in its guidelines for what distros and sysadmins should do today; I feel that that paragraph encourages hopes about a future that's farther away than most people care to plan, and not at all certain.
After reviewing the specifics of the proposed changes in the PR, I'm fine with dropping any reference to future plans for the unqualified "python" name for now (aside from clarifying how we expect it to work in virtual environments). That accurately reflects the status quo anyway - inside a virtual environment, 'python' has a clear expected meaning (the venv's Python), outside a virtual environment it's thoroughly ambiguous at best.
The real deadline for figuring out the preferred post-Python-3 spelling will be once we decide to bump the major version number again, and that's at least a few years away, even if we eventually decide not to do a Python 3.10.
Cheers, Nick.
P.S. Historically, we hadn't been able to get much real traction with the
"py-for-Unix-like-systems" idea, since it wasn't clear what problem, if
any, it would solve at the distro level. However, we may have more luck
if we're able to position it as a more forward-compatible replacement for
explicit references to 'python3' that doesn't conflict with the assumption
that python will refer specifically to python2.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 28 April 2018 at 05:08, Nick Coghlan ncoghlan@gmail.com wrote:
On 28 April 2018 at 12:34, Guido van Rossum guido@python.org wrote: >
Um, the PEP has "Unix-Like Systems" in its heading, so discussing the Windows situation seems out of scope to me.
Sorry, I conflated two issues there - while PEP 394 itself is specific to Unix-like systems, my thoughts on where I'd like to take it in the future are mainly informed by my experiences helping to maintain the Python Packaging User Guide these days, where the current platform dependence of "How does a user run Python, pip, and pip-installed Python tools from the command line?" is a frequent source of problems for folks just starting out. (We really don't want to be maintaining separate "Windows instructions" and "everywhere else instructions" indefinitely, but that's where we are currently)
Strong +1 on this. I am not a Unix user, so my opinions on PEP 394 are not relevant, but I agree with Nick that the current mess over "how to write general instructions for a newcomer on how to run Python or Python-related commands" is unsustainable. We have:
- In a virtualenv - python
- On windows - py (or "python" might work, if you're lucky, and it may or may not work the same as "py")
- On Unix - python3 (or if you have a self-installed Python, or something like pyenv, "python" maybe, who knows).
- If you're using something like Anaconda - quite likely "python", but I'm not honestly sure except on Windows
And it's even worse once you get to something like pip: "pip", "pip3", "python -m pip", "py -m pip", "python3 -m pip", ...
I don't want to push any particular solution here myself (my almost exclusively Windows experience is too biased for me to understand the trade-offs) but IMO, it's definitely something we need to solve.
Paul
On Fri, Apr 27, 2018 at 7:23 PM, Nick Coghlan ncoghlan@gmail.com wrote:
The key missing piece for doing that would be to
define how we'd want a py
launcher to work on *nix systems, and then provide that as part of CPython
3.8+ (and potentially backport it to a 3.7x maintenance release).
I was thinking along the same lines just today, with an email forthcoming. Now I don't have to write it. :)
-eric
On 2018-04-28, 01:23 GMT, Nick Coghlan wrote:
That isn't currently my desired future, as I don't want to see a python3 to python4 naming transition at any point, I want a transition from python3 back to an unqualified name to accurately reflect the change in version management philosophy resulting from the extended Python 3 migration. (And then "python3" would linger solely as a backwards compatibility remnant, even if it referred to python 4+, or a switch to some kind of CalVer based versioning scheme)
+1
Matěj
https://matej.ceplovi.cz/blog/, Jabber: mcepl@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8
We can tell our level of faith in what God wants to do for us by our level of enthusiasm for what we want God to do for other. -- Dave Schmelzer
2018-04-27 17:37 GMT+02:00 Petr Viktorin encukou@gmail.com:
(...)
- The paragraph about the anticipated future where python points to Python 3 is removed.
Instead of editing old PEPs, would it make sense to write a new one which replaces the old one?
The PEP 394 has been written in 2011 and accepted in 2012. Editing an accepted PEP makes it harder to dig into the history of PEPs.
I know that the PEP 8 is updated regularly, so it's not a requirement, just a proposal for the special PEP 394.
Victor
On 30 April 2018 at 18:11, Victor Stinner vstinner@redhat.com wrote:
2018-04-27 17:37 GMT+02:00 Petr Viktorin encukou@gmail.com:
(...)
- The paragraph about the anticipated future where python points to Python 3 is removed.
Instead of editing old PEPs, would it make sense to write a new one which replaces the old one?
The PEP 394 has been written in 2011 and accepted in 2012. Editing an accepted PEP makes it harder to dig into the history of PEPs.
I know that the PEP 8 is updated regularly, so it's not a requirement, just a proposal for the special PEP 394.
We edit PEP 394 in place for the same reason we edit PEP 8 in place: so people have a consistent place to get our current recommendations.
It isn't like a packaging interoperability spec or the Python language & library specs, where folks regularly need to know what changed between particular revisions.
If folks really want to know how the recommendations have changed over time, then browsing https://github.com/python/peps/commits/master/pep-0394.txt isn't overly difficult.
(That said, I also wouldn't be opposed to adding an inline change log to the PEP, since it helps highlights the cases where the recommendations have changed, which can be helpful for folks that were already familiar with the older versions)
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Apr 30, 2018, at 06:36, Nick Coghlan ncoghlan@gmail.com wrote:
Instead of editing old PEPs, would it make sense to write a new one which replaces the old one?
As a general rule, at least the way I think about it, Informational PEPs can mostly be updated inline, evolving with new insights as we go along. As Nick points out, this gives folks a consistent place to read our current recommendations and guidelines. They’re informational so almost by definition contemporaneous.
Standards Track PEPs on the other hand shouldn’t be changed once finalized, except
around the margins, e.g. typos, updated URLs, that kind of thing. These PEPs should
capture the discussion and design at the time that the feature is solidified and lands;
they do not serve as latest up-to-date documentation on the feature. If the
feature changes or becomes obsolete in future versions of Python, a new PEP should be
written to supersede the old PEP. We even have an official Superseded Status
value, and a Superseded-By header.
-Barry
-
Barry Warsaw -
Ben Finney -
Eric Snow -
Guido van Rossum -
Matěj Cepl -
Nick Coghlan -
Paul Moore -
Petr Viktorin -
Victor Stinner