[Linux-SIG] Revisit of PEP 394 -- The "python" Command on Unix-Like Systems

[Nick Coghlan]

On 25 July 2017 at 22:02, Charalampos Stratakis <cstratak at redhat.com> wrote:
> In Fedora, we'd like to change /usr/bin/python to point to Python 3 by 2020 (when PEP 404 kicks in). However, we'd rather not do it without upstream's
> support in PEP 394. Hence we would like to drive the change of PEP 394 to either say that /usr/bin/python should point to Python 3, or to say it is up
> to the distribution maintainers. (And we would prefer the first option, so that the situation may eventually become consistent.)

Some additional context to this idea for non-Fedora-using folks: while
Python 2 and hence "/usr/bin/python" have been missing from our
default installations since the release of Fedora 23 in 2015, it's
still pretty easy to inadvertently get them back:

- an unqualified "dnf install python-<whatever>" will usually do it
- any package with "/usr/bin/python" in a shebang line implicitly
gains a dependency on "python2"

At the distribution level, our plan is to prevent the latter by way of
Fedora's packaging policy and associated automated tests (i.e. all
Python scripts in packages built for Fedora itself will be required to
qualify the required Python version), while the former will be handled
through RPM macros and virtual provides (essentially, there'll be a
cutover release, probably Fedora 32 in early 2020, where "dnf install
python-foo" starts to give you the Python 3 version of things instead
of the Python 2 version).

At the PEP 394 level, what I'd personally like to see us do is to
split the "Recommendations" section into two separate sections:
"Python User Recommendations" and "Platform Publisher Recommendations"

"Python User Recommendations" would cover three main points:

1. Recommend the use of virtual environments and Python level
packaging tools to decouple Python development activities from distro
choices of default Python runtimes
2. Recommend writing hybrid Python 2/3 code when specifically
targeting "/usr/bin/python" (including favouring either
'print(fmt.(*args)) and 'print(fmt % args)' or 'from __future__ import
print_function' over multi-argument print statements)
3. Recommend targeting "/usr/bin/python2" instead of "/usr/bin/python"
when publishing specifically Python 2 code

"Platform Publisher Recommendations" would be updated to say:

1. "/usr/bin/python" should point to either Python 2.7 or Python 3.5+
(to ensure that bytes objects support mod-formatting)
2. If it's updated to point to Python 3.x, then system administrators
switching it back to Python 2.7 *should* be a supported operation
(i.e. it shouldn't break the OS itself if they do that)
3. When a Python 2.7 stack is provided, make sure to also provide
"virtualenv" or some other mechanism for user level environment
management
4. Regardless of whether "/usr/bin/python" points to Python 2 or
Python 3, that default stack should provide the "six" and "future"
cross-compatibility libraries to make life easier for folks writing
unpackaged hybrid 2/3 code that can't readily indicate any third party
library dependencies

As far as the specifics of actually managing a cutover in
"/usr/bin/python" itself go, we think that's mainly a question to be
resolved between distro developers and their particular user
community.

For Fedora for example, we think that dropping "/usr/bin/python" from
the repositories entirely would be too large of a compatibility break,
whereas we're comfortable with the idea that by 2020, it will be
reasonable to require that anything still running in "/usr/bin/python"
be hybrid Python 2/3 code, such that finally changing the default
won't cause too much immediate breakage (and for things that do still
break, the recommended fix will be code modernisation to get into the
hybrid subset prior to upgrading the OS).

There's unfortunately no completely non-disruptive way to manage this
shift (hence why it's taking so long), but we think this is a
reasonable approach that allows each distro to devise a migration plan
that makes sense for them and their userbase while still allow Python
end users to write readable cross-distro code that doesn't particular
care whether it's run under Python 2 or Python 3, and for open source
Python project maintainers to provide developer guidelines that are
entirely independent of particular distro's choices about default
Python runtimes.

Before taking this proposal to python-dev, I'd turn the general
concept into an actual PR with specific proposed wording changes, but
I figured it made sense for us to seek some initial feedback here
before doing that.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia