Clarification of PEP 394 for scripts that run under Python 2 and 3
Hi python-dev, We have a Python script that runs correctly under Python 2.6, 2.7 and 3.3+. It is executed on a *nix system using the "python" executable (ie not python2 or python3 specifically). This works just fine for systems that have Python 2 installed, or 2 and 3, or just 3 and symlink "python" to "python3" (eg Arch Linux). But it fails for systems that have only Python 3 and do not create a "python" symlink, ie only "python3" exists as an executable. We thought that PEP 394 would come to the rescue here but it seems to be unclear on this point. In particular it says: - 4th point of the abstract: "so python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3" - 6th point of the recommendation section: "One exception to this is scripts that are deliberately written to be source compatible with both Python 2.x and 3.x. Such scripts may continue to use python on their shebang line without affecting their portability" - 8th point in the migration notes: "If these conventions are adhered to, it will become the case that the python command is only executed in an interactive manner as a user convenience, or to run scripts that are source compatible with both Python 2 and Python 3." Well, that's pretty clear to me: one can expect the "python" executable to be available to run scripts that are compatible with versions 2.x and 3.x. The confusion comes because there are systems that install Python 3 without creating a "python" symlink (hence breaking the above). And Guido said that "'python' should always be the same as 'python2'" (see https://mail.python.org/pipermail/python-dev/2014-September/136389.html). Further, Nick Coghlan seemed to agree that "when there's only python3 installed, there should be no /usr/bin/python" (see https://mail.python.org/pipermail/python-dev/2014-September/136527.html). My questions are: 1. What is the true intent of PEP 394 when only Python 3 is installed? Is "python" available or not to run scripts compatible with 2.x and 3.x? 2. Is it possible to write a shebang line that supports all variations of Python installations on *nix machines? 3. If the answer to 2 is no, then what is the recommended way to support all Python installations with one standalone script? Thanks! Regards, Damien.
On Nov 13, 2015, at 10:32 PM, Damien George wrote:
1. What is the true intent of PEP 394 when only Python 3 is installed? Is "python" available or not to run scripts compatible with 2.x and 3.x?
2. Is it possible to write a shebang line that supports all variations of Python installations on *nix machines?
3. If the answer to 2 is no, then what is the recommended way to support all Python installations with one standalone script?
It's important to remember that PEP 394 is an informational PEP, still under active (if dormant) discussion. It is also just a recommendation, and can't force any downstream redistributors to do one thing or another. Still, the intent is to provide a set of guidelines for the majority of *nix distributors to (eventually) adopt. I'll also point you to the recently created linux-sig where this topic is being discussed. https://mail.python.org/pipermail/linux-sig/2015-October/000000.html As you've noticed, Arch took a particular approach, but speaking as part of the Debian/Ubuntu community, don't expect that ecosystem to go down the same path. It's highly unlikely /usr/bin/python will ever point to anything other than Python 2, even when only Python 3 is installed by default. That might change once Python 2.7 is actually EOL'd, and that won't happen for quite a long time. PEP 373 currently says there will be bug fix releases until 2020, and I'd expect security-only releases for some time after that. In general, the discussions on linux-sig and elsewhere is coalescing around a launcher-type approach, where you'd put something like /usr/bin/py in your shebang if your script really is bilingual. Then `py` can try to figure out what's available on your system, what your preference is, etc. and then execute your script using that version-specific binary. I would expect the launcher to eventually be provided by the upstream Python development community. AFAIK, there have been a few stabs at writing such a launcher, but nothing that's far enough to be usable. So for now, you don't have great options. Cheers, -Barry
On 14 November 2015 at 08:57, Barry Warsaw <barry@python.org> wrote:
On Nov 13, 2015, at 10:32 PM, Damien George wrote:
1. What is the true intent of PEP 394 when only Python 3 is installed? Is "python" available or not to run scripts compatible with 2.x and 3.x?
2. Is it possible to write a shebang line that supports all variations of Python installations on *nix machines?
3. If the answer to 2 is no, then what is the recommended way to support all Python installations with one standalone script?
It's important to remember that PEP 394 is an informational PEP, still under active (if dormant) discussion. It is also just a recommendation, and can't force any downstream redistributors to do one thing or another. Still, the intent is to provide a set of guidelines for the majority of *nix distributors to (eventually) adopt.
I'll also point you to the recently created linux-sig where this topic is being discussed.
https://mail.python.org/pipermail/linux-sig/2015-October/000000.html
As you've noticed, Arch took a particular approach, but speaking as part of the Debian/Ubuntu community, don't expect that ecosystem to go down the same path. It's highly unlikely /usr/bin/python will ever point to anything other than Python 2, even when only Python 3 is installed by default. That might change once Python 2.7 is actually EOL'd, and that won't happen for quite a long time. PEP 373 currently says there will be bug fix releases until 2020, and I'd expect security-only releases for some time after that.
This is the same situation for the Fedora/RHEL/CentOS ecosystem - for the time being, we think failure -> "yum install /usr/bin/python" is the cleanest user experience we can readily offer for Python 3 only systems. However, we don't think it's an acceptable long term answer, and something *else* needs to be done with /usr/bin/python than just making it a symlink to Python 3 (since the latter will lead to cryptic error messages for Python 2 only scripts).
In general, the discussions on linux-sig and elsewhere is coalescing around a launcher-type approach, where you'd put something like /usr/bin/py in your shebang if your script really is bilingual. Then `py` can try to figure out what's available on your system, what your preference is, etc. and then execute your script using that version-specific binary. I would expect the launcher to eventually be provided by the upstream Python development community.
A couple of other links folks may find worth reading for background: * LWN summary of the discussion at this year's language summit: https://lwn.net/Articles/640296/ * Geoffrey Thomas's launcher concept: https://ldpreload.com/blog/usr-bin-python-23 This thread prompted me to do an update pass over https://wiki.python.org/moin/Python3LinuxDistroPortingStatus, adding these background links to the Python symlink section, and updating the Fedora porting status link to refer to the recently published porting DB: http://portingdb-encukou.rhcloud.com/ Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Barry Warsaw -
Damien George -
Nick Coghlan