[Distutils] Re: Fwd: Re: Use of "python" shebang an installation error?

On 23/7, 2020, at 06:51, David Mathog wrote:

On Wed, Jul 22, 2020 at 1:27 PM Paul Moore wrote:

...

On Wed, 22 Jul 2020 at 19:31, David Mathog wrote:

...

but that shebang has to be corrected when the installation is moved to a normal environment, which my code is doing now.)

Moving files that are installed by Python packaging tools isn't supported. It might work, and you can probably make it work with some effort, but it's very much a case of "don't do it unless you know what you're doing". Correcting shebang lines is definitely something you will need to do.

I understand that moving files is iffy. However, given that I want only 1 copy of each installed python package on the system and I need to be able to install different versions of the same package (to resolve module version number conflicts between packages), moving the files around and replacing most copies with links to the single copy seemed like the only way to go.

Here:

https://www.python.org/dev/peps/pep-0394/#recommendation

It says:

When packaging third party Python scripts, distributors are encouraged to change less specific shebangs to more specific ones. This ensures software is used with the latest version of Python available, and it can remove a dependency on Python 2. The details on what specifics to set are left to the distributors; though. Example specifics could include:

Changing python shebangs to python3 when Python 3.x is supported. Changing python shebangs to python2 when Python 3.x is not yet supported. Changing python3 shebangs to python3.8 if the software is built with Python 3.8.

and then immediately after it says:

When a virtual environment (created by the PEP 405 venv package or a similar tool such as virtualenv or conda) is active, the python command should refer to the virtual environment's interpreter and should always be available. The python3 or python2 command (according to the environment's interpreter version) should also be available.

Which seems to be exactly the opposite of the preceding stanza. Ie,

"always be as specific as possible"

then

"be general, and also provide specific"

The first paragraph is saying it is recommended to rewrite shebangs such as #!python3 to the actual Python interpreter the script is installed against, e.g. the interpreter in a virtual environment. The second paragraph is describing “which° command the installer should choose to refer to an interpreter. For CPython 3.8, for example, up to three commands may be available in a given virtual environment: {prefix}/bin/python {prefix}/bin/python3 {prefix}/bin/python3.8 and the installer should choose the most generic one, i.e. {prefix}/bin/python, because this avoids dealing with interpreter-specific naming conventions, e.g. the Python version (3 or 3.8), implementation (pypy or jython).

Personally I think the generic use of "python" both in shebangs and when invoking scripts as "python script" should be deprecated, with warnings from the installers to force developers to strip it out. It only works now by chance. Sure, there is a high probability it will work, but if one is on the wrong system it fails. If python4 (whenever it arrives) is not fully backwards compatible with python3 the generic use of "python" is going to cause untold grief. Whereas in that scenario all the code which uses "python3" should continue to function normally.

You are assuming “python3” or “python4” is a reliable command to refer to—which is an understandable misconception coming from a Linux background, but a misconception nonetheless. The wheel specification chose “python” for one reason: it is the only name that’s guaranteed to exist across operating systems and interpreter implementations. Also, by your logic (“python” would break when Python 4 comes out), wheels really should have break when a minor Python version is released, since a script written on Python 3.4 does not always work on 3.6 (as an example). So the installer really should use “python3.4” instead, does it not? But it does not, and nothing breaks right now—because wheels have other means to declare compatibility (wheel tags). An installer should be capable to put a script under the interpreter version only if the wheel tags allow it to. If the shebang needs to care about compatibility, something is already going very wrong. TP

Regards,

David Mathog -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/HAZUE...