[Distutils] Shebang lines, /usr/bin/python, and PEP394

[Nick Coghlan]

On 26 July 2013 21:31, Alex Burke <alexjeffburke at gmail.com> wrote:
> On 26 July 2013 00:37, Philip Jenvey <pjenvey at underboss.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA512
>>
>>
>> On Jul 25, 2013, at 9:04 AM, Toshio Kuratomi wrote:
>>
>>> Over on python-dev we're talking about Linux Distributions switching from
>>> python2 to python3, what steps they need to take and in what order.  One of
>>> the things that's come up [1]_ is that a very early step in the process is making
>>> sure that shebang lines use /usr/bin/python2 or /usr/bin/python3 as noted in
>>> PEP394 [2]_.  Faced with the prospect of patching a whole bunch of scripts
>>> in the distribution, I'm wondering what distutils, distlib, setuptools, etc
>>> do with shebang lines.
>>> * Do they rewrite shebang lines?
>>
>> distutils, distlib and setuptools all do.
>
> Hi,
>
> It was interesting that discussion came up on python-dev but I admit
> to being surprised by the suggestion the shebang lines may need to be
> rewritten in end user code.
>
> This may be a callous over-simplification but if #!python is rewritten
> by the python packaging infrastructure, would it not be changed for
> python2/python3 as appropriate at installation time? Thus a python 2
> package (whatever it is named) would be generated by calling a python2
> executable + setuptools while the same is true for v3 except using
> python3. The result is then packaged by rpm/dpkg.
>
> Keen to understand why it can't work this way if that's the case.

It actually occurs to me that the following example (showing how
symlinks affect "sys.executable") illustrates both the problem and the
solution for cases where users are relying on generated script
wrappers or the shebang line rewriting in wheel:

    $ ln -s /usr/bin/python foo
    $ ./foo -c "import sys; print sys.executable"
    /home/ncoghlan/foo

So long as the distro build systems are updated to invoke setup.py
through an appropriately versioned symlink (rather than through
/usr/bin/python), setuptools et al should all automatically do the
right thing when generating script wrappers.

Not everybody uses generated script wrappers, though - if there is a
hardcoded "/usr/bin/env python" or "/usr/bin/python" in a shebang
line, the Python build tools won't touch it. There's also a whole lot
of software that isn't packaged at all - it's sitting around in user
and admin home directories, or maybe in a shared directory on a file
server or even in a private source control repo.

Published software is actually the vanishingly small tip of a very
large iceberg, especially for languages like Python that handle
scripting use cases fairly well and are thus popular for personal
automation tasks amongst developers and system administrators.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia