[Python-Dev] Python-3 transition in Arch Linux

[Allan McRae]

Hi,

While this is not strictly related to python development, I thought that 
developers of python might be interested in some of the lessons provided 
by this. So forgive me if this is really wrong for this list...

Recently Arch Linux did a big transition with respect to python.  Now we 
support two python packages: "python" and "python2".

The "python" package will always contain the latest 3.x release and 
currently has /usr/bin/python3.1 with symlinks to /usr/bin/python3 and 
/usr/bin/python.

The "python2" package contains the latest from the "legacy" python-2.x 
branch and has /usr/bin/python2.7 with a symlink to /usr/bin/python2.

I really do not want to debate the sanity of pointing /usr/bin/python at 
python-3.x here, but it suffices to say that I am of the opinion that if 
python-3.x is really the future of python, then /usr/bin/python must 
eventually point to a 3.x version.  Also, Arch Linux is very bleeding 
edge and we expect our users to be competent enough to deal with thing 
like this.  According to #python, we are all idiots....  And I have been 
(figuratively) yelled at by a couple of Debian developers (which is 
incidentally the only major distro I found without a /usr/bin/python2 
symlink).

Anyway, this transition was rather simple from a distribution point of 
view apart from the sheer number of packages involved.  All our 
supported packages were rebuilt to work with that symlink layout and any 
"porting" software to use that layout was relatively simple.  Most 
packages either detected the python2 symlink during the rebuild and just 
worked while others required some sort "export PYTHON=python2" or 
"--with-python=python2" or "python2 setup.py" or the like during the build.

Software packages tend to fall into three categories at roughly equal 
frequencies:
1) packages that detected or were pointed at python2 and everything worked
2) packages that detected or were pointed at python2 and partially worked
3) packages that needed adjustment to work with the python2 symlink.

The second case was particularly interesting.  These software would 
change some of their #! to point at the python2 symlink and leave the 
rest pointing at python.  Note that python-2.7 itself falls into this 
category as many files in /usr/lib/python2.7 still have "#!/usr/bin/env 
python" even when installed with "make altinstall".  I can not remember 
the exact details, but I recall that some of these files were installed 
with executable permissions which would be bad, but I need to look into 
this again now things have calmed down...

The packages that did not auto-detect and work with /usr/bin/python2 or 
/usr/bin/python2.7 mostly required a sed of their shebangs or a patch to 
any hardcoded /usr/bin/python paths so were easily fixed.

So that is something that python software developers could think about 
for the future.  When someone configures a module using a particular 
version of python, then ALL shebangs need changed to use that version. 
And it is generally bad practice to hardcode /usr/bin/python into any 
application as you are never quite sure which version you are getting. 
Instead allow it to be configured, keeping the current value as default.

Cheers,
Allan

-- 
Allan McRae
Arch Linux Developer