[Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those!

[Mike Meyer]

On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:

> Ralf W. Grosse-Kunstleve wrote:
> > Nick Coghlan wrote:
> > 
> >> I personally haven't seen anything to convince me that the 2.x -> 3.0 
> >> upgrade cycle is going to be significantly worse from a deployment point 
> >> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also 
> >> possible (e.g. code using 'with' or 'as' as identifiers runs just fine 
> >> on 2.4 or 2.5, but that code is going to break in 2.6).
> > 
> > The 2.x -> 3.0 transition will be *very* different. We're using Python
> > since 2000, version 1.5.2. I've always upgraded to the latest release
> > and got it working with a few fairly minor tweaks. Importantly, I
> > could do this without breaking backward compatibility if I wanted to.
> > At some point we had to give up on 1.5.2 compatibility, but that was
> > just because of Boost.Python requirements (only works with new-style
> > classes).
> > 
> > IIUC, there is no conservative upgrade path for Python 3.0. Once the
> > transition is done, the code will only work with 3.0. The bridge to
> > older Python versions is completely broken. Python 2.x will not even
> > parse most Python 3 scripts and vice versa. That's a situation we never
> > had before.
> 
> I still see this as a difference in degree rather than a difference in 
> kind. As soon as a developer puts "from __future__ import 
> absolute_imports" or "from __future__ import with_statement" in their 
> script every version of Python prior to 2.5 is going to refuse to even 
> compile it. If they provide an old script which uses an identifier which 
> is now a keyword, it is the more recent versions which are going to 
> object violently.

The one real difference is that you can generally go from 2.X to 2.Y>X
without mucking with your scripts so long as you update all the
modules you use as well. In an environment managed with package
systems, this is usually easy.

> Either way, the solution is the same: run that particular script with a 
> version of the interpreter that can handle it correctly. Make that 
> happen by whatever means your platform makes available (on Windows it 
> usually means batch files, on Unix'y systems a version specific shebang 
> line or a shell script).

Version-specific shebangs break the above property. It's possible to
have multiple versions installed and have everything work without
going to that. But your system vendor probably won't support it :-(.

> (Note again that all of these issues are only particularly difficult to 
> deal with in the case of applications which are written in Python and 
> intended to be used on a system someone else controls. When the same 
> entity controls both the application and the runtime environment - such 
> as web services, embedded systems, or applications that bundle their own 
> Python interpreter - it is merely necessary that the environment is kept 
> in sync as the needs of the code change)

Right. I live in the last world, because I've found that it's nearly
impossible to get "someone else" to provide adequate python builds.

> > As Python developers you have to realize that for many people Python is
> > "just" a core around which they build much bigger applications.
> 
> Strange as it may seem, at least some of us do exactly that as part of 
> our day jobs ;)

Yup.

> > If the
> > success of Python is to continue, you have to give your large user
> > base a clear path to working with two co-existing Python versions,
> > so that installing a Python 3 application doesn't break all Python 2
> > applications (some of which may never be converted since the original
> > developers have moved on).
> 
> You mean the way you can already have Python 1.5, 2.0, 2.1, 2.2, 2.3, 
> 2.4 and 2.5 all installed on the same machine, and invoke different 
> versions for different applications?
> 
> Yes, only one of them can be the 'default Python' for the machine,

Actually, that's not *quite* true.

One of the reasons you can install Python from source on most systems
- even using the default install - without breaking all the tools that
depend on the system's anointed version is because it install
/usr/local/bin/python. A system required version almost always lives
in /usr/bin, and is invoked with that path. Even systems that don't
have Python in the default install generally put packages somewhere
other than /usr/local (FreeBSD is the one exception I know of).

For one client, we had /usr/bin/python, a 64-bit 2.3 python;
/oap/bin/python, a 32-bit 2.4 python without ssl support (from the IT
support group, because they wanted the same language "everywhere"),
and /home/<application>/bin/python, a 64-bit 2.4 python with the ssl
support we required. The scripts all invoked the python they needed by
absolute path. The biggest problem we had was getting the developers
environment set up so they invoked the right python at the command
line for testing.

> language interpreter. To choose an example which affected me relatively 
> recently, how many shell scripts did the Ubuntu folks break when they 
> decided to change their default shell from bash to dash "because it is 
> faster" instead of explicitly invoking dash for the particular scripts 
> they wanted to speed up? That was one configuration change I reverted 
> pretty darn quickly, because there are a hell of a lot of shell scripts 
> out there which rely on bash extensions, even if the developers didn't 
> realise it when they wrote the scripts.

And those of use who use systems where bash *isn't* the default shell,
and never has been, wish all those developers would get a clue, and
write "#!/usr/bin/env bash" (or at least "#!/bin/bash") instead of
"#!/bin/sh", and reserve the latter for scripts that really are
*shell* scripts. Sorry, that's a sore spot...

> While I'm +1 on the idea of making altinstall the default for setup.py 
> and the Windows installer (forcing the administrator to explicitly 
> request that the new Python version be made the default Python 
> installation for the machine), I'm a pretty strong -1 on officially 
> blessing the idea of a python3 alias or a special py3 extension on 
> Windows (end users are obviously free to create those if they want, of 
> course).

I like the python3 alias (and python2 in 2.X), because it moves us
from a single default python to having a default python2 and python3,
which means scripts that care can ask for that without breaking the
ability to upgrade python2 or python3 without breaking those scripts,
while still allowing the default python to be either python2 or
python3.

> Who knows, with the rate at which virtual machine technology is 
> improving, this whole thing may become a non-issue by the time the 3.x 
> series is even ready for production use. One VM with a 2.x default 
> Python, another VM with a 3.x default Python, various applications 
> associated with each, and seamless integration into the underlying 
> platform so the user never even notices a VM is involved.

That just changes the environment from one where you have to find the
right python to one where you have to find the right VM. But you don't
have to go to that extreme; just keeping the different pythons
separated by directory, and making sure the scripts specify the right
directory as part of the shebang works (at lest for Unix).

	  <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.