Backwards Compatibility of Python versions

[Kragen Sitaker]

martin at v.loewis.de (Martin v. Loewis) writes:
> Sheila King <usenet at thinkspot.net> writes:
> > I responded, that it is essentially FUD. But, Python did recently release
> > an upgrade that breaks old code (the whole division operator issue). Would
> > someone like to respond to these concerns? I told the sysadmin that I would
> > research this issue and report back.
> 
> Please have a look at
> 
> http://python.sourceforge.net/peps/pep-0005.html
> 
> It contains the official procedure for migrating to newer versions,
> without silently breaking old code. The major point here is that no
> breakage should happen silently: there will be advance warnings, so
> users should have enough time to both analyse the costs of upgrading,
> and finding out quickly what to change.

The timeframe given is "at least one year" between the release of the
first version that gives warnings and the first version that breaks
compatibility.  Given that Debian potato (the current stable release)
still includes only 1.5.2, someone else is talking about 1.5.1 on
their RaQ host, and the default Red Hat 7.1 Python is still 1.5.2, I
think this is much, much too short a timeframe.

Because of this drastically abbreviated backwards-compatibility
schedule, right now, the current released Python is not
backwards-compatible with the most-widely-used Python.

Ten years is a much more reasonable timeframe, and thirty years would
be preferable.  

Every backwards incompatibility means the wholesale destruction of
other people's work --- working, debugged code must be rewritten,
occasionally from scratch.  Remember, we're not talking about
previously legal code becoming illegal; we're talking about legal code
subtly changing in meaning so it gives incorrect results.

If this continues, it will put Python users in the same situation as
Microsoft Excel users --- they can quickly write useful code to solve
their problems, sure, but they won't be able to run it two years
later.

> I very much doubt that either PHP or Perl guarantee full backwards
> compatibility; for any new feature introduced, I'm pretty sure that I
> can find a piece of code that breaks with the feature
> present. Python's approach is similar to Java's in this respect:
> Deprecated features will be warned about automatically.

I think Perl has better backward compatibility than we're seeing with
Python, but it doesn't have full backwards compatibility.  Code that
broke during 5.x was already pretty questionable.

Some sample Perl features introduced during 5.x:

   If $x was a reference to a subroutine, you used to have to say
   &{$x}($y} to call it with the argument $y.  Now you can say $x->($y)
   instead.  Previously, this was illegal syntax.

   qr became a new quoting operator for regexes, similar to
   re.compile(r'') in Python.  Old code that used 'qr' as a subroutine
   name will break.

   There is a Perl-to-C compiler.  Previously, there wasn't.

   (?<=RE) is a zero-width positive lookbehind assertion in regular
   expressions.  Previously, it was a syntax error.

   Perl now treats literal carriage returns (\r) as whitespace in program
   text and ignores them inside string literals when followed by newlines.
   Previously, carriage returns in program text were syntax errors, except
   inside string literals, where they were included in the string.

   "Pseudo-hashes" were added --- you can use an array reference as a
   hash reference if its contents are specially structured.
   Previously, this was a runtime error.

   for (1..$large_number) now uses xrange() instead of range().  This
   is an optimization, not a semantic change, unless your code depends
   on exiting with an error when creating too large a range.

   Foo:: is now equivalent to "Foo", a quoted string.  This is because
   new Foo @args would call the function Foo if it was defined and
   then call new on the result.  I'm not sure what it was before,
   probably equivalent to "Foo::".

   substr() now takes a fourth argument which replaces the substring
   with a different string, returning the old string.  Previously, I
   think fourth and subsequent arguments were ignored.

   If $/ is a reference to an integer or a scalar holding an integer,
   <> now reads in that many bytes at a time instead of reading in
   lines.  Previously, I think it would specify something like
   "SCALAR(0x80d1f74)" as a line separator, with the likely effect
   that <> would read in entire files at a time.

You may notice that some of these, such as the $/ one, are incredibly
ugly, specifically to avoid breaking backward compatibility.

Compare these to the nested-scopes and division-operator problems in
Python, both of which will potentially break a lot of existing code,
and "x in somedict", which makes it impossible to have a class
instance faithfully emulate both dictionaries and lists.  I'm not
saying they're not the right thing!  Python should have done things
the new way from the beginning, but Guido's time machine unfortunately
appears to have taken him into an alternate future.

I know that Guido and the rest of python-dev care a lot about
backward-compatibility, and that maintaining a language lots of people
use is bloody hard work with little reward, and that you can't please
everyone.  And I don't expect Python to stop changing so fast just
because *I* want it to.  I'm just hoping to add my voice to the chorus
of "Slow down!"s.