[Python-Dev] Re: Bug fix releases

[Guido van Rossum]

Aahz writes:
> [posted to c.l.py with cc to python-dev]
> 
> [I apologize for the delay in posting this, but it's taken me some time
> to get my thoughts straight.  I hope that by posting this right before
> IPC9 there'll be a chance to get some good discussion in person.]

Excellent.  Even in time for me to mention this in my keynote! :-)

> In article <mailman.982897324.9109.python-list@python.org>,
> Guido van Rossum  <guido@digicool.com> wrote:
> >
> >We have clearly underestimated how much code the nested scopes would
> >break, but more importantly we have underestimated how much value our
> >community places on stability.  
> 
> I think so, yes, on that latter clause.  I think perhaps it wasn't clear
> at the time, but I believe that much of the yelling over "print >>" was
> less over the specific design but because it came so close to the
> release of 2.0 that there wasn't *time* to sit down and talk things
> over rationally.

In my eyes the issues are somewhat different: "print >>" couldn't
possibly break existing code; nested scopes clearly do, and that's why
we decided to use the __future__ statement.

But I understand that you're saying that the community has grown so
conservative that it can't stand new features even if they *are* fully
backwards compatible.

I wonder, does that extend to new library modules?  Is there also
resistance against the growth there?  I don't think so -- if anything,
people are clamoring for more stuff to become standard (while at the
same time I feel some pressure to cut dead wood, like the old SGI
multimedia modules).

So that relegates us at PythonLabs to a number of things: coding new
modules (boring), or trying to improve performance of the virtual
machine (equally boring, and difficult to boot), or fixing bugs (did I
mention boring? :-).

So what can we do for fun?  (Besides redesigning Zope, which is lots
of fun, but runs into the same issues.)

> As I see it, there's a natural tension between between adding features
> and delivering bug fixes.  Particularly because of Microsoft, I think
> that upgrading to a feature release to get bug fixes has become anathema
> to a lot of people, and I think that seeing features added or changed
> close to a release reminds people too much of the Microsoft upgrade
> treadmill.

Actually, I though that the Microsoft way these days was to smuggle
entire new subsystems into bugfix releases.  What else are "Service
Packs" for? :-)

> >So here's the deal: we'll make nested scopes an optional feature in
> >2.1, default off, selectable on a per-module basis using a mechanism
> >that's slightly hackish but is guaranteed to be safe.  (See below.)
> >
> >At the same time, we'll augment the compiler to detect all situations
> >that will break when nested scopes are introduced in the future, and
> >issue warnings for those situations.  The idea here is that warnings
> >don't break code, but encourage folks to fix their code so we can
> >introduce nested scopes in 2.2.  Given our current pace of releases
> >that should be about 6 months warning.
> 
> As some other people have pointed out, six months is actually a rather
> short cycle when it comes to delivering enterprise applications across
> hundreds or thousands of machines.  Notice how many people have said
> they haven't upgraded from 1.5.2 yet!  Contrast that with the quickness
> of the 1.5.1 to 1.5.2 upgrade.

Clearly, we're taking this into account.  If we believed you all
upgraded the day we announced a new release, we'd be even more
conservative with adding new features (at least features introducing
incompatibilities).

> I believe that "from __future__" is a good idea, but it is at best a
> bandage over the feature/bug fix tension.  I think that the real issue
> is that in the world of core Python development, release N is always a
> future release, never the current release; as soon as release N goes out
> the door into production, it immediately becomes release N-1 and forever
> dead to development
> 
> Rather than change that mindset directly, I propose that we move to a
> forked model of development.  During the development cycle for any given
> release, release (N-1).1 is also a live target -- but strictly for bug
> fixes.  I suggest that shortly after the release for Na1, there should
> also be a release for (N-1).1b1; shortly after the release of Nb1, there
> would be (N-1).1b2.  And (N-1).1 would be released shortly after N.

Your math at first confused the hell out of me, but I see what you
mean.  You want us to spend time on 2.0.1 which should be a bugfix
release for 2.0, while at the same time working on 2.1 which is a new
feature release.

Guess what -- I am secretly (together with the PSU) planning a 2.0.1
release.  I'm waiting however for obtaining the ownership rights to
the 2.0 release, so we can fix the GPL incompatibility issue in the
license at the same time.  (See the 1.6.1 release.)  I promise that
2.0.1, unlike 1.6.1, will contain more than a token set of real
bugfixes.  Hey, we already have a branch in the CVS tree for 2.0.1
development!  (Tagged "release20-maint".)

We could use some checkins on that branch though.

> This means that each feature-based release gets one-and-only-one pure
> bugfix release.  I think this will do much to promote the idea of Python
> as a stable platform for application development.

Anything we can do to please those republicans! :-)

> There are a number of ways I can see this working, including setting up
> a separate project at SourceForge (e.g. pythonpatch.sourceforge.net).
> But I don't think this will work at all unless the PythonLabs team is at
> least willing to "bless" the bugfix release.  Uncle Timmy has been known
> to make snarky comments about forever maintaining 1.5.2; I think this is
> a usable compromise that will take relatively little effort to keep
> going once it's set up.

With the CVS branch it's *trivial* to keep it going.  We should have
learned from the Tcl folks, they've had 8.NpM releases for a while.

> I think one key advantage of this approach is that a lot more people
> will be willing to try out a beta of a strict bugfix release, so the
> release N bugfixes will get more testing than they otherwise would.

Wait a minute!  Now you're making it too complicated.  Betas of bugfix
releases?  That seems to defeat the purpose.  What kind of
beta-testing does a pure bugfix release need?  Presumably each
individual bugfix applied has already been tested before it is checked
in!  Or are you thinking of adding small new features to a "bugfix"
release?  That ought to be a no-no according to your own philosophy!

> If there's interest in this idea, I'll write it up as a formal PEP.

Please do.

> It's too late for my proposed model to work during the 2.1 release
> cycle, but I think it would be an awfully nice gesture to the community
> to take a month off after 2.1 to create 2.0.1, before going on to 2.2.

It's not too late, as I mentioned.  We'll also do this for 2.1.

> BTW, you should probably blame Fredrik for this idea.  ;-)  If he had
> skipped providing 1.5.2 and 2.0 versions of sre, I probably wouldn't
> have considered this a workable idea.  I was just thinking that it was
> too bad there wasn't a packaged version of 2.0 containing the new sre,
> and that snowballed into this.

So the new (2.1) sre code should be merged back into 2.0.1, right?
Fredrik, go ahead!  We'll start planning for the 2.0.1 release right
after we're back from the conference.

BTW, See you at the conference!

--Guido van Rossum (home page: http://www.python.org/~guido/)