[Python-Dev] When to remove deprecated stuff

[R. David Murray]

On Thu, 22 Aug 2013 16:45:29 +0300, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> 
> On 22 Aug 2013, at 14:00, Petri Lehtinen <petri at digip.org> wrote:
> 
> > Terry Reedy wrote:
> >> On 8/15/2013 8:29 AM, R. David Murray wrote:
> >> 
> >>> A number of us (I don't know how many) have clearly been thinking about
> >>> "Python 4" as the time when we remove cruft.  This will not cause any
> >>> backward compatibility issues for anyone who has paid heed to the
> >>> deprecation warnings, but will for those who haven't.  The question
> >>> then becomes, is it better to "bundle" these removals into the
> >>> Python 4 release, or do them incrementally?
> >> 
> >> 4.0 will be at most 6 releases after the upcoming 3.4, which is 9 to
> >> 12 years, which is 7 to 10 years after any regular 2.7 maintainance
> >> ends.
> >> 
> >> The deprecated unittest synonyms are documented as being removed in
> >> 4.0 and that already defines 4.0 as a future cruft-removal release.
> >> However, I would not want it defined as the only cruft-removal
> >> release and used as a reason or excuse to suspend removals until
> >> then. I would personally prefer to do little* removals
> >> incrementally, as was done before the decision to put off 2.x
> >> removals to 3.0. So I would have 4.0 be an 'extra' or 'bigger' cruft
> >> removal release, but not the only one.
> >> 
> >> * Removing one or two pure synonyms or little used features from a
> >> module. The unittest synonym removal is not 'little' because there
> >> are 13 synonyms and at least some were well used.
> >> 
> >>> If we are going to do them incrementally we should make that decision
> >>> soonish, so that we don't end up having a whole bunch happen at once
> >>> and defeat the (theoretical) purpose of doing them incrementally.
> >>> 
> >>> (I say theoretical because what is the purpose?  To spread out the
> >>> breakage pain over multiple releases, so that every release breaks
> >>> something?)
> >> 
> >> Little removals will usually break something, but not most things.
> >> Yes, I think it better to upset a few people with each release than
> >> lots of people all at once. I think enabling deprecation notices in
> >> unittest is a great idea. Among other reasons, it should spread the
> >> effect of bigger removals scheduled farther in the future over the
> >> extended deprecation period.
> >> 
> >> Most deprecation notices should provide an alternative. (There might
> >> be an exception is for things that should not be done ;-). For
> >> module removals, the alternative should be a legacy package on PyPI.
> > 
> > Removing some cruft on each release can be very painful for users.
> > 
> > Django's deprecation policy works like this: They deprecate something
> > in version A.B. It still works normally in A.B+1, generates a
> > (silenced) DeprecationWarning in A.B+2, and is finally removed in
> > A.B+3. So if I haven't read through the full release notes of each
> > release nor enabled DeprecationWarnings, I end up having something
> > broken each time I upgrade Django.
> > 
> 
> So you're still using features deprecated three releases ago, you haven't checked for DeprecationWarnings and it's Django making your life difficult?
> 
> Why not check for the deprecation warnings?

Doing so makes very little difference.

This is my opinion (others obviously differ):

Putting in one big chunk of effort at a major release boundary is easier
to schedule than putting in a chunk of effort on *every* feature
release.  More importantly, having it happen only at the major release
boundary means there's only one hard deadline every ten-ish years, rather
than a hard deadline every 1.5 years.

Expecting things to break when you switch to the new feature release
makes one view feature releases with dread rather than excitement.

This applies whether or not one is testing with deprecation warnings on.
Yes, there's a little less pressure if you are making the fixes on
the deprecation release boundary, because you can always ship the
code anyway if it is winds up being too big of a bear, so you have more
scheduling flexibility.   But you still face the *psychological* hurdle of
"feature release upgrade...will need to fix the all the things they've
deprecated...let's put that off".  Especially since what we are talking
about here is the *big* cruft, and thus more likely to be a pain to fix.

So, the operant question is which do the majority of *users* prefer, some
required "fix my code" work at every feature release, or the ability to
schedule the "fix my code" work at their convenience, with a hard deadline
(for anything not already fixed) at the major release boundary?

Note that under this suggested scenario the amount of work people will
need to do for Python4 will be trivial compared to that for Python3, and
there won't be any controversy about single-code-base vs conversion,
because we'll still be maintaining backward compatibility and just
removing cruft.  So the user base will probably heave a sigh of relief
(those that were around for this transition, at least :) rather than a
groan.

On the other hand, it *does* make a Python4 transition still a big deal
("that package doesn't support Python4 yet, it still uses old cruft XXX".)
Ports sure will be easier than with Python3, though!  Also, even without
removing big cruft, there *will* be things that need to be fixed when
switching to a new feature release, so I'm really talking about relative
levels of pain and when the bigger pain occurs.

How does one judge what the optimal amount of change is?

It would be great if we could figure out how to figure out what the
users want.  We more or less have one user opinion so far, from Petri,
based on his experience as a Django user.  We developers are also users,
of course, but our opinions are colored by our needs as developers as
well, so we aren't reliable judges.

--David

PS: When thinking about this, remember that our effective policy for
(the second half of?) Python2 was to hold all the big cruft removal until
Python3.  Even some stuff that was originally scheduled to be removed
sooner got left in.  So our user base is currently used to things being
pretty stable from a deprecation/backward compatibility standpoint.