
2009/6/19 <glyph@divmod.com>:
On 02:17 am, benjamin@python.org wrote:
Backwards compatibility seems to be an issue that arises a lot here. I think we all have an idea of it is, but we need some hard place to point to. So here's my attempt:
PEP: 387 Title: Backwards Compatibility Policy
Thanks for getting started on this. This is indeed a bikeshed that there would be less discussion around if there were a clearer PEP to point to. However, this draft PEP points to the problem rather than the solution. In order to really be useful this needs to answer a whole slew of very very specific questions, because the real problem is that everybody thinks they know what they mean but in fact we all have slightly different definitions of these words.
I don't expect this document at all to be the definitive policy all at once. I expect it will evolve and be filled in as it is applied in real life.
I've taken a stab at this myself in the past while defining a policy for Twisted, which is here: <http://twistedmatrix.com/trac/wiki/CompatibilityPolicy>. I think we might be a bit stricter than Python, but I believe our attempt to outline these terms specifically is worth taking a look at.
Yes, that's helpful. Thanks.
The questions that follow might seem satirical or parodic but I am completely serious and each of these terms really does have a variable definition.
And will always no matter what we do. It's how natural language works.
* The behavior of an API *must* not change between any two consecutive releases.
What is "behavior"? Software is composed of behavior. If no behavior changes, then nothing can ever happen.
I mean that if you pass X and Y into a function and get Z in 2.6, then you should be able to get Z from passing X and Y in 2.7 even if there's a new argument that returns Z' if you pass True to it. (Obviously, this is somewhat simplified, but I hope it helps.)
* A feature cannot be removed without notice between any two consecutive releases.
Presumably removal of a feature is a change in behavior, so isn't this redundant with the previous point?
I wanted to qualify differently from explicit behavior change as I defined above in this message.
* Addition of a feature which breaks 3rd party libraries or applications should have a large benefit to breakage ratio, and/or the incompatibility should be trival to fix in broken code.
How do you propose to measure the benefit to breakage ratio? How do you propose to define "trivial" to fix? Most projects, including Python, Twisted, Django, and Zope, have an ever-increasing bug count, which means that trivial issues fall off the radar pretty easily.
Well, if you're tests aren't failing from it, is it an incompatibility?
One of the big problems here in detecting and fixing "trivial" changes is that they can occur in test code as well. So if the answer is "run your tests", that's not much help if you can't call the actual APIs whose behavior has changed. The standard library would need to start providing a lot more verified test stubs for things like I/O in order for this to be feasible.
Making Incompatible Changes ===========================
It's a fact: design mistakes happen. Thus it is important to be able to change APIs or remove misguided features. This is accomplished through a gradual process over several releases:
1. Discuss the change. Depending on the size of the incompatibility, this could be on the bug tracker, python-dev, python-list, or the appropriate SIG. A PEP or similar document may be written. Hopefully users of the affected API will pipe up to comment.
There are a very large number of users of Python, the large percentage of whom do not read python-dev. A posting on python-list is likely to provoke an unproductive pile-on. I suggest a dedicated list, which should ideally be low traffic, "python-incompatible-announce", or something like that, so that *everyone* who maintains Python software can subscribe to it, even if they're not otherwise interested in Python development.
And that won't generate a pile-on?
DeprecationWarning is the usual warning category to use, but PendingDeprecationWarning may be used in special cases were the old and new versions of the API will coexist for many releases.
Why is PendingDeprecationWarning the special case?
It should be used for things like string exceptions which are deprecated but are not removed for a while.
As outlined in the Twisted compatibility proposal, I'd prefer it if every warning went through multiple phases: pending, deprecated, gone: so that careful developers could release new versions of their software to quash noisy warnings *before* the new version of Python that actually made them user-visible was released.
3. Wait for a release.
How does this affect the parallel 2.x/3.x release cycle?
I just added something about this.
4. See if there's any feedback. Users not involved in the original discussions may comment now after seeing the warning. Perhaps reconsider.
At this point, many developers may already have been porting over to the new, non-deprecated API. I realize that there may be no way around that, but it's worth considering.
Thanks for tackling this thorny problem. Good luck!
I think I'll need it. Thanks. :) -- Regards, Benjamin