On 2008.04.18 15:06:10 +1000, Jonathan Lange wrote:

CompatibilityPolicy suggests release-based deprecation that is graded over 4 releases: release N PendingDeprecationWarning, N+1 DeprecationWarning, N+2 DeprecationError, N+3 remove the functionality.

Ticket #1216 suggests a release-based deprecation policy that is not graded and over 1 release: release N, add DeprecationWarning, release N+1, remove the functionality.

As a user, I don't see much difference between a PendingDeprecationWarning and a DeprecationWarning. Or between a DeprecationError and the code being removed. Simple is good.

Now, here's what I think.

* Release-based, not time-based.

Now that version numbers are based on the year, there's not a big difference. For example, if you wanted to guarantee at least a year of DeprecationWarning before removing a feature, you could implement that by saying that if the warning goes in in 8.x, then the feature can come out in 10.0. Or vice-versa. -- David Ripton dripton@ripton.net

On 18 Apr, 05:06 am, jml@mumak.net wrote:

Hello everyone,

Now that the release is out[1], I'd like to start work on fixing #1216 -- official deprecation policy.

Sorry for the delay. Due to a miscommunication, I thought that Chris would be the one writing up our meeting notes and announcing this. After I realized I was the one who was supposed to be doing it, I wanted to write up something official-sounding on the wiki first, but of course, there's never enough time. So here's a rough cut at the policy that we (the current "TSF board", such as it is: chris, me, JP, and itamar) agreed on at the sprint in Cambridge. Thanks, Jonathan, for motivating this; we've needed a clear statement for a long time. Someone (and this will probably be me) needs to make a complete and unambiguous index of official Twisted policy and put it together somewhere, probably on the wiki, at least at first.

I think these are all the questions we need to answer:

The proposed policy does break down as a list of answers to these questions, so one point at a time:

- Is it time-based or release-based?

Both. There will be a minimum amount of time and a minimum number of releases before a deprecated feature may be removed.

- How long for?

Every deprecation must last for a minimum of 2 releases and a minimum of one year. Of course, anyone who wants to keep maintaining deprecated APIs for longer than that may do so. I know this is somewhat longer than you've suggested, but at the meeting we ran down a laundry list of real-world usages of Twisted and pretty much nobody upgrades Twisted more than once per year. In fact, once per 2 years seems to be the average, and that's a fairly ambitious estimate. The objective of providing a year-long deprecation period is to allow a user who is performing their bi-yearly Twisted upgrade to do so with no more than 2 intermediary upgrades.

- Do we grade deprecations? How?

There will be three phases of deprecation. When a deprecation is first introduced (in trunk) the "deprecation counter" will be at 3. When that deprecation sees its first release, it decrements to 2. Deprecations at levels 3 and 2 will be PendingDeprecationWarning. When that same deprecation sees its second release after a 6-month period, the counter decrements to 1, and finally, after a second 6-month period, it decrements to 0 and can be removed. Deprecations with a counter at 1 and at 0 (if they still exist) should emit a DeprecationWarning. The counter may not move down past 2 until *all instances of the emitted warning* have been removed from the Twisted test suite, with the exception of tests specifically for the deprecated functionality itself (i.e. tests that call assertWarns). The reason that this is described as a counter rather than primarily in terms of (Pending)DeprecationWarning is that this was a subject of intense debate during the meeting. There is definitely a consensus that the Python warnings system is not a sufficient tool we have to deal with the kind of notifications that we want to provide. There is also a consensus that it is the only tool available. So, let me provide some more detail on the desired behavior here. The idea behind using PendingDeprecationWarning is that *users* of a piece of software which depends on Twisted should, in principle, be able to upgrade Twisted by one version without seeing any error output. However, *maintainers* of that software should immediately see all the warnings when they test with a new release. Trial should change to show all PendingDeprecationWarnings by default. However, this might be impractical due to other uses of PendingDeprecationWarning - if that is the case, Twisted should include its own warning class. It may be helpful to do that in order to include the additional metadata required for an automated deprecation workflow, however. The desired deprecation workflow is: * Twisted developer deprecates functionality, by adding a warnings.warn(TwistedDeprecationWarning(some useful metadata)) to the code. * The release process automatically runs "update-deprecations" over the codebase, updating the deprecation counters in the developer-supplied metadata appropriately. The release goes out. (Ideally this would be done without actually having to modify the whole codebase, updating a central deprecation / release index.) * Another release happens that sets some counters to zero. * A developer drops support for some deprecated functionality, confident in their ability to do so by observing the zero (or negative) value for the counter. An obvious remaining piece of work to implement this final policy is to define some temporary manual things that we can do until such a tool is implemented, and implement the tool itself as soon as possible. I don't believe any tickets have yet been filed for this, and *that* I am leaving in our venerable release manager's court, since this is all a part of the release process except for the fairly minor change to trial. However, in the absence of any tools, this policy does apply now, it will just be a bit of a pain to work out where the counter stands on any particular deprecation until we've got some more metadata to help us.

- Which code is subject to this policy?

All Python code released in Twisted, with the exceptions of private modules and test modules. Only code which has been present in a release needs to be deprecated. Anything which was added after the last release and has not yet been present in any other release may be modified at will. (We try to keep trunk stable, but this only applies to code that was relying on features in previous releases.) I think that CompatibilityPolicy's explanation of an "incompatible change" explains this. Users: do not import anything from a package named "test", or any name (module, class, or function) which begins with "_"; bears will eat you. This code may change at any time. (Not a part of the policy: I personally want to add some caveats here, where users of Twisted can ask for a grace period on private APIs that people actually needed to use, but I think for now let's err on the side of allowing the deprecation - I don't want to encourage people to use private APIs. Users: if you find that there are things you can't do with the public APIs and you really need a grace period to upgrade your applications, please speak up. Eventually, I'd like to have a very aggressive "you can't even import (or invoke) this if it's private" regime where users can't accidentally use private functionality without first seeing a runtime error of some kind that they have to explicitly silence, as with my "pyexport" hack. Once we've done *that* we should definitely have a zero-tolerance policy on supporting private stuff, though...) There's obviously some work to do here to make this policy easy and convenient to follow. This needs to be edited, CompatibilityPolicy updated, and prominently linked. We need someone to write up at least two different summaries: one for maintainers of projects using Twisted ("What kind of compatibility should I expect from Twisted releases?") one for Twisted contributors ("How do I add a deprecation? When can I remove it?") and maybe one for users and one for packagers, too. We need some tools to help us update deprecations. We need trial to change so that developers are notified earlier than their users. Do I hear any volunteers...? :)