[python-committers] what's going on with Misc/NEWS?

[Brett Cannon]

On Sat, May 25, 2013 at 4:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, May 25, 2013 at 2:01 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>> On Fri, May 24, 2013 at 9:46 PM, Raymond Hettinger
>> <raymond.hettinger at gmail.com> wrote:
>>> and it recognizes that users don't really need to look across merge
>>> boundaries.
>>
>> This is tricky though for any patch that is forward-ported to a
>> release branch (a la 3.2->3.3).  How can you tell from MISC/News in
>> which release (e.g. 3.3.0 or 3.3.1) of that branch the bug was fixed?
>> The entry would only show up in MISC/News for the previous release
>> (e.g. 3.2.3).
>>
>> Granted, this would impact much fewer patches than our current pain point does.
>
> Let's take a step back for a moment and define the workflows we want to handle.
>
> 1. Feature development
>
> - simplest case
> - just edit Misc/NEWS on default
>
> 2. Normal 3.x only bug fix
>
> - second simplest case
> - exit Misc/NEWS on 3.3
> - merge to default
> - frequently conflict (due to entries for new features and different
> release headers)
> ....
>
> OK, I'm going to stop enumerating the cases there, because it already
> suggests to me that splitting the *whole* NEWS file entirely by
> version is the wrong thing to do. Instead, we really only need to
> split the handling for NEWS items that haven't been released yet.

Yes, that's a very good point.

>
> To avoid needing to copy info from other branches between files when
> doing a merge, we could instead set up the following:
>
> 1. Add a new directory called NEWS.next
> 2. New NEWS entries go in version specific files in that directory
> 3. When a new release is made, ALL entries in NEWS.next are added to
> the top of the main NEWS file for the relevant (a script to help with
> the merging would be a good idea) and the contents of NEWS.next are
> cleared.

Still with you.

>
> So, suppose we're about to release 3.4.0a1. In the meantime, we have
> accumulated bug fixes for 3.3 and new features and bug fixes that
> couldn't be backported for 3.4. (The scheme could be extended to
> security branches too, but I'm assuming for the moment those will be
> handled via selective backporting rather than the normal forward
> porting path)
>
> The 3.3 branch layout would look like this:
>
>     NEWS.next/
>         3.3.txt  # Categorised changes

Categorized how? E.g. Core,Lib,Docs, etc.? Or "3.3 only", "3.3 and 3.4"?

>     NEWS  # Existing partial entry for 3.3.x

And the accumulated history of all previous versions.

>
> The default branch layout would look like this:
>
>     NEWS.next/
>         3.3.txt  # Forward ported categorised changes
>         3.4.txt  # Categorised changes
>    NEWS  # Existing partial entry for 3.4.0a1
>
> Any changes that were specific to 3.3 would be listed in
> NEWS.next/3.3.txt on that branch, but not on the default branch (since
> the associated null-merge would have skipped adding them)
>
> Now, we go to create 3.4.0a1. This will involve transferring *all* the
> NEWS.next entries on default into the main NEWS file and clearing the
> files in NEWS.next.
>
>     NEWS.next/
>         3.3.txt  # Empty file
>         3.4.txt  # Empty file
>    NEWS  # Complete entry for 3.4.0
>
> The part I'm not clear on is whether we'd then start getting conflicts
> when forwarding merging changes to NEWS.next/3.3.txt from the 3.3
> branch, or if Mercurial would be smart enough to cope with the
> deletion for the file contents and not try to add them back in. If it
> can't cope, then it would be possible to do the following on 3.3 when
> creating the initial 3.4.0a1 release:
>
>     NEWS.next/
>         3.3.txt  # Empty file
>    NEWS  # Longer partial entry for 3.3.x

Right, and that's the tricky bit. You can have a 3.3 file which is
stuff that is not forward-ported (and thus just delete the file in
default and won't hg just ignore the diff for that file?) and you can
have a 3.4 file that only exists in default so you don't have to worry
about any merge issues.

But it's the 3.3+ changes that exist in both versions. That's the
sticking point and where we always have merge problems. We might want
to classify commits as 3.3 only, 3.3+3.4, or 3.4 only and have
separate files for each case.

Since these files are for staging NEWS entries essentially and are not
meant to be seen by the general public, we can be a little messy here.
So why don't we simply use markers in a 3.3+3.4 file that gets merged
across 3.3 and default that represents "from this line down,
everything has been merged into the main NEWS file under 3.4" and then
some other line that represents "from this line down everything has
been merged into the main NEWS file for 3.3". Then above those lines
we just start a new set of sections we continue to populate every time
there is some merger in some version for NEWS. A script can easily
figure out that some region is new to 3.4 or 3.3 based on reasonable
markers and just searching far enough back in the file to the last
merge for a specific feature release. And who cares if we have a
bazillion Library sections as long as the Library section at the top
is where we always put new entries that go into 3.3 and 3.4? Then
there is no merge conflict, no repeating ourselves and it's still easy
to add NEWS entries.

>
> We then continuing accumulating NEWS.next entries in parallel during
> the 3.4 alpha and beta cycle, moving the entries into the appropriate
> NEWS files as releases are made.
>
> The other advantage of this is that it's an approach we can adopt
> *today*, so long as we acknowledge we'll need the tooling to merge the
> NEWS entries and clear NEWS.next before we can next do a release for
> 3.3 and 3.4, and Georg and Larry are happy with that notion.

Yes, it's yet another step for release managers, but if we tool it
right and add it to release.py the burden should be low to
non-existent (biggest issue would be resolving the potential merge
conflict from 3.3 to default after a release when the main NEWS file
is updated).