Hi Nathan,

That is a good point about Bitbucket's bookmark support.  As I see it, a feature-named bookmark would only exist until the point at which the feature is merged.  We would immediately remove it after that.

This idea came mainly after thinking about the volume render refactor and what we might do if we had two such features being worked on concurrently.  My feeling was that the name "experimental" wasn't very descriptive and also allowed for only one such effort to exist.  As it stands, I would not consider my reloadable datasets PR to be needing of this type of action as it's really only me that's working on it.  I think what I proposed should only be used when we have a clear situation of something that multiple people need to work on together.  So far, I can only think of two instances where this should have been used: 1) the current vr refactor and 2) yt-3.0 and the units work.  This should really be just a last resort.

I would disagree with your last point about easing restrictions on pulling in experimental work.  If anything, I think our history has shown that simply making a commitment to eventually getting docs done is not a good idea.  What's more, by merging multiple new features into a single branch, we end up with a situation where a single undocumented feature can hold up a release because it is tangled up with everything else that's ready to go.  I don't mean to say people are deceptive or break promises, only that life just gets in the way of plans far too often.

I do think your concern of having too many heads in yt is very reasonable and we should use something like this only when absolutely necessary.

Britton

On Thu, Sep 10, 2015 at 3:01 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:

Just a first initial reaction: I don't think introducing even more heads to the yet branch is a good idea, even if they are marked by bookmarks. In fact I'd prefer to only have one head on the yt branch right now.

In addition, bitbucket's support for bookmark's in the pull request interface amounts only to labeling entries in the source and destination dropdown menu, meaning that once a pull request is merged, someone would need to manually update the corresponding bookmark to the correct head.  This is both error prone and easy to skip, so I predict that if we *did* start doing this, bookmarks would wuickly go out of sync with heads, leaving a bunch of un-bookmarked heads.

Was this prompted by your work on the reloadable datasets PR? Without something like this do you expect it to sit in limbo for a long time?

I think the solution we need is to jot be afraid to pull in more "experimental" work-in-progress stuff to the yt branch. Of course that would need to come with assurances from the author or community members that they're committed to "finishing" the code before the next major release. That way people will be able to test new code more easily (they'll get it without updating explicitly to it) and we'll catch breakages to unrelated code and wordflows more quickly.

Nathan

On Thursday, September 10, 2015, Britton Smith <brittonsmith@gmail.com> wrote:

Hi everyone,

I had some ideas for improving the yt development process that I

wanted to run by everyone.  This can be discussed further at our

upcoming team meeting and if people are in favor, I will issue a pull

request to the relevant YTEP.

STATEMENT OF PROBLEM

Currently, development proceeds roughly as follows.  The two main

active branches within the central yt repository are yt and stable.

The tip of stable is the latest release and the yt branch is the de

facto "development version" of the code.  Until recently, we have not

been very good at regularly scheduled minor releases and so the stable

branch sits for quite some time with many bugs that are fixed within

the development branch.  This effectively makes stable unusable and

pushes most users to the yt branch.

When new features are developed, pull requests are issued to the

single head of the yt branch.  Because this is the version most people

are actually using, the current policy is to not allow PR with new

functionality to be accepted until they are 100% ready (full

functionality, tests, docs, etc).  As we have already seen, this makes

collaborative development very cumbersome, as it requires people to

create forks of the fork from which the PR originates.  They then must

issue PRs to that fork after which time the original PR is updated.

The current volume render refactor is the perfect example of this.

PROPOSED SOLUTION

Before I lay out the proposed solution, I want to list a number of

recent developments that I think will make this possible:

1. Nathan's new script for backporting changes now keeps stable and yt

   synced on bugfixes.

2. We have returned to doing minor releases containing only bugfixes,

   thanks again to Nathan's hard work.  This and point 1 means that

   users are once again safe to be on stable, and now should be there

   most of the time.

3. Bitbucket now supports bookmarks, meaning that PRs can be issued to

   specific bookmarks instead of to branches or heads named only by the

   changeset hash.

4. The weekly PR triage hangouts are making it easier to process PRs

   and also providing a place to strategize getting larger PRs

   accepted.  Thanks to Hilary for keeping this going.

With the above in mind, I propose the following:

1. Create a "development" bookmark to sit at the tip of the yt

   branch.  All PRs containing relatively small new features are

   issued to this.  The requirements for acceptance remain the same:

   PRs accepted to "development" must contain all intended

   functionality and be fully documented.  This allows the

   "development" bookmark to be defined explicitly as everything that

   will be included in the next major release.

2. Large new features should have a corresponding YTEP that has been

   accepted.  After the YTEP has been accepted, a PR should be issued

   to the yt branch.  After some initial discussion, this PR is pulled

   into the main yt repo with a bookmark named after the feature.

   Once this has happened, developers can now issue new PRs

   specifically to this bookmark.  This is effectively what we have

   now with the volume render work in the "experimental" bookmark,

   only we would rename the bookmark to something like "vr-refactor".

   As with PRs issued directly to "development", only after the new

   feature is 100% ready shall it be merged into the "development"

   bookmark.

3. We continue to make use of the PR triage hangouts to establish when

   large features are ready to be merged.

I believe this will have the following benefits:

1. Large, new features can be developed collaboratively without the

   need for forks of forks of forks.

2. New, underdevelopment features are more accessible to the larger

   community by simply updating to named bookmarks from the main repo

   (no need for "just pull these changes from my fork").

3. The "development" branch is preserved as a place only for

   ready-to-be-released features (i.e., polished and documented).

All told, this is really just a small tweak on our current process.

Please comment with any thoughts, or even a +/-1 if your feelings can

be summed up thusly.

Britton

_______________________________________________
yt-dev mailing list
yt-dev@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________
yt-dev mailing list
yt-dev@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org