Well, I was not trying to suggest they rested solely on your shoulders. On Mon, May 7, 2018, 7:35 PM John ZuHone wrote:

+1, but to be honest I think that index arrays are in trouble without some help (especially with unyt being a thing now)

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138 (m) 781-708-5004

john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone =====================================

On May 7, 2018, at 8:29 PM, Matthew Turk wrote:

+1 to a new branch and to allowing slightly lower barrier to entry for changes in 4.0.

I would also like to point out that in contrast to when 3.0 was under development, we now have a much, much more robust testing and review infrastructure in place. This is in no small part due to your work, Nathan, as well as Kacper, and I also want to highlight that people like Cameron and Bili are already using "4.0" in production and contributing changes and bug fixes. Plus, other changes (like index arrays) are being developed by people like John, and the number of people involved in this is much more comfortable than it was back then.

We are in a much better state than we were back then,and I think this will be a much easier development process and (eventual) transition. I am excited!

-Matt

On Mon, May 7, 2018, 6:52 PM Nathan Goldbaum wrote:

...

Hi all,

Currently there is active ongoing work in my fork of yt on the "sph-viz" branch. This branch contains the bitmap index work and accompanying changes to the particle selection API. Collectively these changes constitute a major backward incompatible change and thus cannot be easily integrated into the master branch on the main repo without major disruption. Doing that would also prevent us from being able to do a 3.5 or 3.6 release (currently there's no plan for a 3.6 release but crazier things have happened).

We've already come to some level of consensus both here and in YTEP discussions that we want to make some major backward incompatible changes to yt in yt-4.0 and that the demeshening work should be one of those changes.

To ease development going forward and make it clearer both how to get started and how to contribute, I'd like to push the current state of the "sph-viz" branch to the main yt repo and rename the branch "yt-4.0". Afterwards there will be "yt-2.x", "master", "stable", and "yt-4.0" branches in the main repo. Most pull requests will still go to master, but pull requests that implement changes or fixes that are specific to the demeshening or for planned features in yt-4.0 will go to the yt-4.0 branch.

I'd additionally like to suggest that the threshold for merging code to the yt-4.0 branch should be lower than in master, possibly by reducing the number of required reviewers to one. I'd prefer not to make it harder to merge code than that to keep development velocity high and to make sure we're not blocked on getting code merged in for lack of reviewers who are familiar with the changes in the yt-4.0 branch.

I don't think it's necessary to create a new repository, since the master branch will remain the default branch. People will only get the experimental yt-4.0 branch if they explicitly fetch it and check it out.

As for testing, I think we'd just need to set up something in Jenkins to check the yt-4.0 branch as well as master for new PRs. The cloud CI services should "just work". We may end up turning off answer testing if it's too hard to have two separate sets of test answers for master and yt-4.0.

Eventually, just like for yt 3.0 if you remember that, we will merge the yt 4.0 branch in. Doing that will require ensuring the documentation has been updated and developing community consensus that the new features are ready for widespread testing among people running on the yt master branch.

Looking forward to hearing what people think about this,

-Nathan Goldbaum _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

I'm +1 on this change, as there are a number of people (e.g., Iryna Butsky, Bili Dong, and myself at least) using sph-viz for production. By bringing it into the main yt repo, it will aid in its usage and contributions by other developers. Thanks for proposing this, Nathan! Cameron On Mon, May 7, 2018 at 6:01 PM, John ZuHone wrote:

I know you weren’t. I was just thinking out loud that we need a rethink of this, more hands, and some solid motivation for what we want to accomplish with it.

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138 (m) 781-708-5004

john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone =====================================

On May 7, 2018, at 8:40 PM, Matthew Turk wrote:

Well, I was not trying to suggest they rested solely on your shoulders.

On Mon, May 7, 2018, 7:35 PM John ZuHone wrote:

...

+1, but to be honest I think that index arrays are in trouble without some help (especially with unyt being a thing now)

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138 (m) 781-708-5004

john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone =====================================

On May 7, 2018, at 8:29 PM, Matthew Turk wrote:

+1 to a new branch and to allowing slightly lower barrier to entry for changes in 4.0.

I would also like to point out that in contrast to when 3.0 was under development, we now have a much, much more robust testing and review infrastructure in place. This is in no small part due to your work, Nathan, as well as Kacper, and I also want to highlight that people like Cameron and Bili are already using "4.0" in production and contributing changes and bug fixes. Plus, other changes (like index arrays) are being developed by people like John, and the number of people involved in this is much more comfortable than it was back then.

We are in a much better state than we were back then,and I think this will be a much easier development process and (eventual) transition. I am excited!

-Matt

On Mon, May 7, 2018, 6:52 PM Nathan Goldbaum wrote:

...

Hi all,

Currently there is active ongoing work in my fork of yt on the "sph-viz" branch. This branch contains the bitmap index work and accompanying changes to the particle selection API. Collectively these changes constitute a major backward incompatible change and thus cannot be easily integrated into the master branch on the main repo without major disruption. Doing that would also prevent us from being able to do a 3.5 or 3.6 release (currently there's no plan for a 3.6 release but crazier things have happened).

We've already come to some level of consensus both here and in YTEP discussions that we want to make some major backward incompatible changes to yt in yt-4.0 and that the demeshening work should be one of those changes.

To ease development going forward and make it clearer both how to get started and how to contribute, I'd like to push the current state of the "sph-viz" branch to the main yt repo and rename the branch "yt-4.0". Afterwards there will be "yt-2.x", "master", "stable", and "yt-4.0" branches in the main repo. Most pull requests will still go to master, but pull requests that implement changes or fixes that are specific to the demeshening or for planned features in yt-4.0 will go to the yt-4.0 branch.

I'd additionally like to suggest that the threshold for merging code to the yt-4.0 branch should be lower than in master, possibly by reducing the number of required reviewers to one. I'd prefer not to make it harder to merge code than that to keep development velocity high and to make sure we're not blocked on getting code merged in for lack of reviewers who are familiar with the changes in the yt-4.0 branch.

I don't think it's necessary to create a new repository, since the master branch will remain the default branch. People will only get the experimental yt-4.0 branch if they explicitly fetch it and check it out.

As for testing, I think we'd just need to set up something in Jenkins to check the yt-4.0 branch as well as master for new PRs. The cloud CI services should "just work". We may end up turning off answer testing if it's too hard to have two separate sets of test answers for master and yt-4.0.

Eventually, just like for yt 3.0 if you remember that, we will merge the yt 4.0 branch in. Doing that will require ensuring the documentation has been updated and developing community consensus that the new features are ready for widespread testing among people running on the yt master branch.

Looking forward to hearing what people think about this,

-Nathan Goldbaum _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

+1 for me too! I'm however slightly confused about the fact this may prevent a release of a yt 3.5 ? On 2018-05-08 14:14, Britton Smith wrote:

I'm very +1 on this. It makes the development more collaborative and easier to follow along with. I'm also fine with a lower bar for PR acceptance into the yt-4.0 branch while development continues.  I think it makes sense for Nathan to either be the person merging PRs in yt-4.0 or to be designating qualified reviewers as needs be.

The index arrays development is an interesting point of discussion. What's the best way to solicit help for this? Perhaps keeping open, labeled issues or a single issue with multiple milestones?

On Mon, May 7, 2018 at 9:21 PM, Cameron Hummels mailto:chummels@gmail.com> wrote:

I'm +1 on this change, as there are a number of people (e.g., Iryna Butsky, Bili Dong, and myself at least) using sph-viz for production.  By bringing it into the main yt repo, it will aid in its usage and contributions by other developers.  Thanks for proposing this, Nathan!

Cameron

On Mon, May 7, 2018 at 6:01 PM, John ZuHone mailto:jzuhone@gmail.com> wrote:

I know you weren’t. I was just thinking out loud that we need a rethink of this, more hands, and some solid motivation for what we want to accomplish with it.

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67(w) 617-496-1816 tel:617-496-1816 Cambridge, MA 02138(m) 781-708-5004 tel:781-708-5004

john.zuhone@cfa.harvard.edu mailto:john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone http://hea-www.cfa.harvard.edu/%7Ejzuhone =====================================

On May 7, 2018, at 8:40 PM, Matthew Turk mailto:matthewturk@gmail.com> wrote:

...

Well, I was not trying to suggest they rested solely on your shoulders.

On Mon, May 7, 2018, 7:35 PM John ZuHone mailto:jzuhone@gmail.com> wrote:

+1, but to be honest I think that index arrays are in trouble without some help (especially with unyt being a thing now)

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67(w) 617-496-1816 tel:617-496-1816 Cambridge, MA 02138(m) 781-708-5004 tel:781-708-5004

john.zuhone@cfa.harvard.edu mailto:john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone http://hea-www.cfa.harvard.edu/%7Ejzuhone =====================================

On May 7, 2018, at 8:29 PM, Matthew Turk mailto:matthewturk@gmail.com> wrote:

...

+1 to a new branch and to allowing slightly lower barrier to entry for changes in 4.0.

I would also like to point out that in contrast to when 3.0 was under development, we now have a much, much more robust testing and review infrastructure in place. This is in no small part due to your work, Nathan, as well as Kacper, and I also want to highlight that people like Cameron and Bili are already using "4.0" in production and contributing changes and bug fixes. Plus, other changes (like index arrays) are being developed by people like John, and the number of people involved in this is much more comfortable than it was back then.

We are in a much better state than we were back then,and I think this will be a much easier development process and (eventual) transition. I am excited!

-Matt

On Mon, May 7, 2018, 6:52 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Hi all,

Currently there is active ongoing work in my fork of yt on the "sph-viz" branch. This branch contains the bitmap index work and accompanying changes to the particle selection API. Collectively these changes constitute a major backward incompatible change and thus cannot be easily integrated into the master branch on the main repo without major disruption. Doing that would also prevent us from being able to do a 3.5 or 3.6 release (currently there's no plan for a 3.6 release but crazier things have happened).

We've already come to some level of consensus both here and in YTEP discussions that we want to make some major backward incompatible changes to yt in yt-4.0 and that the demeshening work should be one of those changes.

To ease development going forward and make it clearer both how to get started and how to contribute, I'd like to push the current state of the "sph-viz" branch to the main yt repo and rename the branch "yt-4.0". Afterwards there will be "yt-2.x", "master", "stable", and "yt-4.0" branches in the main repo. Most pull requests will still go to master, but pull requests that implement changes or fixes that are specific to the demeshening or for planned features in yt-4.0 will go to the yt-4.0 branch.

I'd additionally like to suggest that the threshold for merging code to the yt-4.0 branch should be lower than in master, possibly by reducing the number of required reviewers to one. I'd prefer not to make it harder to merge code than that to keep development velocity high and to make sure we're not blocked on getting code merged in for lack of reviewers who are familiar with the changes in the yt-4.0 branch.

I don't think it's necessary to create a new repository, since the master branch will remain the default branch. People will only get the experimental yt-4.0 branch if they explicitly fetch it and check it out.

As for testing, I think we'd just need to set up something in Jenkins to check the yt-4.0 branch as well as master for new PRs. The cloud CI services should "just work". We may end up turning off answer testing if it's too hard to have two separate sets of test answers for master and yt-4.0.

Eventually, just like for yt 3.0 if you remember that, we will merge the yt 4.0 branch in. Doing that will require ensuring the documentation has been updated and developing community consensus that the new features are ready for widespread testing among people running on the yt master branch.

Looking forward to hearing what people think about this,

-Nathan Goldbaum _______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org mailto:yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org mailto:yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France phone: +33.6.43.18.66.83

Since no one objected to this I'm planning to create the branch this afternoon. If anyone noticed weirdness related to adding the new branch to the repo please let me know so I can fix it. On Tue, May 8, 2018 at 8:13 AM, Nathan Goldbaum wrote:

On Tue, May 8, 2018 at 7:59 AM Corentin CADIOU wrote:

...

+1 for me too! I'm however slightly confused about the fact this may prevent a release of a yt 3.5 ?

Only if we didn’t create this new branch, because we’d be introducing backward incompatible changes to the master branch, forcing the next release to be yt 4.0. Because we seem to have consensus to create this new branch that won’t be necessary and we should be able to do a yt 3.5 release as soon as we feel it’s ready.

...

On 2018-05-08 14:14, Britton Smith wrote:

I'm very +1 on this. It makes the development more collaborative and easier to follow along with. I'm also fine with a lower bar for PR acceptance into the yt-4.0 branch while development continues. I think it makes sense for Nathan to either be the person merging PRs in yt-4.0 or to be designating qualified reviewers as needs be.

The index arrays development is an interesting point of discussion. What's the best way to solicit help for this? Perhaps keeping open, labeled issues or a single issue with multiple milestones?

On Mon, May 7, 2018 at 9:21 PM, Cameron Hummels wrote:

...

I'm +1 on this change, as there are a number of people (e.g., Iryna Butsky, Bili Dong, and myself at least) using sph-viz for production. By bringing it into the main yt repo, it will aid in its usage and contributions by other developers. Thanks for proposing this, Nathan!

Cameron

On Mon, May 7, 2018 at 6:01 PM, John ZuHone wrote:

...

I know you weren’t. I was just thinking out loud that we need a rethink of this, more hands, and some solid motivation for what we want to accomplish with it.

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138 (m) 781-708-5004

john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone =====================================

On May 7, 2018, at 8:40 PM, Matthew Turk wrote:

Well, I was not trying to suggest they rested solely on your shoulders.

On Mon, May 7, 2018, 7:35 PM John ZuHone wrote:

...

+1, but to be honest I think that index arrays are in trouble without some help (especially with unyt being a thing now)

===================================== John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics

60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138 (m) 781-708-5004

john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone =====================================

On May 7, 2018, at 8:29 PM, Matthew Turk wrote:

+1 to a new branch and to allowing slightly lower barrier to entry for changes in 4.0.

I would also like to point out that in contrast to when 3.0 was under development, we now have a much, much more robust testing and review infrastructure in place. This is in no small part due to your work, Nathan, as well as Kacper, and I also want to highlight that people like Cameron and Bili are already using "4.0" in production and contributing changes and bug fixes. Plus, other changes (like index arrays) are being developed by people like John, and the number of people involved in this is much more comfortable than it was back then.

We are in a much better state than we were back then,and I think this will be a much easier development process and (eventual) transition. I am excited!

-Matt

On Mon, May 7, 2018, 6:52 PM Nathan Goldbaum wrote:

...

Hi all,

Currently there is active ongoing work in my fork of yt on the "sph-viz" branch. This branch contains the bitmap index work and accompanying changes to the particle selection API. Collectively these changes constitute a major backward incompatible change and thus cannot be easily integrated into the master branch on the main repo without major disruption. Doing that would also prevent us from being able to do a 3.5 or 3.6 release (currently there's no plan for a 3.6 release but crazier things have happened).

We've already come to some level of consensus both here and in YTEP discussions that we want to make some major backward incompatible changes to yt in yt-4.0 and that the demeshening work should be one of those changes.

To ease development going forward and make it clearer both how to get started and how to contribute, I'd like to push the current state of the "sph-viz" branch to the main yt repo and rename the branch "yt-4.0". Afterwards there will be "yt-2.x", "master", "stable", and "yt-4.0" branches in the main repo. Most pull requests will still go to master, but pull requests that implement changes or fixes that are specific to the demeshening or for planned features in yt-4.0 will go to the yt-4.0 branch.

I'd additionally like to suggest that the threshold for merging code to the yt-4.0 branch should be lower than in master, possibly by reducing the number of required reviewers to one. I'd prefer not to make it harder to merge code than that to keep development velocity high and to make sure we're not blocked on getting code merged in for lack of reviewers who are familiar with the changes in the yt-4.0 branch.

I don't think it's necessary to create a new repository, since the master branch will remain the default branch. People will only get the experimental yt-4.0 branch if they explicitly fetch it and check it out.

As for testing, I think we'd just need to set up something in Jenkins to check the yt-4.0 branch as well as master for new PRs. The cloud CI services should "just work". We may end up turning off answer testing if it's too hard to have two separate sets of test answers for master and yt-4.0.

Eventually, just like for yt 3.0 if you remember that, we will merge the yt 4.0 branch in. Doing that will require ensuring the documentation has been updated and developing community consensus that the new features are ready for widespread testing among people running on the yt master branch.

Looking forward to hearing what people think about this,

-Nathan Goldbaum _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org

-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France

phone: +33.6.43.18.66.83

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org