Where to discuss NEPs (was: Re: new NEP: np.AbstractArray and np.asabstractarray)
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 7:06 AM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Well, there's a PR here: https://github.com/numpy/numpy/pull/10706 But, this raises a question :-). (One which also came up here: https://github.com/numpy/numpy/pull/10704#issuecomment-371684170) There are sensible two workflows we could use (or at least, two that I can think of): 1. We merge updates to the NEPs as we go, so that whatever's in the repo is the current draft. Anyone can go to the NEP webpage at http://numpy.org/neps (WIP, see #10702) to see the latest version of all NEPs, whether accepted, rejected, or in progress. Discussion happens on the mailing list, and line-by-line feedback can be done by quote-replying and commenting on individual lines. From time to time, the NEP author takes all the accumulated feedback, updates the document, and makes a new post to the list to let people know about the updated version. This is how python-dev handles PEPs. 2. We use Github itself to manage the review. The repo only contains "accepted" NEPs; draft NEPs are represented by open PRs, and rejected NEPs are represented by PRs that were closed-without-merging. Discussion uses Github's commenting/review tools, and happens in the PR itself. This is roughly how Rust handles their RFC process, for example: https://github.com/rust-lang/rfcs Trying to do some hybrid version of these seems like it would be pretty painful, so we should pick one. Given that historically we've tried to use the mailing list for substantive features/planning discussions, and that our NEP process has been much closer to workflow 1 than workflow 2 (e.g., there are already a bunch of old NEPs already in the repo that are effectively rejected/withdrawn), I think we should maybe continue that way, and keep discussions here? So my suggestion is discussion should happen on the list, and NEP updates should be merged promptly, or just self-merged. Sound good? -n -- Nathaniel J. Smith -- https://vorpus.org
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 8:22 PM, Nathaniel Smith <njs@pobox.com> wrote:
Agreed that overall (1) is better than (2), rejected NEPs should be visible. However there's no need for super-quick self-merge, and I think it would be counter-productive. Instead, just send a PR, leave it open for some discussion, and update for detailed comments (as well as long in-depth discussions that only a couple of people care about) in the Github UI and major ones on the list. Once it's stabilized a bit, then merge with status "Draft" and update once in a while. I think this is also much more in like with what python-dev does, I have seen substantial discussion on Github and have not seen quick self-merges. Ralf
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 10:26 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Not sure what you mean about python-dev. Are you looking at the peps repository? https://github.com/python/peps procedural stuff, maybe formatting issues or fixes to the PEP tooling. Anyway, just because python-dev does it that way doesn't mean that we have to too. But if we split discussions between GH and the mailing list, then we're definitely going to end up discussing substantive issues there (how do we know which discussions only a couple of people care about?), and trying to juggle that seems confusing to me, plus makes it harder to track down what happened later, after we've had multiple PRs each with their own comments... -n -- Nathaniel J. Smith -- https://vorpus.org
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Fri, Mar 9, 2018 at 12:00 AM, Nathaniel Smith <njs@pobox.com> wrote:
I was mostly thinking about packaging PEPs that are now also there, but were separate. Stuff like https://github.com/pypa/interoperability-peps/pull/54. There seems to be significantly more comments on packaging things than on other PEPs.
It's not imho, because it's what we already do on this list. Github is a superior review interface over mailing list, so my vote goes to using that interface, while keeping this list in the loop on critical stuff and decisions about to be made. Ralf
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Apparently, where and how to discuss enhancement proposals was recently a topic on the python mailing list as well -- see the write-up at LWN: https://lwn.net/SubscriberLink/749200/4343911ee71e35cf/ The conclusion seems to be that one should switch to mailman3... -- Marten
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Reviving this discussion -- I don't really care what our policy is, but can we make a decision one way or the other about where we discuss NEPs? We've had a revival of NEP writing recently, so this is very timely. Previously, I was in slight favor of doing discussion on GitHub. Now that I've started doing a bit of NEP writing, I've started to swing the other way, since it would be nice to be able to reference draft/rejected NEPs in a consistent way -- and rendered HTML is more readable than raw RST in pull requests. On Wed, Mar 14, 2018 at 6:52 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Tue, May 29, 2018 at 9:46 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
My understanding of the discussion at the sprint was that we favored quick commits of NEPs with extended discussions of them on the list. Updates and changes would go in through the normal PR process. In practice, I expect there will be some overlap, I think the important thing is the quick commit with the understanding that the NEPs are only proposals until formally adopted. I think the formal adoption process is not well defined... <snip> Chuck
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Tue, May 29, 2018 at 9:24 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
For the formal adoption part, how about: 1. When discussions/disagreements appear to have been resolved, a NEP author or a core developer may propose that the NEP is formally adopted. 2. The formal decision is made by consensus, according to https://docs.scipy.org/doc/numpy/dev/governance/governance.html#consensus-ba... (which also covers how to handle consensus not being reached). Ralf
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 11:26 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
I have a slight preference for managing the discussion on Github. Note that I added a `component: NEP` label and that discussion can take place on merged/closed PRs, the index could also contain links to proposed NEP PRs. If we just left PR open until acceptance/rejection the label would allow the proposed NEPs to be easily found, especially if we include the NEP number in the title, something like `NEP-10111: ` . Chuck
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
I also have a slight preference for managing the discussion on GitHub, which is a bit more fully featured than email for long discussion (e.g., it supports code formatting and editing comments). But I'm really OK either way as long as discussion is kept in one place. We could still stipulate that NEPs are advertised on the mailing list: first, to announce them, and second, before merging them marked as accepted. We could even still merge rejected/abandoned NEPs as long as they are clearly marked. On Fri, Mar 9, 2018 at 7:24 AM Charles R Harris <charlesr.harris@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
I'll note that we basically used GitHub for revising __array_ufunc__ NEP, and I think that worked out better for everyone involved. The discussion was a little too specialized and high volume to be well handled on the mailing list. On Fri, Mar 9, 2018 at 8:58 AM Stephan Hoyer <shoyer@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Fri, 09 Mar 2018 17:00:43 +0000, Stephan Hoyer wrote:
A disadvantage of GitHub PR comments is that they do not track sub-threads of conversation, so you cannot "reply to" a previous concern directly. PRs also mix inline comments (that become much less visible after rebases and updates) and "story line" comments. These two "modes" of commenting, substantive discussion around ideas, v.s. concerns about specific phrasing, usage of words, typos, content of code snippets, etc., may require different approaches. It would be quite easy to redirect the prior to the mailing list and the latter to the GitHub PR. I'm also not too keen on repeated PR creation and merging (it splits up the PR discussion even further). Why not simply hold off until the PEP is ready, and view the documents on GitHub? The rendering there is just as good. +1 also on merging rejected PEPs, once they are fully developed. Stéfan
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Hi Nathaniel, astropy is an example of a project that does essentially all discussion of its "Astropy Proposals for Enhancement" on github. I actually like the numpy approach of sending anything to the mailing list that deserves community input (which includes NEP by their very nature). I don't think it has to be either/or, though; maybe the preferred approach is in fact a combination, where the draft is send to the mailing list, initial general comments are incorporated, and then discussion moves to github when one is past the "general interest" stage. When exactly this happens will be somewhat subjective, but probably is not important to nail down anyway. All the best, Marten p.s. I think the __array_ufunc__ discussion indeed showed that github can work, but only once the general ideas are agreed upon - the initial discussion become hopeless to follow (though I'm not sure a mailing list discussion would have been any better).
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Fri, Mar 9, 2018 at 11:51 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Yeah, I actually find email much easier for this kind of complex high-volume discussion. Even if lots of people don't use traditional threaded mail clients anymore [1], archives are still threaded, and the tools that make line-by-line responses easy and the ability to split off conversations are both really helpful. (E.g., the way I split this thread off from the original one :-).) The __array_ufunc__ discussion was almost impenetrable on GH, I think. I admit though that some of this is probably just that I'm more used to the email-based discussion workflow. Honestly none of these tools are particularly amazing, and the __array_ufunc__ conversation would have been difficult and inaccessible to outsiders no matter what medium we used. It's much more important that we just pick something and use it consistently than that pick the Most Optimal Solution. [1] Meaning this, not gmail's threads: https://en.wikipedia.org/wiki/Conversation_threading#/media/File:Nntp.jpg
I don't think we should worry about this. Fiddly detail comments are, by definition, not super important, and generally make up a tiny volume of the discussion around a proposal. Also in practice reviewers are no good at splitting up substantive comments from fiddly details: the review workflow is that you read through and as thoughts occur you write them down, so even if you start out thinking "okay, I'm only going to comment on typos", then half-way through some paragraph sparks a thought and suddenly you're writing something substantive (and I'm as guilty of this as anyone, maybe more so...). Asking people to classify their comments and then chiding them for putting them in the wrong place etc. isn't a good use of time. Let's just pick one place for everything and stick with it.
Well, if we aren't using PRs for discussion then multiple PRs are fine :-). And merging changes quickly is helpful because it makes the rendered NEPs page a single one-stop-shop to see all the latest NEPs, no matter what their current status. If we do use PRs for discussion, then I agree that we should try to keep the PR open until the NEP is "done", to minimize the splitting of discussion. This does create a bit of extra friction because it turns out that "is this done?" is not something you can really ever answer for certain :-). Even after PEPs are accepted they usually end up getting some further tweaks once people start implementing them. Sometimes PEPs get abandoned in "Draft" state without ever being accepted/rejected, and sometimes a PEP that had been abandoned for years gets picked up and finished. You can see this in the Rust RFC guidelines too [2]; they specifically address the issue of post-merge changes, and it sounds like their solution is that if a substantive issue is discovered in an accepted RFC, then you have to create a new "fixup" RFC, which then gets its own PR for discussion. I guess if this were our process then __array_ufunc__ would have ended up with ~3 NEPs :-). This is all doable -- every approach has trade-offs. But we should pick one, so we can adapt to those trade-offs. [2] https://github.com/rust-lang/rfcs#the-rfc-life-cycle -n -- Nathaniel J. Smith -- https://vorpus.org
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Thu, 08 Mar 2018 20:22:29 -0800, Nathaniel Smith wrote:
If we go this route, it may also be useful to give some more guidance on how complete we expect a first draft of a NEP to be before it is submitted as a PR. We currently only have: """ The NEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is suitable for a NEP. Posting to the numpy-discussion mailing list is the best way to go about doing this. Following a discussion on the mailing list, the proposal should be submitted as a draft NEP via a GitHub pull request to the doc/neps directory [...] """ Best regards Stéfan
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 8:22 PM, Nathaniel Smith <njs@pobox.com> wrote:
Agreed that overall (1) is better than (2), rejected NEPs should be visible. However there's no need for super-quick self-merge, and I think it would be counter-productive. Instead, just send a PR, leave it open for some discussion, and update for detailed comments (as well as long in-depth discussions that only a couple of people care about) in the Github UI and major ones on the list. Once it's stabilized a bit, then merge with status "Draft" and update once in a while. I think this is also much more in like with what python-dev does, I have seen substantial discussion on Github and have not seen quick self-merges. Ralf
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 10:26 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Not sure what you mean about python-dev. Are you looking at the peps repository? https://github.com/python/peps procedural stuff, maybe formatting issues or fixes to the PEP tooling. Anyway, just because python-dev does it that way doesn't mean that we have to too. But if we split discussions between GH and the mailing list, then we're definitely going to end up discussing substantive issues there (how do we know which discussions only a couple of people care about?), and trying to juggle that seems confusing to me, plus makes it harder to track down what happened later, after we've had multiple PRs each with their own comments... -n -- Nathaniel J. Smith -- https://vorpus.org
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Fri, Mar 9, 2018 at 12:00 AM, Nathaniel Smith <njs@pobox.com> wrote:
I was mostly thinking about packaging PEPs that are now also there, but were separate. Stuff like https://github.com/pypa/interoperability-peps/pull/54. There seems to be significantly more comments on packaging things than on other PEPs.
It's not imho, because it's what we already do on this list. Github is a superior review interface over mailing list, so my vote goes to using that interface, while keeping this list in the loop on critical stuff and decisions about to be made. Ralf
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Apparently, where and how to discuss enhancement proposals was recently a topic on the python mailing list as well -- see the write-up at LWN: https://lwn.net/SubscriberLink/749200/4343911ee71e35cf/ The conclusion seems to be that one should switch to mailman3... -- Marten
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Reviving this discussion -- I don't really care what our policy is, but can we make a decision one way or the other about where we discuss NEPs? We've had a revival of NEP writing recently, so this is very timely. Previously, I was in slight favor of doing discussion on GitHub. Now that I've started doing a bit of NEP writing, I've started to swing the other way, since it would be nice to be able to reference draft/rejected NEPs in a consistent way -- and rendered HTML is more readable than raw RST in pull requests. On Wed, Mar 14, 2018 at 6:52 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Tue, May 29, 2018 at 9:46 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
My understanding of the discussion at the sprint was that we favored quick commits of NEPs with extended discussions of them on the list. Updates and changes would go in through the normal PR process. In practice, I expect there will be some overlap, I think the important thing is the quick commit with the understanding that the NEPs are only proposals until formally adopted. I think the formal adoption process is not well defined... <snip> Chuck
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Tue, May 29, 2018 at 9:24 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
For the formal adoption part, how about: 1. When discussions/disagreements appear to have been resolved, a NEP author or a core developer may propose that the NEP is formally adopted. 2. The formal decision is made by consensus, according to https://docs.scipy.org/doc/numpy/dev/governance/governance.html#consensus-ba... (which also covers how to handle consensus not being reached). Ralf
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Thu, Mar 8, 2018 at 11:26 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
I have a slight preference for managing the discussion on Github. Note that I added a `component: NEP` label and that discussion can take place on merged/closed PRs, the index could also contain links to proposed NEP PRs. If we just left PR open until acceptance/rejection the label would allow the proposed NEPs to be easily found, especially if we include the NEP number in the title, something like `NEP-10111: ` . Chuck
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
I also have a slight preference for managing the discussion on GitHub, which is a bit more fully featured than email for long discussion (e.g., it supports code formatting and editing comments). But I'm really OK either way as long as discussion is kept in one place. We could still stipulate that NEPs are advertised on the mailing list: first, to announce them, and second, before merging them marked as accepted. We could even still merge rejected/abandoned NEPs as long as they are clearly marked. On Fri, Mar 9, 2018 at 7:24 AM Charles R Harris <charlesr.harris@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
I'll note that we basically used GitHub for revising __array_ufunc__ NEP, and I think that worked out better for everyone involved. The discussion was a little too specialized and high volume to be well handled on the mailing list. On Fri, Mar 9, 2018 at 8:58 AM Stephan Hoyer <shoyer@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Fri, 09 Mar 2018 17:00:43 +0000, Stephan Hoyer wrote:
A disadvantage of GitHub PR comments is that they do not track sub-threads of conversation, so you cannot "reply to" a previous concern directly. PRs also mix inline comments (that become much less visible after rebases and updates) and "story line" comments. These two "modes" of commenting, substantive discussion around ideas, v.s. concerns about specific phrasing, usage of words, typos, content of code snippets, etc., may require different approaches. It would be quite easy to redirect the prior to the mailing list and the latter to the GitHub PR. I'm also not too keen on repeated PR creation and merging (it splits up the PR discussion even further). Why not simply hold off until the PEP is ready, and view the documents on GitHub? The rendering there is just as good. +1 also on merging rejected PEPs, once they are fully developed. Stéfan
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Hi Nathaniel, astropy is an example of a project that does essentially all discussion of its "Astropy Proposals for Enhancement" on github. I actually like the numpy approach of sending anything to the mailing list that deserves community input (which includes NEP by their very nature). I don't think it has to be either/or, though; maybe the preferred approach is in fact a combination, where the draft is send to the mailing list, initial general comments are incorporated, and then discussion moves to github when one is past the "general interest" stage. When exactly this happens will be somewhat subjective, but probably is not important to nail down anyway. All the best, Marten p.s. I think the __array_ufunc__ discussion indeed showed that github can work, but only once the general ideas are agreed upon - the initial discussion become hopeless to follow (though I'm not sure a mailing list discussion would have been any better).
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Fri, Mar 9, 2018 at 11:51 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Yeah, I actually find email much easier for this kind of complex high-volume discussion. Even if lots of people don't use traditional threaded mail clients anymore [1], archives are still threaded, and the tools that make line-by-line responses easy and the ability to split off conversations are both really helpful. (E.g., the way I split this thread off from the original one :-).) The __array_ufunc__ discussion was almost impenetrable on GH, I think. I admit though that some of this is probably just that I'm more used to the email-based discussion workflow. Honestly none of these tools are particularly amazing, and the __array_ufunc__ conversation would have been difficult and inaccessible to outsiders no matter what medium we used. It's much more important that we just pick something and use it consistently than that pick the Most Optimal Solution. [1] Meaning this, not gmail's threads: https://en.wikipedia.org/wiki/Conversation_threading#/media/File:Nntp.jpg
I don't think we should worry about this. Fiddly detail comments are, by definition, not super important, and generally make up a tiny volume of the discussion around a proposal. Also in practice reviewers are no good at splitting up substantive comments from fiddly details: the review workflow is that you read through and as thoughts occur you write them down, so even if you start out thinking "okay, I'm only going to comment on typos", then half-way through some paragraph sparks a thought and suddenly you're writing something substantive (and I'm as guilty of this as anyone, maybe more so...). Asking people to classify their comments and then chiding them for putting them in the wrong place etc. isn't a good use of time. Let's just pick one place for everything and stick with it.
Well, if we aren't using PRs for discussion then multiple PRs are fine :-). And merging changes quickly is helpful because it makes the rendered NEPs page a single one-stop-shop to see all the latest NEPs, no matter what their current status. If we do use PRs for discussion, then I agree that we should try to keep the PR open until the NEP is "done", to minimize the splitting of discussion. This does create a bit of extra friction because it turns out that "is this done?" is not something you can really ever answer for certain :-). Even after PEPs are accepted they usually end up getting some further tweaks once people start implementing them. Sometimes PEPs get abandoned in "Draft" state without ever being accepted/rejected, and sometimes a PEP that had been abandoned for years gets picked up and finished. You can see this in the Rust RFC guidelines too [2]; they specifically address the issue of post-merge changes, and it sounds like their solution is that if a substantive issue is discovered in an accepted RFC, then you have to create a new "fixup" RFC, which then gets its own PR for discussion. I guess if this were our process then __array_ufunc__ would have ended up with ~3 NEPs :-). This is all doable -- every approach has trade-offs. But we should pick one, so we can adapt to those trade-offs. [2] https://github.com/rust-lang/rfcs#the-rfc-life-cycle -n -- Nathaniel J. Smith -- https://vorpus.org
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Thu, 08 Mar 2018 20:22:29 -0800, Nathaniel Smith wrote:
If we go this route, it may also be useful to give some more guidance on how complete we expect a first draft of a NEP to be before it is submitted as a PR. We currently only have: """ The NEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is suitable for a NEP. Posting to the numpy-discussion mailing list is the best way to go about doing this. Following a discussion on the mailing list, the proposal should be submitted as a draft NEP via a GitHub pull request to the doc/neps directory [...] """ Best regards Stéfan
participants (7)
-
Charles R Harris
-
Marten van Kerkwijk
-
Matti Picus
-
Nathaniel Smith
-
Ralf Gommers
-
Stefan van der Walt
-
Stephan Hoyer