Hoop jumping, and other sports
Hi All, I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need them, for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line. Furthermore, that got me thinking that there are probably other things we could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them. Chuck
I think that makes sense. Incidentally, I had imitated numpy's abbreviated prefixes on various other projects until it was recently pointed out to me that abbreviations, especially more cryptic ones like "ENH", "REF", and "MNT", are a particularly tricky "hoop" to jump through for non-native speakers of English. Daniel B. Allan, Ph.D Associate Computational Scientist, Brookhaven National Lab (631) 344-3281 (no voicemail set up) ________________________________ From: NumPy-Discussion [numpy-discussion-bounces+dallan=bnl.gov@python.org] on behalf of Charles R Harris [charlesr.harris@gmail.com] Sent: Wednesday, February 07, 2018 4:26 PM To: numpy-discussion Subject: [Numpy-discussion] Hoop jumping, and other sports Hi All, I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need them, for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line. Furthermore, that got me thinking that there are probably other things we could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them. Chuck
Other factors hindering new contributors: 1) Being unfamiliar with git. e.g. knowing that you have to fork numpy first, then clone from your fork, then create a feature branch. That's just the """straightforward bit""". It's hell the first time you have to rebase something/correct commit messages/squash, etc. 2) I have a mindblock on the correct way to use ReStructured Text in docstrings. When do I use backticks/double backticks, etc. Getting documentation changes to perfection is hard.
On Wed, Feb 7, 2018 at 11:04 PM, Andrew Nelson <andyfaff@gmail.com> wrote:
Other factors hindering new contributors:
1) Being unfamiliar with git. e.g. knowing that you have to fork numpy first, then clone from your fork, then create a feature branch. That's just the """straightforward bit""". It's hell the first time you have to rebase something/correct commit messages/squash, etc.
Is there anything specific to NumPy's use of git which is unsual which needs particular warning for newcomers?
2) I have a mindblock on the correct way to use ReStructured Text in docstrings. When do I use backticks/double backticks, etc. Getting documentation changes to perfection is hard.
Me too. I still regularly copy-paste my docstrings from the Python code into an RST editor to confirm the formatting, or do any non-trivial editing. I also found I'd often only discover invalid RST when trying to build a project's documentation (using Sphinx or epydoc as appropriate for the project at hand), so I wrong a flake8 plugin to do basic reStructuredText validation: https://pypi.python.org/pypi/flake8-rst-docstrings https://github.com/peterjc/flake8-rst-docstrings One could imagine checking NumPy's specific docstring style with a more complicated flake8 plugin? Peter
On Wed, 07 Feb 2018 14:26:37 -0700, Charles R Harris wrote:
I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need them, for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line. Furthermore, that got me thinking that there are probably other things we could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them.
For scikit-image, we've started a policy of pushing minor edits (spelling corrections, sentence restructuring, etc.) directly to the PR branch, instead of flagging those during a review (we also have a PEP8 bot that mentions PEP8 issues, which makes the human reviews less nitpicky). To avoid users having to rebase, we now squash all PRs before merge (and, typically, remove the commit messages). This also means we can allow merges with master to exist in the PR history. Of course, if it is clear that a user took particular care to hand-craft each patch we don't squash, but those cases seem to be in the minority. Thank you for bringing this up, Chuck; on projects with very limited developer hours (almost any open source project?) whatever we can do to lower the contribution barrier helps a great deal. Best regards Stéfan
On Wed, Feb 7, 2018 at 11:29 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need
On Wed, 07 Feb 2018 14:26:37 -0700, Charles R Harris wrote: them,
for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line.
+1 we got more picky over time, and that was never the intention. They're still useful, but we shouldn't request rework for such a minor thing.
Furthermore, that got me thinking that there are probably other things we
could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them.
For scikit-image, we've started a policy of pushing minor edits (spelling corrections, sentence restructuring, etc.) directly to the PR branch, instead of flagging those during a review (we also have a PEP8 bot that mentions PEP8 issues, which makes the human reviews less nitpicky).
+1. Especially useful for docstring formatting, that doesn't get picked up by a PEP8 bot Ralf
To avoid users having to rebase, we now squash all PRs before merge (and, typically, remove the commit messages). This also means we can allow merges with master to exist in the PR history. Of course, if it is clear that a user took particular care to hand-craft each patch we don't squash, but those cases seem to be in the minority.
Thank you for bringing this up, Chuck; on projects with very limited developer hours (almost any open source project?) whatever we can do to lower the contribution barrier helps a great deal.
Best regards Stéfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Thu, Feb 8, 2018 at 11:22 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Feb 7, 2018 at 11:29 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
For scikit-image, we've started a policy of pushing minor edits (spelling corrections, sentence restructuring, etc.) directly to the PR branch, instead of flagging those during a review (we also have a PEP8 bot that mentions PEP8 issues, which makes the human reviews less nitpicky).
+1. Especially useful for docstring formatting, that doesn't get picked up by a PEP8 bot
Ralf
Were is the current NumPy PEP8 bot setup? I don't see flake8 or similar under the TravisCI or AppVeyor setup - which is where I am using a flake8 plugin to at least partially validate RST docstrings [*] (and catch things automatically in pull requests). Peter [*] My plugin: https://github.com/peterjc/flake8-rst-docstrings https://pypi.python.org/pypi/flake8-rst-docstrings
On Thu, Feb 8, 2018 at 11:32 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Thu, Feb 8, 2018 at 11:22 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Feb 7, 2018 at 11:29 PM, Stefan van der Walt < stefanv@berkeley.edu> wrote:
For scikit-image, we've started a policy of pushing minor edits (spelling corrections, sentence restructuring, etc.) directly to the PR branch, instead of flagging those during a review (we also have a PEP8 bot that mentions PEP8 issues, which makes the human reviews less nitpicky).
+1. Especially useful for docstring formatting, that doesn't get picked up by a PEP8 bot
Ralf
Were is the current NumPy PEP8 bot setup? I don't see flake8 or similar under the TravisCI or AppVeyor setup - which is where I am using a flake8 plugin to at least partially validate RST docstrings [*] (and catch things automatically in pull requests).
Good question. It's missing, I confused it with the scipy version: https://github.com/scipy/scipy/blob/master/.travis.yml#L18. We should copy that build matrix entry for numpy. Ralf
Peter
[*] My plugin:
https://github.com/peterjc/flake8-rst-docstrings https://pypi.python.org/pypi/flake8-rst-docstrings _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On 02/07/2018 04:26 PM, Charles R Harris wrote:
Hi All,
I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need them, for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line. Furthermore, that got me thinking that there are probably other things we could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them.
Chuck
When I was first contributing, the main obstacle was not the nitpicks but reading through all the contributor guidelines pages, as well as learning github. I also remember finding it hard to find that documentation in the first place. It is at https://docs.scipy.org/doc/numpy/dev/index.html and a shorter summary at https://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html Maybe we should have a much more prominent link about how to contribute, eg on the main "README.md" front page, or at the start of the user guide, which links to a "really really" short contributing guide for someone who does not use github, maybe a screenful or two only. Even the short development workflow above has lots of info that usually isn't needed and takes a long time to read through. Allan
On Thu, Feb 8, 2018 at 12:35 AM, Allan Haldane <allanhaldane@gmail.com> wrote:
On 02/07/2018 04:26 PM, Charles R Harris wrote:
Hi All,
I was thinking about things to do to simplify the NumPy development process. One thing that came to mind was our use of prefixes on commits, BUG, TST, etc. Those prefixes were originally introduced by David Cournapeau when he was managing releases in order help him track commits that might need backports. I like the prefixes, but now that we are organized by PRs, rather than commits, the only place we really need them, for some meaning of "need", is in the commit titles, and maintainers can change and edit those without problems. So I would like to propose that we no longer be picky about having them in the commit summary line. Furthermore, that got me thinking that there are probably other things we could do to simplify the development process. So I'd like folks to weigh in with other ideas for simplification or complaints about nit picky things that have annoyed them.
Chuck
When I was first contributing, the main obstacle was not the nitpicks but reading through all the contributor guidelines pages, as well as learning github. I also remember finding it hard to find that documentation in the first place.
It is at https://docs.scipy.org/doc/numpy/dev/index.html and a shorter summary at https://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html
Maybe we should have a much more prominent link about how to contribute, eg on the main "README.md" front page, or at the start of the user guide, which links to a "really really" short contributing guide for someone who does not use github, maybe a screenful or two only. Even the short development workflow above has lots of info that usually isn't needed and takes a long time to read through.
Allan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
As a new (though so far superficial) contributor I can say that I didn't find it difficult at all to locate the resources needed for contributing. I found the gitwash link very easily and naturally, and I didn't find it too long nor confusing (but I did have a fresh understanding of git itself, so I can't reliably assess the contents from a git newcomer's standpoint). The willingness to touch numpy without a thorough understanding of its internals is much more of a barrier at least in my case. András
participants (8)
-
Allan Haldane -
Allan, Daniel -
Andras Deak -
Andrew Nelson -
Charles R Harris -
Peter Cock -
Ralf Gommers -
Stefan van der Walt