Hi all, Since we expect to be writing some NEPs in the near future, Nathaniel and I were looking at how they're organized, and realized that the process is a bit underspecified and it's a hard to tell the status of things. So I'm thinking of putting together some better tools and documentation, and wanted to get some quick feedback before I runoff in the wrong direction. The goal is not to add a ton of new process, but to better document the current process. I would also like to add a little additional structure to make it easier to understand the process for new contributors and to make the status of NEPS easier to understand. After reviewing the existing system, looking at how other projects do this, and discussing this with Nathaniel, I tentatively plan to work on the following: 1. Standardize the NEP metadata (e.g., whether they're draft/accepted/implemented) 2. Write a NEP to explain the purpose and process (think PEP 1) 3. Create a NEP template (think PEP 12) 4. Add metadata to old NEPs 5. Automate updating the NEP website and autogenerating the index based on the NEP metadata. Nathaniel and I have already started discussing some of these items and would love to get some feedback soon. Assuming that sounds good, my tentative next steps are: - I'll draft a purpose and process NEP based on PEP 1 and a few other projects. - I'll also create a draft NEP template. - I'll move the NEPs into their own repo (something like numpy/neps), and set up an automated system (RTD or Github pages) to render and publish them with some useful index. Does this seem like a reasonable way to start? Best regards, Jarrod
On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman <millman@berkeley.edu> wrote:
That all sounds good to me.
sounds good - I'll move the NEPs into their own repo (something like numpy/neps),
This doesn't sound ideal to me - NEPs are important pieces of documentation, so I'd rather keep them included in the main docs. and set up an automated system (RTD or Github pages) to
render and publish them with some useful index.
If you could copy over the scipy method to rebuild the docs on each merge into master, that would achieve the same purpose. Compare https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to http://scipy.github.io/devdocs/, always up-to-date). Ralf
On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Yeah, we were debating back and forth on this -- I can see arguments either way. The reasons we were leaning towards splitting them out are: - it would be great to make our regular docs auto-generated, but we didn't necessarily want to block this on that - part of the idea is to auto-generate the NEP index out of the metadata inside each NEP file, which is going to involve writing some code and integrating it into the NEP build. This seems easier if we don't have to integrate it into the overall doc build process too, which already has a lot of custom code. - NEPs are really part of the development process, not an output for end-users -- they're certainly useful to have available as a reference, but if we're asking end-users to look at them on a regular basis then I think we've messed up and should improve our actual documentation :-) - NEPs have a different natural life-cycle than numpy itself. Right now, if I google "numpy neps", the first hit is the 1.13 version of the NEPs, and the third hit is someone else's copy of the 1.9 version of the NEPs. What you actually want in every case is the latest development version of the NEPs, and the idea of "numpy 1.13 NEPs" doesn't even make sense, because NEPs are not describing a specific numpy release. -n -- Nathaniel J. Smith -- https://vorpus.org
On Wed, Dec 6, 2017 at 9:49 AM, Nathaniel Smith <njs@pobox.com> wrote:
I agree. The usefulness of NEPs (and PEPs) as documentation degrades over time. The functionality matches the NEP at the time that it is accepted, but features evolve over time through the normal, non-NEP development process. Personally, I've frequently drove myself into corners reading a PEP to learn about a new feature only to "learn" things that did not make it into the final implementation or were quickly overtaken by later development. We should not rely on NEPs to provide documentation for users. The regular docs should be the authority. To the extent that the NEPs happen to provide useful documentation for the new feature (and represent a significant amount of sunk effort to draft that documentation), we should feel free to copy-paste that into the regular docs and evolve it from there. -- Robert Kern
On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith <njs@pobox.com> wrote:
This is easy, it's just copying a trick from SciPy. (and that's work that anyway needs doing at some point, sooner rather than later) - part of the idea is to auto-generate the NEP index out of the
This is easy too, if you have your script to auto-generate an index file, it's just calling that file from within `doc/Makefile`. - NEPs are really part of the development process, not an output for
The last two points are good arguments, I agree that they shouldn't serve as documentation. A separate repo has downsides though (discoverability etc.), we also keep our dev docs within the numpy repo and you can make exactly the same argument about those as about NEPs. So I'd still suggest keeping them where they are. Or otherwise move all development related docs. It's probably less effort even to keep them where they are rather than creating a separate repo, setting up RTD for it, and linking to that from our main docs. Ralf
I was planning on looking at/working on the main doc generating system and the main webpage (for numpy and scipy) soon (over the winter break), but I didn't want to get too many things in the discussion right now. My immediate interest is getting agreement on the first two items: - A purpose and process NEP based on PEP 1 and a few other projects. - A draft NEP template. I am ambivalent about whether we need a separate repo. My immediate plan is start drafting text as a PR to numpy/doc/neps on Thursday (assuming no major objections arise before then). That way we can discuss the NEP purpose/process and template docs, while I look into the build system. I was imaging doing something similar to what I did for for networkx this summer: https://networkx.github.io/ The latest docs are autogenerated whenever a PR is merged into master: https://networkx.github.io/documentation/latest/ I will look into the scipy system as well and make sure we don't have an explosion of systems. Jarrod On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Dec 6, 2017 at 2:58 PM, Jarrod Millman <millman@berkeley.edu> wrote:
+1 thanks Jarrod. And thanks Nelle for offering to work on the automation! Ralf
Note that the build/doc/release systems of numpy and scipy are ~95% common (because they're mostly maintained by the same people), so syncing improvements between numpy and scipy will almost always require the least effort. Ralf
On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Are these the dev docs you're thinking of? https://docs.scipy.org/doc/numpy-dev/dev/index.html Regarding discoverability, right now it looks like the only way to find the latest NEPs on google is by searching for something like "numpy-dev neps", which is pretty obscure. (It took me 4 tries to find something that worked. "numpy neps" seemed to work, but actually sent me to an out-of-date snapshot.) In Python, the PEP web pages are rebuilt on something like a 6 hour timer, and it's actually super annoying, because it means that when someone posts to the list like "hey, I just pushed a new version, tell me what you think", everyone goes and finds the old stale version, sometimes people start critiquing it, ... it's just confusing all around. So I do think we want to make sure there's some simple way to find them, and that it leads to the latest version, not a stale build or an old snapshot. Moving NEPs + development docs to their own dedicated repo would resolve this and seems like a plausible option to me. We could probably do better than we are now with the regular docs too. Though the experience with PEPs does make me a bit nervous about having versioned snapshots of the NEPs in all our old versioned manuals (which have tons of google-juice). -n -- Nathaniel J. Smith -- https://vorpus.org
On Tue, Dec 5, 2017 at 10:44 PM, Nathaniel Smith <njs@pobox.com> wrote:
maybe I have a different google, but the first search result for me for "numpy nep" is https://docs.scipy.org/doc/numpy-dev/neps/index.html developer guides are on top here https://docs.scipy.org/doc/ using the link for "Complete Numpy Manual" on that page has dev docs and neps listed. But I don't see "numpy doc standard" which I usually need to google and the search leads to github page for it. Searching for specific items leads often to older versions in google and needs manual interventions, e.g. edit the version number in link Aside: My main occasion for reading Peps (besides quoting ZEN) is to follow the links in the python what's new pages, e.g. https://docs.python.org/3/whatsnew/3.6.html Josef
On Wed, Dec 6, 2017 at 5:39 PM, <josef.pktd@gmail.com> wrote:
indeed. possibly governance as well.
These have recently been moved to https://numpydoc.readthedocs.io/en/latest/format.html Ralf
Would be great to have structure, and especially a template - ideally, the latter is enough for someone to create a NEP, i.e., has lots of in-template documentation. One thing I'd recommend thinking a little about is to what extend a NEP is "frozen" after acceptance. In astropy we've seen situations where it helps to clarify details later, and it may be good to think beforehand what one wants. In my opinion, one should allow clarifications of accepted NEPs, and major editing of ones still pending (as happened for __[numpy|array]_ufunc__). I think the location is secondary, but for what it is worth, I'm not fond of the astropy APEs being in a separate repository, mostly because I like detailed discussion of everything related in the project to happen in one place on github. Also, having to clone a repository is yet another hurdle for doing stuff. So, I'd suggest to keep the NEPs in the main repository. -- Marten
Great idea -- thanks for pushing this forward all. In the end, you can have the NEPs in a separate repo, and still publish them closely with the main docs (intersphinx is pretty cool), or have them in the same repo and publish them separately. So I say let the folks doing the work decide what workflow works best for them. Comments on a couple other points: I find myself going back to PEPs quite a bit -- mostly to understand the hows an whys of a feature, rather than the how-to-use its. And yes -- we should keep NEPs updated -- they certainly should be edited for typos and minor clarifications, but It's particularly important if the implementation ends up differing a bit from what was expected when the NEP was written. I'm not sure what the PEP policy is about this, but they are certainly maintained with regard to typos and the like. -CHB On Wed, Dec 6, 2017 at 10:43 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman <millman@berkeley.edu> wrote:
That all sounds good to me.
sounds good - I'll move the NEPs into their own repo (something like numpy/neps),
This doesn't sound ideal to me - NEPs are important pieces of documentation, so I'd rather keep them included in the main docs. and set up an automated system (RTD or Github pages) to
render and publish them with some useful index.
If you could copy over the scipy method to rebuild the docs on each merge into master, that would achieve the same purpose. Compare https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to http://scipy.github.io/devdocs/, always up-to-date). Ralf
On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Yeah, we were debating back and forth on this -- I can see arguments either way. The reasons we were leaning towards splitting them out are: - it would be great to make our regular docs auto-generated, but we didn't necessarily want to block this on that - part of the idea is to auto-generate the NEP index out of the metadata inside each NEP file, which is going to involve writing some code and integrating it into the NEP build. This seems easier if we don't have to integrate it into the overall doc build process too, which already has a lot of custom code. - NEPs are really part of the development process, not an output for end-users -- they're certainly useful to have available as a reference, but if we're asking end-users to look at them on a regular basis then I think we've messed up and should improve our actual documentation :-) - NEPs have a different natural life-cycle than numpy itself. Right now, if I google "numpy neps", the first hit is the 1.13 version of the NEPs, and the third hit is someone else's copy of the 1.9 version of the NEPs. What you actually want in every case is the latest development version of the NEPs, and the idea of "numpy 1.13 NEPs" doesn't even make sense, because NEPs are not describing a specific numpy release. -n -- Nathaniel J. Smith -- https://vorpus.org
On Wed, Dec 6, 2017 at 9:49 AM, Nathaniel Smith <njs@pobox.com> wrote:
I agree. The usefulness of NEPs (and PEPs) as documentation degrades over time. The functionality matches the NEP at the time that it is accepted, but features evolve over time through the normal, non-NEP development process. Personally, I've frequently drove myself into corners reading a PEP to learn about a new feature only to "learn" things that did not make it into the final implementation or were quickly overtaken by later development. We should not rely on NEPs to provide documentation for users. The regular docs should be the authority. To the extent that the NEPs happen to provide useful documentation for the new feature (and represent a significant amount of sunk effort to draft that documentation), we should feel free to copy-paste that into the regular docs and evolve it from there. -- Robert Kern
On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith <njs@pobox.com> wrote:
This is easy, it's just copying a trick from SciPy. (and that's work that anyway needs doing at some point, sooner rather than later) - part of the idea is to auto-generate the NEP index out of the
This is easy too, if you have your script to auto-generate an index file, it's just calling that file from within `doc/Makefile`. - NEPs are really part of the development process, not an output for
The last two points are good arguments, I agree that they shouldn't serve as documentation. A separate repo has downsides though (discoverability etc.), we also keep our dev docs within the numpy repo and you can make exactly the same argument about those as about NEPs. So I'd still suggest keeping them where they are. Or otherwise move all development related docs. It's probably less effort even to keep them where they are rather than creating a separate repo, setting up RTD for it, and linking to that from our main docs. Ralf
I was planning on looking at/working on the main doc generating system and the main webpage (for numpy and scipy) soon (over the winter break), but I didn't want to get too many things in the discussion right now. My immediate interest is getting agreement on the first two items: - A purpose and process NEP based on PEP 1 and a few other projects. - A draft NEP template. I am ambivalent about whether we need a separate repo. My immediate plan is start drafting text as a PR to numpy/doc/neps on Thursday (assuming no major objections arise before then). That way we can discuss the NEP purpose/process and template docs, while I look into the build system. I was imaging doing something similar to what I did for for networkx this summer: https://networkx.github.io/ The latest docs are autogenerated whenever a PR is merged into master: https://networkx.github.io/documentation/latest/ I will look into the scipy system as well and make sure we don't have an explosion of systems. Jarrod On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Dec 6, 2017 at 2:58 PM, Jarrod Millman <millman@berkeley.edu> wrote:
+1 thanks Jarrod. And thanks Nelle for offering to work on the automation! Ralf
Note that the build/doc/release systems of numpy and scipy are ~95% common (because they're mostly maintained by the same people), so syncing improvements between numpy and scipy will almost always require the least effort. Ralf
On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Are these the dev docs you're thinking of? https://docs.scipy.org/doc/numpy-dev/dev/index.html Regarding discoverability, right now it looks like the only way to find the latest NEPs on google is by searching for something like "numpy-dev neps", which is pretty obscure. (It took me 4 tries to find something that worked. "numpy neps" seemed to work, but actually sent me to an out-of-date snapshot.) In Python, the PEP web pages are rebuilt on something like a 6 hour timer, and it's actually super annoying, because it means that when someone posts to the list like "hey, I just pushed a new version, tell me what you think", everyone goes and finds the old stale version, sometimes people start critiquing it, ... it's just confusing all around. So I do think we want to make sure there's some simple way to find them, and that it leads to the latest version, not a stale build or an old snapshot. Moving NEPs + development docs to their own dedicated repo would resolve this and seems like a plausible option to me. We could probably do better than we are now with the regular docs too. Though the experience with PEPs does make me a bit nervous about having versioned snapshots of the NEPs in all our old versioned manuals (which have tons of google-juice). -n -- Nathaniel J. Smith -- https://vorpus.org
On Tue, Dec 5, 2017 at 10:44 PM, Nathaniel Smith <njs@pobox.com> wrote:
maybe I have a different google, but the first search result for me for "numpy nep" is https://docs.scipy.org/doc/numpy-dev/neps/index.html developer guides are on top here https://docs.scipy.org/doc/ using the link for "Complete Numpy Manual" on that page has dev docs and neps listed. But I don't see "numpy doc standard" which I usually need to google and the search leads to github page for it. Searching for specific items leads often to older versions in google and needs manual interventions, e.g. edit the version number in link Aside: My main occasion for reading Peps (besides quoting ZEN) is to follow the links in the python what's new pages, e.g. https://docs.python.org/3/whatsnew/3.6.html Josef
On Wed, Dec 6, 2017 at 5:39 PM, <josef.pktd@gmail.com> wrote:
indeed. possibly governance as well.
These have recently been moved to https://numpydoc.readthedocs.io/en/latest/format.html Ralf
Would be great to have structure, and especially a template - ideally, the latter is enough for someone to create a NEP, i.e., has lots of in-template documentation. One thing I'd recommend thinking a little about is to what extend a NEP is "frozen" after acceptance. In astropy we've seen situations where it helps to clarify details later, and it may be good to think beforehand what one wants. In my opinion, one should allow clarifications of accepted NEPs, and major editing of ones still pending (as happened for __[numpy|array]_ufunc__). I think the location is secondary, but for what it is worth, I'm not fond of the astropy APEs being in a separate repository, mostly because I like detailed discussion of everything related in the project to happen in one place on github. Also, having to clone a repository is yet another hurdle for doing stuff. So, I'd suggest to keep the NEPs in the main repository. -- Marten
Great idea -- thanks for pushing this forward all. In the end, you can have the NEPs in a separate repo, and still publish them closely with the main docs (intersphinx is pretty cool), or have them in the same repo and publish them separately. So I say let the folks doing the work decide what workflow works best for them. Comments on a couple other points: I find myself going back to PEPs quite a bit -- mostly to understand the hows an whys of a feature, rather than the how-to-use its. And yes -- we should keep NEPs updated -- they certainly should be edited for typos and minor clarifications, but It's particularly important if the implementation ends up differing a bit from what was expected when the NEP was written. I'm not sure what the PEP policy is about this, but they are certainly maintained with regard to typos and the like. -CHB On Wed, Dec 6, 2017 at 10:43 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (9)
-
Charles R Harris -
Chris Barker -
Jarrod Millman -
josef.pktd＠gmail.com -
Marten van Kerkwijk -
Nathaniel Smith -
Nelle Varoquaux -
Ralf Gommers -
Robert Kern