Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later.
If you are available at different times in different parts of the world, jump in and pick something to work on. Jarrod and/or I will put out a list of priorities in the next few hours.
-Travis O.
On Dec 27, 2007 7:42 PM, Travis E. Oliphant oliphant@enthought.com wrote:
Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later.
If you are available at different times in different parts of the world, jump in and pick something to work on.
Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines
I will be working on making the roadmaps more detailed and better documenting the discussions from the coding sprint.
Travis O. will be mostly working on NumPy docstrings and possibly deprecation warnings for scipy.io functions.
Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.
I hope to see several of you on IRC tomorrow.
Happy holidays,
On Dec 27, 2007 10:27 PM, Jarrod Millman millman@berkeley.edu wrote:
Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines
Care to make the Example section mandatory, instead of optional? I really think it should be mandatory. We may not do a good job of it initially, but at least we should express that it's of critical importance that every function contains at least one small example, whenever feasible.
I also think that the above wiki page should have a minimal, self-contained example of a proper docstring with all 8 sections implemented. I'm honestly not sure at this point what the actual changes to epydoc are (in ipython we use regular epydoc with reST), and I think for many it would be much easier to get started by reading a small example rather than trying to abstract out what the exact markup should be from reading the description and the various documents linked to (doctest, reST, epydoc...).
With such a guiding example, tomorrow people will be able to get up and going quickly...
I will be working on making the roadmaps more detailed and better documenting the discussions from the coding sprint.
Travis O. will be mostly working on NumPy docstrings and possibly deprecation warnings for scipy.io functions.
Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.
I'm afraid I won't be able to participate tomorrow, but one thing to remember is that with nose, any and all doctest examples should be automatically picked up (with the appropriate flags). So a *very easy* way for anyone to contribute is to simply add doctest examples to the codebase. Those serve automatically two purposes: they are small tests for each function, and they make the library vastly easier to use, since any function is just one
foo?
away from an example. As a reminder, those of you using ipython >= 0.8.2 can use this feature:
In [1]: %doctest_mode *** Pasting of code with ">>>" or "..." has been enabled. Exception reporting mode: Plain Doctest mode is: ON
for i in range(10):
... print i, ... 0 1 2 3 4 5 6 7 8 9
for i in range(10):
... ... print i, ... 0 1 2 3 4 5 6 7 8 9
%doctest_mode
Exception reporting mode: Context Doctest mode is: OFF
########
The %doctest_mode magic switches the ipython prompt to >>> so you can continue using ipython but get the proper prompts for making pasteable docstests, and it also allows you to paste input that begins with '>>>' for execution, so you can try your doctests again.
HTH,
f
On Dec 27, 2007 9:41 PM, Fernando Perez fperez.net@gmail.com wrote:
Care to make the Example section mandatory, instead of optional? I really think it should be mandatory. We may not do a good job of it initially, but at least we should express that it's of critical importance that every function contains at least one small example, whenever feasible.
+1
I also think that the above wiki page should have a minimal, self-contained example of a proper docstring with all 8 sections implemented. I'm honestly not sure at this point what the actual changes to epydoc are (in ipython we use regular epydoc with reST), and I think for many it would be much easier to get started by reading a small example rather than trying to abstract out what the exact markup should be from reading the description and the various documents linked to (doctest, reST, epydoc...).
I think we already have that. Take a look at the example at the bottom: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines#example
It points to a plain text example: http://svn.scipy.org/svn/numpy/trunk/numpy/doc/example.py and what it looks like rendered: http://www.scipy.org/doc/example
It also explains how to check the example code out of svn and generate the html using epydoc.
On Dec 27, 2007 10:50 PM, Jarrod Millman millman@berkeley.edu wrote:
I also think that the above wiki page should have a minimal, self-contained example of a proper docstring with all 8 sections implemented. I'm honestly not sure at this point what the actual changes to epydoc are (in ipython we use regular epydoc with reST), and I think for many it would be much easier to get started by reading a small example rather than trying to abstract out what the exact markup should be from reading the description and the various documents linked to (doctest, reST, epydoc...).
I think we already have that. Take a look at the example at the bottom: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines#example
Oops, sorry. I read the page quickly and my brain was scanning for a big block of text in {{{ }}}, so I didn't look at that section carefully enough. My mistake.
Cheers,
f
Since I won't be able to participate tomorrow, here's a small contribution. I just added this to ipython:
http://projects.scipy.org/ipython/ipython/browser/ipython/trunk/IPython/dtut...
You can load it in your startup file or interactively via
from IPython.dtutils import idoctest
Type idoctest? for calling details.
This will enable you to do things like:
In [96]: idoctest -------> idoctest() # at this point, you start typing/pasting your doctest. Hit Ctrl-D or two blanks to end:
>>> for i in range(10): ... print i, ... 0 1 2 3 4 5 6 7 8 9
1 items passed all tests: 1 tests in interactive doctest 1 tests in 1 items. 1 passed and 0 failed. Test passed.
If you have a failing doctest that you'd like to debug, you can use 'eraise=True' and errors will be immediately raised, so you can then call %debug to debug them.
This should come in handy for all those doctests that you're all going to write tomorrow. If you aren't running off SVN ipython, you can simply copy the above file somewhere and load it interactively once ipython is running, it doesn't depend on any other changes to ipython.
It probably still needs a bit of work to make it more convenient, so I'm open to feedback. But it should make life easier when writing frequent doctests.
Cheers,
f
On Thu, Dec 27, 2007 at 09:27:09PM -0800, Jarrod Millman wrote:
On Dec 27, 2007 7:42 PM, Travis E. Oliphant oliphant@enthought.com wrote:
Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later.
If you are available at different times in different parts of the world, jump in and pick something to work on.
Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines
I have some questions regarding Travis' latest modifications to the documentation guidelines:
The following section was removed, why?
""" A reST-documented module should define::
__docformat__ = 'restructuredtext en' at the top level in accordance with `PEP 258 http://www.python.org/dev/peps/pep-0258`__. Note that the ``__docformat__`` variable in a package's ``__init__.py`` file does not apply to objects defined in subpackages and submodules. """
We had a long discussion on the mailing list on the pros and cons of "*Parameters*:" vs. "Parameters:". I see now that it has been changed to
Parameters ----------
Is this still recognised as a list?
I noted that the examples are now no longer indented: does ReST allow this?
Note that building the example documentation, `epydoc example.py`, now warns:
File /tmp/example.py, line 19, in example.foo Warning: Line 24: Wrong underline character for heading. Warning: Lines 27, 30, 32, 37, 39, 41, 43, 48, 50: Improper paragraph indentation.
While I understand the rationale behind
"The guiding principle is that human readers of the text itself are given precedence over contorting the docstring so that epydoc_ produces nice output."
I think that it would be impractical to break compatibility with the only documentation builder we currently have (unless there are others I am unaware of?).
Regards Stéfan
Stefan van der Walt wrote:
On Thu, Dec 27, 2007 at 09:27:09PM -0800, Jarrod Millman wrote:
On Dec 27, 2007 7:42 PM, Travis E. Oliphant oliphant@enthought.com wrote:
Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later.
If you are available at different times in different parts of the world, jump in and pick something to work on.
Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines
I have some questions regarding Travis' latest modifications to the documentation guidelines:
The following section was removed, why?
""" A reST-documented module should define::
__docformat__ = 'restructuredtext en' at the top level in accordance with `PEP 258 http://www.python.org/dev/peps/pep-0258`__. Note that the ``__docformat__`` variable in a package's ``__init__.py`` file does not apply to objects defined in subpackages and submodules. """
I don't see the point in every file having a __docformat__ line, when our documentaion formatting tool should already know the standard we are using is. It's just more cruft. Besides the PEP was rejected, so I don't know why we should be following it.
We obviously need a pre-processor to map our files to epydoc, so let's do that instead of contorting docstrings into a format demanded by the tool. The tool is a "tool" for us to use not be chained by.
We had a long discussion on the mailing list on the pros and cons of "*Parameters*:" vs. "Parameters:". I see now that it has been changed to
Parameters
Is this still recognised as a list?
I noted that the examples are now no longer indented: does ReST allow this?
Note that building the example documentation, `epydoc example.py`, now warns:
File /tmp/example.py, line 19, in example.foo Warning: Line 24: Wrong underline character for heading. Warning: Lines 27, 30, 32, 37, 39, 41, 43, 48, 50: Improper paragraph indentation.
While I understand the rationale behind
"The guiding principle is that human readers of the text itself are given precedence over contorting the docstring so that epydoc_ produces nice output."
I think that it would be impractical to break compatibility with the only documentation builder we currently have (unless there are others I am unaware of?).
In my mind, the rationale trumps the "impracticality" especially since epydoc still produces readable output. It is only the formatting that is not pretty-printed (although it doesn't look bad there either). So, AFAIK, nothing is actually broken (except a little HTML formatting on output of epydoc). But, now the docstrings don't look crufty to my eyes.
Basically, the problem is that I *really* don't like the *Parameters*: thing and all the indentation that takes place inside of docstrings to satisfy epydoc. It is also inconsistent with the docstring standard that Enthought uses for the ETS tools suite. It was really hard for me to start writing docstrings that followed that pattern.
It seems better to have docstrings than to have them fit into an epydoc_-defined pattern.
I'm sorry if I made changes unilaterally. I didn't think there would be much concern as long as docstrings were moving forward.
-Travis
On Fri, Dec 28, 2007 at 11:32:03AM -0600, Travis E. Oliphant wrote:
I don't see the point in every file having a __docformat__ line, when our documentaion formatting tool should already know the standard we are using is. It's just more cruft. Besides the PEP was rejected, so I don't know why we should be following it.
Sorry, I didn't see the PEP was rejected. Moot point, then.
We obviously need a pre-processor to map our files to epydoc, so let's do that instead of contorting docstrings into a format demanded by the tool.
That's a good idea, thanks. I'll take a look.
It seems better to have docstrings than to have them fit into an epydoc_-defined pattern.
Certainly.
Thanks for the feedback.
Regards Stéfan
On Dec 28, 2007 5:26 AM, Stefan van der Walt stefan@sun.ac.za wrote:
On Thu, Dec 27, 2007 at 09:27:09PM -0800, Jarrod Millman wrote:
On Dec 27, 2007 7:42 PM, Travis E. Oliphant oliphant@enthought.com
wrote:
Doc-day will start tomorrow (in about 12 hours). It will be Friday
for
much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC
from
about 9:30 am CST to 6:00 pm CST and then possibly later.
If you are available at different times in different parts of the world, jump in and pick something to work on.
Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines
I have some questions regarding Travis' latest modifications to the documentation guidelines:
The following section was removed, why?
""" A reST-documented module should define::
__docformat__ = 'restructuredtext en'
at the top level in accordance with `PEP 258 http://www.python.org/dev/peps/pep-0258`__. Note that the ``__docformat__`` variable in a package's ``__init__.py`` file does not apply to objects defined in subpackages and submodules. """
We had a long discussion on the mailing list on the pros and cons of "*Parameters*:" vs. "Parameters:". I see now that it has been changed to
Parameters
If we are going to avoid ReST and html definition lists, i.e.
Parameters:
then we should probably avoid ReST and headings
Parameters ----------
because that will likely be moved about. In fact, we should probably throw ReST overboard if we are going to avoid ReST'isms. I personally think the underline is ugly (hi Travis), but I suppose that's beside the point. If we stay with ReST it would be nice to retain the links in the see also section.
I'm not at home at the moment, so I don't know how the new style is rendered by epydoc, or even if we are still going to use epydoc. If not, we should definitely decide on the structure of the docstrings and stick to it. With a bit more pain in the appearence, we could also format for doxygen, which is pretty standard and would also work for the C code. I suspect that's not going to fly, though ;)
Chuck
On Dec 29, 2007 6:51 PM, Charles R Harris charlesr.harris@gmail.com wrote:
If not, we should definitely decide on the structure of the docstrings and stick to it.
+100
I'm about to commit docstrings for scimath (what I started yesterday). After some fixes to the numpy build, I can now work in-place and get things like
tlon[~]> nosetests --with-doctest numpy.lib.scimath ....... ---------------------------------------------------------------------- Ran 7 tests in 0.660s
OK
Which is nice. For now I'm adhering to what was in the guide:
log?
Type: function Base Class: <type 'function'> Namespace: Interactive File: /home/installers/src/scipy/numpy/numpy/lib/scimath.py Definition: log(x) Docstring: Return the natural logarithm of x.
If x contains negative inputs, the answer is computed and returned in the complex domain.
Parameters ---------- x : array_like
Returns ------- array_like
Examples -------- >>> import math
>>> log(math.exp(1)) 1.0
Negative arguments are correctly handled (recall that for negative arguments, the identity exp(log(z))==z does not hold anymore):
>>> log(-math.exp(1)) == (1+1j*math.pi) True
But it's easy enough to fix them if a change is made. But let's settle this *soon* please, so that if changes need to be made it's just a few, and we can just move on.
At this point we just need *a* decision. It doesn't need to be perfect, but any docstring is better than our canonical:
np.cumsum?
Type: function Base Class: <type 'function'> Namespace: Interactive File: /home/fperez/usr/opt/lib/python2.5/site-packages/numpy/core/fromnumeric.py Definition: np.cumsum(a, axis=None, dtype=None, out=None) Docstring: Sum the array over the given axis.
Blah, Blah.
I got that 'Blah blah' while lecturing to 20 developers and scientists at a workshop at NCAR (the National Center for Atmospheric Research in Boulder), projected on a 20 foot-wide screen. A bit embarrassing, I must admit.
So please, no more code without full docstrings in numpy/scipy....
Cheers,
f
On Dec 29, 2007 7:59 PM, Fernando Perez fperez.net@gmail.com wrote:
On Dec 29, 2007 6:51 PM, Charles R Harris charlesr.harris@gmail.com wrote:
If not, we should definitely decide on the structure of the docstrings and stick to it.
+100
I have raised the topic of documentation formats on the list several times, and the discussions have always petered out. So I made a decision, posted what *worked* with epydoc, and also generated a significant amount of documentation. Now that is tossed out with hardly a mention. I would like to propose that anyone who submits a new documentation standard also submit code to parse the result, compare it to the existing practice, discuss why it is an improvement, and generate documentation to show how it works. I did all of those things, I expect nothing less from others if we are going to reinvent the wheel. And the end result should be available as soon the the formatting changes are made, not at some indefinite point in the future.
Chuck
Charles R Harris wrote:
On Dec 29, 2007 7:59 PM, Fernando Perez <fperez.net@gmail.com mailto:fperez.net@gmail.com> wrote:
On Dec 29, 2007 6:51 PM, Charles R Harris <charlesr.harris@gmail.com <mailto:charlesr.harris@gmail.com>> wrote: > If not, we should > definitely decide on the structure of the docstrings and stick to it. +100I have raised the topic of documentation formats on the list several times, and the discussions have always petered out. So I made a decision, posted what *worked* with epydoc, and also generated a significant amount of documentation. Now that is tossed out with hardly a mention. I would like to propose that anyone who submits a new documentation standard also submit code to parse the result, compare it to the existing practice, discuss why it is an improvement, and generate documentation to show how it works. I did all of those things, I expect nothing less from others if we are going to reinvent the wheel. And the end result should be available as soon the the formatting changes are made, not at some indefinite point in the future.
Hey Chuck,
I'm the guilty party in the docstring revisions, and I'm very sorry that my actions seemed to undermine your contributions (which are very much appreciated).
The changes to the docstring format that I pushed are not much different from what was put in place. However, they make it so that the documentation standard for SciPy and NumPy is not different from the ETS standard. In addition, I think it conserves horizontal real-estate and looks better when just printed as plain text (which is mostly how docstrings get read).
I ran epydoc (on the example.py file) and it still produces output that is very readable even if it is not currently as good looking as the previously rendered result. A relatively simple pre-processer should be able to produce fancier output with epydoc.
As we were going to have a Doc-Day, and several people were going to be adding documentation, I wanted to cement the docstring standard and I did not want it to be based on a "technological" reason (i.e. we have all this stuff in the docstring so that epydoc can produce pretty output). My experience is that >90% of docstring-reading is done in plain-text, so this should drive the decision, and not the availability of a tool to render it.
We had to cement a decision, so I did it. I am very sorry for stepping on your toes. I should have looked at who committed the most recent changes to the documentation information pages and taken more time to work things out with you privately (but it was very early in the morning on Friday (2-4 am ish). I also did not want to have another mailing-list discussion on docstring formats because as you've witnessed, it is not usually helpful on something which is primarily in the eye of the beholder.
I'll take responsibility to get something in place to render the docstrings more beautifully. If anybody is interested in helping with that, please let me know.
Sheepishly yours,
-Travis O.
On Sat, 29 Dec 2007, "Travis E. Oliphant" apparently wrote:
they make it so that the documentation standard for SciPy and NumPy is not different from the ETS standard
1. Is the Enthought Tool Suite standard published?
2. I think it miscasts things to say that the previous standard was built around a rendering tool. The previous standard was tied to reStructuredText, which happens to be renderable by epydoc. In the future, other tools will render reStructuredText. Indeed, we are all free to offer one right now (or to contribute to epydoc).
3. The point of the reStructuredText, which has been developed over many years, is to make (with very light markup) a number of semantic distinctions which have proved useful and desirable. If these distinctions are not drawn in the markup, they cannot be used by any renderer.
4. reST offers way to render math (as MathML or LaTeX->PDF). At one point, iirc, this was considered a plus.
I am NOT trying to push for any particular documentation standard. I'm just a NumPy user, not a NumPy developer. But I would like to suggest that it might be useful to ask David Goodger for a quick overview of what will be lost by using the ETS markup standard instead of reST.
fwiw, Alan Isaac
Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.
I must have missed Fernando's email because I can't find the references for nose :( What are its advantages against the current numpy.testing framework ?
Matthieu
Matthieu Brucher wrote:
Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.I must have missed Fernando's email because I can't find the references for nose :(
Look in "SciPy Sprint Results". It's only a brief mention, though.
What are its advantages against the current numpy.testing framework ?
Primarily:
* It is supported by someone else and gaining wide adoption by the rest of the Python community.
Secondarily:
* More flexible organization of tests. For nose, if it looks like a test, it's a test. numpy.testing collects test modules which are named like the module it is testing. E.g. for module.py <=> tests/test_module.py.
* Test generators:
def test_evens(): for i in range(0, 5): yield check_even, i, i*3
def check_even(n, nn): assert n % 2 == 0 or nn % 2 == 0
* Package- and module-level setup() and teardown() functions.
* Test functions can be simple functions. They do not need to be organized into classes if you don't need classes.
* Integrated doctest collection.
* Detailed error/failure reporting. nose can print out the values of variables at the location of the error.
* Integrated code coverage and profiling.
* Dropping into pdb on errors and failures.
* More flexible running of specific tests. E.g. when I'm working on getting a particular test function running, I can specify that exact test and not run the rest of the test suite.
* Output capture. Tests can print out anything they like to be more informative, but they won't appear unless if the test fails.
More thoroughly:
http://somethingaboutorange.com/mrl/projects/nose/
On Dec 28, 2007 12:55 PM, Robert Kern robert.kern@gmail.com wrote:
Matthieu Brucher wrote:
Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.I must have missed Fernando's email because I can't find the references for nose :(
Look in "SciPy Sprint Results". It's only a brief mention, though.
What are its advantages against the current numpy.testing framework ?
Primarily:
[...]
It's worth noting that there's already a preliminary commit of this stuff here:
http://projects.scipy.org/scipy/scipy/browser/branches/testing_cleanup/scipy...
For now we put it in exmplpackage just so Matthew and I could quickly coordinate, but I think he's working on propagating that throughout more properly across the whole codebase.
But that directory already has a small readme (to which your email could be added) as well as a little example, the test_foo file, that showcases the various features.
Cheers,
f
participants (8)
-
Alan G Isaac -
Charles R Harris -
Fernando Perez -
Jarrod Millman -
Matthieu Brucher -
Robert Kern -
Stefan van der Walt -
Travis E. Oliphant