On Dec 27, 2007 10:27 PM, Jarrod Millman wrote:

Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines

Care to make the Example section mandatory, instead of optional? I really think it should be mandatory. We may not do a good job of it initially, but at least we should express that it's of critical importance that every function contains at least one small example, whenever feasible. I also think that the above wiki page should have a minimal, self-contained example of a proper docstring with all 8 sections implemented. I'm honestly not sure at this point what the actual changes to epydoc are (in ipython we use regular epydoc with reST), and I think for many it would be much easier to get started by reading a small example rather than trying to abstract out what the exact markup should be from reading the description and the various documents linked to (doctest, reST, epydoc...). With such a guiding example, tomorrow people will be able to get up and going quickly...

I will be working on making the roadmaps more detailed and better documenting the discussions from the coding sprint.

Travis O. will be mostly working on NumPy docstrings and possibly deprecation warnings for scipy.io functions.

Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.

I'm afraid I won't be able to participate tomorrow, but one thing to remember is that with nose, any and all doctest examples should be automatically picked up (with the appropriate flags). So a *very easy* way for anyone to contribute is to simply add doctest examples to the codebase. Those serve automatically two purposes: they are small tests for each function, and they make the library vastly easier to use, since any function is just one foo? away from an example. As a reminder, those of you using ipython >= 0.8.2 can use this feature: In [1]: %doctest_mode *** Pasting of code with ">>>" or "..." has been enabled. Exception reporting mode: Plain Doctest mode is: ON

...

...

for i in range(10): ... print i, ... 0 1 2 3 4 5 6 7 8 9

...

...

...

...

...

for i in range(10): ... ... print i, ... 0 1 2 3 4 5 6 7 8 9

...

...

%doctest_mode Exception reporting mode: Context Doctest mode is: OFF

######## The %doctest_mode magic switches the ipython prompt to >>> so you can continue using ipython but get the proper prompts for making pasteable docstests, and it also allows you to paste input that begins with '>>>' for execution, so you can try your doctests again. HTH, f

On Dec 27, 2007 10:50 PM, Jarrod Millman wrote:

...

I also think that the above wiki page should have a minimal, self-contained example of a proper docstring with all 8 sections implemented. I'm honestly not sure at this point what the actual changes to epydoc are (in ipython we use regular epydoc with reST), and I think for many it would be much easier to get started by reading a small example rather than trying to abstract out what the exact markup should be from reading the description and the various documents linked to (doctest, reST, epydoc...).

I think we already have that. Take a look at the example at the bottom: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines#example

Oops, sorry. I read the page quickly and my brain was scanning for a big block of text in {{{ }}}, so I didn't look at that section carefully enough. My mistake. Cheers, f

On Thu, Dec 27, 2007 at 09:27:09PM -0800, Jarrod Millman wrote:

On Dec 27, 2007 7:42 PM, Travis E. Oliphant wrote:

...

Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later.

If you are available at different times in different parts of the world, jump in and pick something to work on.

Since this is our first doc-day, it will be fairly informal. Travis is going to be trying to get some estimate of which packages need the most work. But if there is some area of NumPy or SciPy you are familiar with, please go ahead and pitch in. Here is the current NumPy/ SciPy coding standard including docstring standards: http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines

I have some questions regarding Travis' latest modifications to the documentation guidelines: The following section was removed, why? """ A reST-documented module should define:: __docformat__ = 'restructuredtext en' at the top level in accordance with `PEP 258 http://www.python.org/dev/peps/pep-0258`__. Note that the ``__docformat__`` variable in a package's ``__init__.py`` file does not apply to objects defined in subpackages and submodules. """ We had a long discussion on the mailing list on the pros and cons of "*Parameters*:" vs. "Parameters:". I see now that it has been changed to Parameters ---------- Is this still recognised as a list? I noted that the examples are now no longer indented: does ReST allow this? Note that building the example documentation, `epydoc example.py`, now warns: File /tmp/example.py, line 19, in example.foo Warning: Line 24: Wrong underline character for heading. Warning: Lines 27, 30, 32, 37, 39, 41, 43, 48, 50: Improper paragraph indentation. While I understand the rationale behind "The guiding principle is that human readers of the text itself are given precedence over contorting the docstring so that epydoc_ produces nice output." I think that it would be impractical to break compatibility with the only documentation builder we currently have (unless there are others I am unaware of?). Regards Stéfan

On Fri, Dec 28, 2007 at 11:32:03AM -0600, Travis E. Oliphant wrote:

I don't see the point in every file having a __docformat__ line, when our documentaion formatting tool should already know the standard we are using is. It's just more cruft. Besides the PEP was rejected, so I don't know why we should be following it.

Sorry, I didn't see the PEP was rejected. Moot point, then.

We obviously need a pre-processor to map our files to epydoc, so let's do that instead of contorting docstrings into a format demanded by the tool.

That's a good idea, thanks. I'll take a look.

It seems better to have docstrings than to have them fit into an epydoc_-defined pattern.

Certainly. Thanks for the feedback. Regards Stéfan

On Dec 29, 2007 6:51 PM, Charles R Harris wrote:

If not, we should definitely decide on the structure of the docstrings and stick to it.

+100 I'm about to commit docstrings for scimath (what I started yesterday). After some fixes to the numpy build, I can now work in-place and get things like tlon[~]> nosetests --with-doctest numpy.lib.scimath ....... ---------------------------------------------------------------------- Ran 7 tests in 0.660s OK Which is nice. For now I'm adhering to what was in the guide:

...

...

log? Type: function Base Class: Namespace: Interactive File: /home/installers/src/scipy/numpy/numpy/lib/scimath.py Definition: log(x) Docstring: Return the natural logarithm of x.

If x contains negative inputs, the answer is computed and returned in the complex domain. Parameters ---------- x : array_like Returns ------- array_like Examples -------- >>> import math >>> log(math.exp(1)) 1.0 Negative arguments are correctly handled (recall that for negative arguments, the identity exp(log(z))==z does not hold anymore): >>> log(-math.exp(1)) == (1+1j*math.pi) True But it's easy enough to fix them if a change is made. But let's settle this *soon* please, so that if changes need to be made it's just a few, and we can just move on. At this point we just need *a* decision. It doesn't need to be perfect, but any docstring is better than our canonical:

...

...

np.cumsum? Type: function Base Class: Namespace: Interactive File: /home/fperez/usr/opt/lib/python2.5/site-packages/numpy/core/fromnumeric.py Definition: np.cumsum(a, axis=None, dtype=None, out=None) Docstring: Sum the array over the given axis.

Blah, Blah. I got that 'Blah blah' while lecturing to 20 developers and scientists at a workshop at NCAR (the National Center for Atmospheric Research in Boulder), projected on a 20 foot-wide screen. A bit embarrassing, I must admit. So please, no more code without full docstrings in numpy/scipy.... Cheers, f

Charles R Harris wrote:

On Dec 29, 2007 7:59 PM, Fernando Perez mailto:fperez.net@gmail.com> wrote:

On Dec 29, 2007 6:51 PM, Charles R Harris mailto:charlesr.harris@gmail.com> wrote:

> If not, we should > definitely decide on the structure of the docstrings and stick to it.

+100

I have raised the topic of documentation formats on the list several times, and the discussions have always petered out. So I made a decision, posted what *worked* with epydoc, and also generated a significant amount of documentation. Now that is tossed out with hardly a mention. I would like to propose that anyone who submits a new documentation standard also submit code to parse the result, compare it to the existing practice, discuss why it is an improvement, and generate documentation to show how it works. I did all of those things, I expect nothing less from others if we are going to reinvent the wheel. And the end result should be available as soon the the formatting changes are made, not at some indefinite point in the future.

Hey Chuck, I'm the guilty party in the docstring revisions, and I'm very sorry that my actions seemed to undermine your contributions (which are very much appreciated). The changes to the docstring format that I pushed are not much different from what was put in place. However, they make it so that the documentation standard for SciPy and NumPy is not different from the ETS standard. In addition, I think it conserves horizontal real-estate and looks better when just printed as plain text (which is mostly how docstrings get read). I ran epydoc (on the example.py file) and it still produces output that is very readable even if it is not currently as good looking as the previously rendered result. A relatively simple pre-processer should be able to produce fancier output with epydoc. As we were going to have a Doc-Day, and several people were going to be adding documentation, I wanted to cement the docstring standard and I did not want it to be based on a "technological" reason (i.e. we have all this stuff in the docstring so that epydoc can produce pretty output). My experience is that >90% of docstring-reading is done in plain-text, so this should drive the decision, and not the availability of a tool to render it. We had to cement a decision, so I did it. I am very sorry for stepping on your toes. I should have looked at who committed the most recent changes to the documentation information pages and taken more time to work things out with you privately (but it was very early in the morning on Friday (2-4 am ish). I also did not want to have another mailing-list discussion on docstring formats because as you've witnessed, it is not usually helpful on something which is primarily in the eye of the beholder. I'll take responsibility to get something in place to render the docstrings more beautifully. If anybody is interested in helping with that, please let me know. Sheepishly yours, -Travis O.

Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.

I must have missed Fernando's email because I can't find the references for nose :( What are its advantages against the current numpy.testing framework ? Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher

On Dec 28, 2007 12:55 PM, Robert Kern wrote:

Matthieu Brucher wrote:

...

Matthew B. will be working on converting SciPy tests to use nose per Fernando's email. If you are familiar with nose and want to help, please make sure to check with Matthew or Fernando first.

I must have missed Fernando's email because I can't find the references for nose :(

Look in "SciPy Sprint Results". It's only a brief mention, though.

...

What are its advantages against the current numpy.testing framework ?

Primarily:

[...] It's worth noting that there's already a preliminary commit of this stuff here: http://projects.scipy.org/scipy/scipy/browser/branches/testing_cleanup/scipy... For now we put it in exmplpackage just so Matthew and I could quickly coordinate, but I think he's working on propagating that throughout more properly across the whole codebase. But that directory already has a small readme (to which your email could be added) as well as a little example, the test_foo file, that showcases the various features. Cheers, f