Hi Scipy-dev, I'm forwarding this from William Stein, the lead developer of SAGE (http://modular.math.washington.edu/sage/). William, I won't have any email access until next Monday, but hopefully others may pitch in. It's probably also worth mentioning these two pages: http://scipy.org/Numpy_Example_List_With_Doc http://www.hjcb.nl/python/Arrays.html Their contents may serve as the starting point for material for docstring examples (this has been suggested several times before, it just hasn't happened). Cheers, f ---------- Forwarded message ---------- From: William Stein <wstein@gmail.com> Date: Dec 5, 2006 11:41 PM Subject: [sage-devel] numpy in SAGE, etc. To: "sage-devel@googlegroups.com" <sage-devel@googlegroups.com> Cc: "Fernando.Perez@colorado.edu" <Fernando.Perez@colorado.edu>, oliphant@ee.byu.edu, pearu@cens.ioc.ee Hello, In case you don't know, SAGE-1.5 (http://sage.math.washington.edu/sage) will included numpy by default. Inclusion of the scipy distribution might not be far off either. We will also definitely continue to include GSL in SAGE and develop its unique functionality. I want SAGE to develop into a truly viable alternative to MATLAB (in addition to everything else it is), and it's clear to me that numpy/scipy/vtk/mayavi/gsl are crucial pieces of software if there is any hope of succcess. Anyway, to the first point of this email. I want to try some functions in numpy, so I type numpy.[tab], then say numpy.array? and I see some _minimal_ documentation but ABSOLUTELY NO EXAMPLES. The same is true for tons of the functions in numpy (and Numeric), and even Python for that matter. Anyway, this is simply *not* up to snuff for what is needed for SAGE. For SAGE my goal is that every mathematical function in the system is illustrated by examples that the user can paste into the interpreter and have work (and moreover, they are autotested). Currently there are about 12000 lines of such input already. I think this is unrelated to the whole issue with the official numpy documentation being commercial. Given the extremely limited number of examples in Numeric, numpy, and the official Python docs, it must be a conscience design decision to *not* have lots and lots of doctests. In SAGE, often files have way more docs and doctes than actual code -- again this is a design decision. The question, then, it was to do if numpy is to be included in SAGE in a way that satisfies our design goals? Some options include: (1) The file numpy/add_newdocs.py in the numpy distribution defines somehow docstrings for a lot of the numpy constructors. A SAGE developer could simply add tons of examples to this file, based on playing around, and reading the numpy book to learn what is relevant to illustrate. As each numpy distribution is released, we would *merge* this file with the one in the new numpy distribution (e.g., very easily using Mercurial). (2) You might think it would be possible to change the docstrings at runtime, but I think they may be hardcoded in (many are for code defined in extension classes). OK, so I don't have many options. Thoughts? Does anybody want to help? Any person who wants to learn numpy could probably easily write these examples along the way. Instead of just learning numpy, you could more systematically learn numpy and at the same time contribute tons of useful doctests. And finally, am I just wrong -- would Travis, etc., want these docstrings with tons of examples? Travis -- since I cc'd you, maybe you can just answer. I can completely understand if you don't want tons of doctests; it's fine if your design goals are different. By the way, SAGE Days 3 is in LA at IPAM Feb 17-21, and I hope both Fernando and Travis will consider coming. Some travel funding is available. By the way, most of the remarks above also apply to Networkx -- it's docs seem to have almost no examples. Actually, I don't think I know of *any* Python packages that do have much in the way of examples in the docstrings, at least nothing on the order of SAGE. ------------------------- Here's the official statement about the scipy module documentation, in the DEVELOPERS.txt file of the scipy distribution: "Currently there are * A SciPy tutorial by Travis E. Oliphant. This is maintained using LyX. The main advantage of this approach is that one can use mathematical formulas in documentation. * I (Pearu) have used reStructuredText formated .txt files to document various bits of software. This is mainly because ``docutils`` might become a standard tool to document Python modules. The disadvantage is that it does not support mathematical formulas (though, we might add this feature ourself using e.g. LaTeX syntax). * Various text files with almost no formatting and mostly badly out dated. * Documentation strings of Python functions, classes, and modules. Some SciPy modules are well-documented in this sense, others are very poorly documented. Another issue is that there is no consensus on how to format documentation strings, mainly because we haven't decided which tool to use to generate, for instance, HTML pages of documentation strings." -------- So evidently the scipy people have for some reason not even decided on a format for their documentation, which is partly why they don't have it. For the record, in SAGE there is a very precise documentation format that is systematically and uniformly applied throughout the system: * We liberally use latex in the docstrings. I wrote a Python function that preparses this to make it human readable, when one types foo? * The format of each docstring is as follows: function header """ 1-2 sentences summarizing what the function does. INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple) EXAMPLES: a *bunch* of examples, often a whole page. NOTES: misc other notes ALGORITHM: notes about the implementation or algorithm, if applicable AUTHORS: -- name (date): notes about what was done """ The INPUT and OUTPUT blocks are typeset as a latex verbatim environment. The rest is typeset using normal latex. It's good to use the itemize environment when necessary for lists, also. Essentially all the documtation in SAGE has exactly this format. VERY often I rewrite documentation for code people send me so that it is formated as above. (So if you send me code, please format it exactly as above!!!) The SAGE reference manual is autogenerated from the source code using a script I wrote that basically extracts the documentation (mainly using Python introspection), and puts it in the same format as the standard Python documentation suite, then runs the standard Python documentation tools on it. Note, however, that the structure of the reference manual is laid out by hand (i.e., order of chapters, some heading text, etc.), which is I think very important. I suggest scipy consider a similar strategy. Hey, I just got: "Successfully installed scipy-2006-12-05 Now cleaning up tmp files." on my MacBookPro, after installing the new Intel fortran compiler. -- William --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to sage-devel-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---
I suspect William is right about the lack of a standard format hindering the documentation. I hope something like his format can be adopted. I'd also specify some way of delineating examples so that they can be preprocessed and run as part of the unit tests. SAGE uses matplotlib and it's currently a de facto standard. Could it be adopted in examples somehow, with any preprocessor perhaps skipping examples which contained visual examples or modifying them to suppress plot windows appearing? Maybe a standard way to provide references to example code outside a docstring should also be defined so that examples covering multiple functions can be referenced from several places. I suspect this sort of thing would require breaking away from the standard docstring idea. As I've mentioned previously, this would also allow doing things like running the example code interactively. Another thing that William's email doesn't cover is non-function documentation; it might be worth defining a similar documentation format for numpy classes and types. Finally, should the NOTES: section be used for "see also" tags or should a SEE ALSO: section be added? Gary R. <snip>
(2) You might think it would be possible to change the docstrings at runtime, but I think they may be hardcoded in (many are for code defined in extension classes).
OK, so I don't have many options. Thoughts? Does anybody want to help? Any person who wants to learn numpy could probably easily write these examples along the way. Instead of just learning numpy, you could more systematically learn numpy and at the same time contribute tons of useful doctests.
<snip>
So evidently the scipy people have for some reason not even decided on a format for their documentation, which is partly why they don't have it. For the record, in SAGE there is a very precise documentation format that is systematically and uniformly applied throughout the system:
* We liberally use latex in the docstrings. I wrote a Python function that preparses this to make it human readable, when one types foo?
* The format of each docstring is as follows: function header """ 1-2 sentences summarizing what the function does.
INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple)
EXAMPLES: a *bunch* of examples, often a whole page.
NOTES: misc other notes
ALGORITHM: notes about the implementation or algorithm, if applicable
AUTHORS: -- name (date): notes about what was done """
The INPUT and OUTPUT blocks are typeset as a latex verbatim environment. The rest is typeset using normal latex. It's good to use the itemize environment when necessary for lists, also.
On Dec 7, 2006, at 8:19 AM, Gary Ruben wrote:
I suspect William is right about the lack of a standard format hindering the documentation. I hope something like his format can be adopted.
I'm all for a standard format (and certainly all for encouraging documentation (although I'm a bit skeptical that agreeing on a format is suddenly going to open the floodgates of heretofore amorphous documentation that the SciPy community has been hoarding to themselves)). It's a shame that he felt the need to invent YADMUS (Yet Another Docstring MarkUp Specification), though. Python has too many systems for this already.
I'd also specify some way of delineating examples so that they can be preprocessed and run as part of the unit tests.
You mean like doctest?
Even though I've recently suggested the following idea on this mailing list and got rather pessimistic response, I'll try it once again. I believe it's relatively easy to create a web site with one page per docstringable entity in the sources (function, class, module …). This page can be either freely editable by the readers in a wiki style or can have the option to add comments at the bottom (which will be once in a while integrated into the main text by an editor, MySQL online docs work this way). The content of this page will be injected (preferably by an automatic script) as the doctring into the appropriate function before releasing a new build of the package (this last stage is the only non trivial one). One argument against wiki style documentation was that it tends to get messy. I believe that if the system suggests (rather than imposes) some structure, the mess could be avoided. For example the basic structure of page per entity + some standard template for the page (like the one suggested by William) would be sufficient to maintain a reasonable degree of order.
On Thu, 7 Dec 2006, Mark Koudritsky apparently wrote:
I believe it's relatively easy to create a web site with one page per docstringable entity in the sources (function, class, module …).
Sorry if this is orthogonal to your interests, but just wanted to make sure you knew about http://www.scipy.org/Numpy_Example_List_With_Doc Cheers, Alan Isaac
On 12/7/06, Alan G Isaac <aisaac@american.edu> wrote:
Sorry if this is orthogonal to your interests, but just wanted to make sure you knew about http://www.scipy.org/Numpy_Example_List_With_Doc
I've seen it only briefly before but now when you pointed out, I realized that it covers most of what I talked about in my previous message, thanks. Should we maybe add a paragraph on the main documentation page instructing people that "Numpy Example List" is the place to contribute functions documentation? Many people don't know about it (I didn't) Maybe it should be even renamed to something like "Numpy functions reference" ?
I'd also specify some way of delineating examples so that they can be preprocessed and run as part of the unit tests.
You mean like doctest? Ha, thank you for this information, I always wondered if this was possible.
Is there a preferred way to do that for unit testing ? David
On Dec 8, 2006, at 3:27 AM, David Cournapeau wrote:
I'd also specify some way of delineating examples so that they can be preprocessed and run as part of the unit tests.
You mean like doctest? Ha, thank you for this information, I always wondered if this was possible.
Possible and really desirable. We've pretty much gotten rid of all our other unittests. doctests (including our full example scripts) cover all the things our old unittest suites did, and a great deal more, and they're a lot easier to write, read, and maintain.
Is there a preferred way to do that for unit testing ?
I'm not sure exactly what you mean by this. If you mean, how do you go about integrating doctest with unittest, then the work's already done for you. doctest.DocTestSuite(module) returns a TestSuite suitable for unittest. FiPy adds a bunch of infrastructure to delay the actual imports when we test our entire codebase. I'll need to go back and see if this is still necessary, but we found that import failures in one or another non-essential part of FiPy would cause the entire test suite to fail to build, rather than just reporting test failures on that one bit. We may not need this, anymore, though, because those failed imports were biting us in other circumstances, so we've moved the imports of optional modules into the functions where they're actually invoked, instead of at the head of their modules. Regardless, we're happy to share any elements of our test harness that seem useful to SciPy.
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
doctests (including our full example scripts) cover all the things our old unittest suites did, and a great deal more, and they're a lot easier to write, read, and maintain.
Would you agree with the assessment here (at the bottom): http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305292 ? Thanks, Alan Isaac
On Dec 8, 2006, at 10:32 AM, Alan G Isaac wrote:
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
doctests (including our full example scripts) cover all the things our old unittest suites did, and a great deal more, and they're a lot easier to write, read, and maintain.
Would you agree with the assessment here (at the bottom): http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305292 ?
By and large, yes. I'm aware of the arguments that unit tests should be separate from code, but I'm not really persuaded by that in practice. I think it's much more likely that separate tests will be completely out of date and irrelevant (whether or not they pass). As far as
A potential problem with doctest is that you may have so many tests that your docstrings would hinder rather than help understanding of your code we do have a few cases of very pedantic testing that don't serve much use as documentation. For those, we put them in hidden _test* functions at the end of the code. They don't appear in our documentation, but they do get exercised. Putting them in a separate file would be OK, too.
Hi Jonathan,
I'm all for a standard format (and certainly all for encouraging documentation (although I'm a bit skeptical that agreeing on a format is suddenly going to open the floodgates of heretofore amorphous documentation that the SciPy community has been hoarding to themselves)).
It's a shame that he felt the need to invent YADMUS (Yet Another Docstring MarkUp Specification), though. Python has too many systems for this already.
Can you suggest something existing for the extra markup? I've never used anything fancier than plain docstrings. The other existing options seem to be (from a brief look) reST/docutils epydoc pythondoc doxygen happydoc Some (all?) of these specify their own markup. Here's a summary: <http://epydoc.sourceforge.net/relatedprojects.html> I'm told by a doxygen advocate that it now has proper Python support and should be considered. If someone else has looked at these maybe they can pipe in, otherwise I may have to find out the relative advantages and disadvantages. I like the idea of supporting LaTeX markup, cross references. I like the idea of supporting matplotlib examples (opinions?). I like Mark's idea of a wiki page per function/class/module if it can be done. I also don't mind William's YADMUS.
I'd also specify some way of delineating examples so that they can be preprocessed and run as part of the unit tests.
You mean like doctest?
The problem I see with plain doctests is that you can't easily choose to run, say, just the 3rd example out of 5 without specifying some sort of delineation between them, but this would be easy; something like: # Ex1: or Ex2 etc. between each doctest. Gary R.
On Sat, 09 Dec 2006, Gary Ruben apparently wrote:
Can you suggest something existing for the extra markup? I've never used anything fancier than plain docstrings. The other existing options seem to be (from a brief look) reST/docutils epydoc pythondoc doxygen happydoc
epydoc has very good support for reST, which is a wonderful and powerful markup. Alan Isaac PS reST has writer in the sandbox that supports LaTeX as a text role! (Writes to LaTeX or nicely to XHTML+MathML.) When epydoc supports that, we'll really be in good shape!
On Dec 8, 2006, at 8:11 PM, Alan G Isaac wrote:
epydoc has very good support for reST, which is a wonderful and powerful markup.
This is what we do, and basically like it. I've hacked epydoc 2.1 rather severely, mostly to separate markup from style (and mostly for LaTeX (I like what epydoc *does*, but I don't like how its output looks)). I keep meaning to clean up my changes and send them back to Ed Loper. Now that he's got epydoc 3 in progress, I think I'll just integrate my changes with that and then send it to him.
PS reST has writer in the sandbox that supports LaTeX as a text role! (Writes to LaTeX or nicely to XHTML+MathML.)
Do you know if any of that actually works? The last time I looked at it, I couldn't figure out what to do with it. I think it would be great, though.
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
Do you know if any of that actually works? The last time I looked at it, I couldn't figure out what to do with it.
*If* it works? Yes: it works great. I use it all the time. Use it like any other writer, but use file extension .xml or FireFox won't recognize it. (And of course IE has always been completely braindead for XML, although perhaps IE 7 finally will parse XHTML+MathML? I have not tried.) *How* it works? Sorry. I did not contribute any of the Python code. If I missed your question, ask more. Cheers, Alan Isaac PS Hint: for LaTeX output, make raw-latex the default text role, and it all looks beautiful!
On Dec 9, 2006, at 12:31 AM, Alan G Isaac wrote:
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
Do you know if any of that actually works? The last time I looked at it, I couldn't figure out what to do with it.
*If* it works? Yes: it works great. I use it all the time.
Hint: for LaTeX output, make raw-latex the default text role, and it all looks beautiful!
Thanks, I'll have to experiment. LaTeX output is what we're interested in, although we'd probably do more with the web if we could show math.
On Dec 9, 2006, at 12:31 AM, Alan G Isaac wrote:
Hint: for LaTeX output, make raw-latex the default text role, and it all looks beautiful!
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
Thanks, I'll have to experiment. LaTeX output is what we're interested in
That should have been: make **latex-math** the default role. Like this:: .. default-role:: latex-math Cheers, Alan Isaac
On Dec 9, 2006, at 12:31 AM, Alan G Isaac wrote:
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
Do you know if any of that actually works? The last time I looked at it, I couldn't figure out what to do with it.
*If* it works? Yes: it works great.
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details. Cheers, Alan Isaac
On Sat, 2006-12-09 at 21:52 -0500, Alan G Isaac wrote:
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details.
The problem was that I registered the latex-math directive in an old- fashioned way that did not work with new docutils versions. I fixed it in svn so that it should work with both old and new versions. I also changed the name of the rst2latex.py script to rst2latexmath.py. Jens Jørgen
Cheers, Alan Isaac
------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Docutils-users mailing list Docutils-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/docutils-users
Please use "Reply All" to reply to the list.
Jens Jørgen Mortensen wrote:
On Sat, 2006-12-09 at 21:52 -0500, Alan G Isaac wrote:
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details.
The problem was that I registered the latex-math directive in an old- fashioned way that did not work with new docutils versions. I fixed it in svn so that it should work with both old and new versions. I also changed the name of the rst2latex.py script to rst2latexmath.py.
I'm interested in playing with this. I grabbed the current docutils svn, but it doesn't seem to work: rst2latex.py README.txt > stuff README.txt:1: (ERROR/3) Unknown interpreted text role "latex-math". .. default-role:: latex-math I don't know anything about rest or docutils, any hints?
I don't know anything about Rest either, but I think that the setup script fetches another version of rst2latex. If you use the version in the sandbox, rst2latexmath.py, it should work. By the way, it's pretty neat. Kudos to the devs. David 2006/12/12, Neal Becker <ndbecker2@gmail.com>:
Jens Jørgen Mortensen wrote:
On Sat, 2006-12-09 at 21:52 -0500, Alan G Isaac wrote:
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details.
The problem was that I registered the latex-math directive in an old- fashioned way that did not work with new docutils versions. I fixed it in svn so that it should work with both old and new versions. I also changed the name of the rst2latex.py script to rst2latexmath.py.
I'm interested in playing with this. I grabbed the current docutils svn, but it doesn't seem to work: rst2latex.py README.txt > stuff README.txt:1: (ERROR/3) Unknown interpreted text role "latex-math".
.. default-role:: latex-math
I don't know anything about rest or docutils, any hints?
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
David Huard wrote:
I don't know anything about Rest either, but I think that the setup script fetches another version of rst2latex. If you use the version in the sandbox, rst2latexmath.py, it should work.
By the way, it's pretty neat. Kudos to the devs.
David
2006/12/12, Neal Becker <ndbecker2@gmail.com>:
Jens Jørgen Mortensen wrote:
On Sat, 2006-12-09 at 21:52 -0500, Alan G Isaac wrote:
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details.
The problem was that I registered the latex-math directive in an old- fashioned way that did not work with new docutils versions. I fixed it in svn so that it should work with both old and new versions. I also changed the name of the rst2latex.py script to rst2latexmath.py.
I'm interested in playing with this. I grabbed the current docutils svn, but it doesn't seem to work: rst2latex.py README.txt > stuff README.txt:1: (ERROR/3) Unknown interpreted text role "latex-math".
.. default-role:: latex-math
I don't know anything about rest or docutils, any hints?
OK, so the installation procedure is: 1) svn checkout svn://svn.berlios.de/docutils/trunk docutils 2) ( cd docutils/docutils; python setup.py build && sudo python setup.py install ) 3) cd sandbox/jensj/latex_math/; sudo install *.py /usr/bin I was a bit confused by the doc docutils/sandbox/jensj/latex_math/README.txt, which says "The plug-in adds..." So I'm looking around the docutils install directory for a place to install plugins. I guess that's not the correct procedure.
On Dec 12, 2006, at 1:46 PM, Neal Becker wrote:
So I'm looking around the docutils install directory for a place to install plugins. I guess that's not the correct procedure.
Since we have Jens' ear, I'm sure he can confirm, but it does not appear that the latex-math role is actually added to doctutils yet. You need to run one of the latex_math script to get the effect.
On Tue, 2006-12-12 at 14:07 -0500, Jonathan Guyer wrote:
On Dec 12, 2006, at 1:46 PM, Neal Becker wrote:
So I'm looking around the docutils install directory for a place to install plugins. I guess that's not the correct procedure.
Since we have Jens' ear, I'm sure he can confirm, but it does not appear that the latex-math role is actually added to doctutils yet.
Correct! You need to use the special rst2mathml.py and rst2latexmath.py scripts.
You need to run one of the latex_math script to get the effect.
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Neal, maybe the easiest solution is to add two lines to the setup.py scripts: 'scripts' : ['tools/rst2html.py', 'tools/rst2s5.py', 'tools/rst2latex.py', 'tools/rst2newlatex.py', 'tools/rst2xml.py', 'tools/rst2pseudoxml.py', '../sandbox/jensj/latex_math/tools/rst2latexmath.py', '../sandbox/jensj/latex_math/tools/rst2mathml.py'],} David 2006/12/12, Neal Becker <ndbecker2@gmail.com>:
David Huard wrote:
I don't know anything about Rest either, but I think that the setup script fetches another version of rst2latex. If you use the version in the sandbox, rst2latexmath.py, it should work.
By the way, it's pretty neat. Kudos to the devs.
David
2006/12/12, Neal Becker <ndbecker2@gmail.com>:
Jens Jørgen Mortensen wrote:
On Sat, 2006-12-09 at 21:52 -0500, Alan G Isaac wrote:
On Sat, 9 Dec 2006, Jonathan Guyer apparently wrote:
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine.
This problem that docutils SVN head creates for the latex-math module in the sandbox should interest the docutils developers and perhaps the latex-math developer (Jens Jørgen Mortensen), so I'll forward your comment to the docutils list. You may wish to follow-up with any useful details.
The problem was that I registered the latex-math directive in an old- fashioned way that did not work with new docutils versions. I fixed
it
in svn so that it should work with both old and new versions. I also changed the name of the rst2latex.py script to rst2latexmath.py.
I'm interested in playing with this. I grabbed the current docutils svn, but it doesn't seem to work: rst2latex.py README.txt > stuff README.txt:1: (ERROR/3) Unknown interpreted text role "latex-math".
.. default-role:: latex-math
I don't know anything about rest or docutils, any hints?
OK, so the installation procedure is:
1) svn checkout svn://svn.berlios.de/docutils/trunk docutils 2) ( cd docutils/docutils; python setup.py build && sudo python setup.py install ) 3) cd sandbox/jensj/latex_math/; sudo install *.py /usr/bin
I was a bit confused by the doc docutils/sandbox/jensj/latex_math/README.txt, which says "The plug-in adds..."
So I'm looking around the docutils install directory for a place to install plugins. I guess that's not the correct procedure.
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Hi all, I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ? If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps. Cheers, David 2006/12/9, Jonathan Guyer <guyer@nist.gov>:
On Dec 9, 2006, at 12:31 AM, Alan G Isaac wrote:
On Fri, 8 Dec 2006, Jonathan Guyer apparently wrote:
Do you know if any of that actually works? The last time I looked at it, I couldn't figure out what to do with it.
*If* it works? Yes: it works great.
I see my confusion. I'd gotten the sandbox via svn along with docutils HEAD, and it doesn't work with that (it throws an error deep within docutils). I switched to docutils-0.4 and it seems fine. _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
David Huard wrote:
Hi all,
I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ?
If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps.
I'm curious about this also. Is the idea that epydoc can be hacked to work with a hacked docutils? So, the plan is to incorporate docutils modifications (so that rst2latex would recognize latex-math role), and then to somehow hook this into epydoc? (And, neither of these is yet available for testing, right?)
On Dec 13, 2006, at 9:08 AM, Neal Becker wrote:
I'm curious about this also. Is the idea that epydoc can be hacked to work with a hacked docutils? So, the plan is to incorporate docutils modifications (so that rst2latex would recognize latex-math role), and then to somehow hook this into epydoc? (And, neither of these is yet available for testing, right?)
Unhacked epydoc works with unhacked docutils right now. My hacks to epydoc are because I don't like how it looks on the page and too much of the LaTeX output is hard-coded. I refactored it so that you can supply your own .sty file to get the output to look the way you want (and I wrote a style to mimic the output that epydoc already had for people who like the way it was (presumably Ed Loper is one of those)). We build our manuals (and webpage) via a "build_docs" command to our setup.py script. This command invokes the eypdoc command-line tool, which in turn imports docutils. I'm not sure what would be needed to get epydoc to see the latex-math role. epydoc appeared to have stagnated for awhile, but it's been active lately, with the last alpha release in August and svn checkins as recently as ten days ago, so I think this is probably a good time to put our two cents in for what we'd like to see. Certainly it's a good time for me to bring my changes up to date and pass them back.
Jonathan Guyer wrote:
On Dec 13, 2006, at 9:08 AM, Neal Becker wrote:
I'm curious about this also. Is the idea that epydoc can be hacked to work with a hacked docutils? So, the plan is to incorporate docutils modifications (so that rst2latex would recognize latex-math role), and then to somehow hook this into epydoc? (And, neither of these is yet available for testing, right?)
Unhacked epydoc works with unhacked docutils right now. ...
Sorry, I'm not sure what you mean here. I am looking at epydoc3.0alpha3 source. I see that epydoc can _parse_ rst. Is that what you mean by working with docutils? I also see that epydoc can write (some kind of) latex. Is that the idea?
On Dec 13, 2006, at 10:29 AM, Neal Becker wrote:
Sorry, I'm not sure what you mean here. I am looking at epydoc3.0alpha3 source.
I have no experience with epydoc 3, other than knowing it exists. It's on my list.
I see that epydoc can _parse_ rst. Is that what you mean by working with docutils?
Yes. What did you mean?
I also see that epydoc can write (some kind of) latex. Is that the idea?
Yes. Feed Python code (with reST docstrings) through epydoc and you get LaTeX or HTML out. For straight reST files, like READMEs, we just invoke docutils directly. I don't think epydoc is of any use there.
Jonathan Guyer wrote:
On Dec 13, 2006, at 10:29 AM, Neal Becker wrote:
Sorry, I'm not sure what you mean here. I am looking at epydoc3.0alpha3 source.
I have no experience with epydoc 3, other than knowing it exists. It's on my list.
I see that epydoc can _parse_ rst. Is that what you mean by working with docutils?
Yes. What did you mean?
I also see that epydoc can write (some kind of) latex. Is that the idea?
Yes.
Feed Python code (with reST docstrings) through epydoc and you get LaTeX or HTML out. For straight reST files, like READMEs, we just invoke docutils directly. I don't think epydoc is of any use there.
I'm looking to markup my python code, including latex so I can nicely include math. I see I could use epydoc, and that epydoc can read rst, but does this allow me to include latex? If so, how? And what backends would then be used to generate either pdf or (x)html?
On Dec 13, 2006, at 11:21 AM, Neal Becker wrote:
I'm looking to markup my python code, including latex so I can nicely include math. I see I could use epydoc, and that epydoc can read rst, but does this allow me to include latex? If so, how? And what backends would then be used to generate either pdf or (x)html?
Right now, you use .. raw:: latex directives. Kind of clumsy for inline expressions, but not too bad for display equations. For long stretches of inline math in text, we just do ".. raw:: latex" and write everything in LaTeX. Not ideal, but works OK. I'm certainly looking forward to cleaning everything up with latex-math. Once you have that, invoking epydoc --latex mypython.py generates a directory of .tex files and epydoc --html mypython.py generates a directory of .html files. Epydoc has a --pdf option, but it goes via .dvi --> .ps --> .pdf, which is kind of silly in this day and age. epydoc generates a master api.tex file, but we ignore and/or manipulate that and use our own that we feed straight through pdflatex.
On Wed, 13 Dec 2006, Neal Becker apparently wrote:
I'm curious about this also. Is the idea that epydoc can be hacked to work with a hacked docutils? So, the plan is to incorporate docutils modifications (so that rst2latex would recognize latex-math role), and then to somehow hook this into epydoc? (And, neither of these is yet available for testing, right?)
Do not forget rest2mathml, which produces XHTML+MathML. FireFox handles this great. (I haven't tried IE 7: does it yet handle XML?) If someone can patch (already wonderful) epydoc to allow this, that would allow truly lovely online documentation of technical code. (In the meantime, I imagine you could pass the LaTeX raw: the people who care about the math will often be able to read the LaTeX.) Cheers, Alan Isaac
On Tuesday 12 December 2006 12:20, David Huard wrote:
Hi all,
I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ?
If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps.
I have also been very interested in this discussion. Once the issue is settled, would someone please write a wiki page outlining the preferred format for scipy/numpy documentation, along with a short explanation of how to do the markup? Maybe every function could get a documentation template, which would make it easier for less experienced SciPy users (such as myself) to contribute to the community. Darren
On 12/14/06, Darren Dale <dd55@cornell.edu> wrote:
On Tuesday 12 December 2006 12:20, David Huard wrote:
Hi all,
I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ?
If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps.
I have also been very interested in this discussion. Once the issue is settled, would someone please write a wiki page outlining the preferred format for scipy/numpy documentation, along with a short explanation of how to do the markup?
Numpy contains a good deal of C code that needs documentation, will Epydoc do the job? It might make sense to use different systems for the python and C code, say Epydoc for the former and doxygen for the latter. Also, the docstrings for the python visible functions in numpy are separated from the code and probably need to be parsed. Some code that writes phony functions with the appropriate comments might do the trick, although some provision would probably be needed to pull out the function signature. Chuck
In case I gave the impression I was planning to do something on this, I still am. It's just that I've been very busy. I am still planning to try to build the FiPy docs and then spin a skeleton document based on it for numpy, perhaps with content from the numarray docs, and another for scipy during the upcoming Christmas break. Each of these docs would have one appendix which contains a reference built from the individual method/function docstrings. Each would also have another appendix of long full-module examples with LaTeX and plotting results. I was also going to try to write a docstring documentation spec as part of this. This is quite a bit of work for me as I expect I need a few days preparation to get a Linux distro up to date with a build system. I thought the concensus was headed toward epydoc + ReST with LaTeX markup, so I was planning to adopt FiPy's nice model for using this. I still hadn't ruled out doxygen and I was planning on looking at it too to see if it has advantages. My guess is that either would be fine, although people here might be more comfortable with epydoc. David, if this sounds like a good plan to you, perhaps you can move ahead with this, as it'll be a few days before I can start. Gary R. David Huard wrote:
Hi all,
I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ?
If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps.
Cheers,
David
Gary, I got epydoc running on plain docstrings and the latex-math role from Jens running. However, I havent't looked at how to use both together. In any case, its not yet clear to me how the information should be organized. The first post on this thread suggested: """ 1-2 sentences summarizing what the function does. INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple) EXAMPLES: a *bunch* of examples, often a whole page. NOTES: misc other notes ALGORITHM: notes about the implementation or algorithm, if applicable AUTHORS: -- name (date): notes about what was done """ But I think this is maybe too heavy for doctrings. Anyone using IDLE will see large yellow docs pop up. I think docstrings should limit themselves to: """ 1-2 sentences summarizing what the function does. INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple) SEE ALSO: """ The question now is: where too put all the other stuff, namely the examples, notes, algorithm and authorship. Unless I'm mistaken, epydoc has support for documentation outside of the docstring, so we could use that. Another idea is to assign "documentation attributes" to functions. For instance: def func(a,b,c): """definition :Input: ... :Output: ... :See also: ... """ func.__examples__=[ """ >>> func(1,2,3) [4,5,6] ... """, """...""", ...] func.__author__='Bobby' func.__notes__=' Those attributes could be accessed by an example function def example(function, i=None): """Run example i from function. If i is None, run all examples. """ and eventually an ipython magic command (func?!). The other solution is to put the examples in completely different files, or fetch them from a wiki page (but I don't think this is implemented yet in epydoc). I think we should dig more into what epydoc offers before rushing into this. Then we could propose a couple of different layouts, and once everyone agrees on which is best, go ahead and apply it at large. I started looking at FiPY's documentation and their setup is impressive. I'll look more deeply into it. David 2006/12/16, Gary Ruben <gruben@bigpond.net.au>:
In case I gave the impression I was planning to do something on this, I still am. It's just that I've been very busy. I am still planning to try to build the FiPy docs and then spin a skeleton document based on it for numpy, perhaps with content from the numarray docs, and another for scipy during the upcoming Christmas break.
Each of these docs would have one appendix which contains a reference built from the individual method/function docstrings. Each would also have another appendix of long full-module examples with LaTeX and plotting results. I was also going to try to write a docstring documentation spec as part of this.
This is quite a bit of work for me as I expect I need a few days preparation to get a Linux distro up to date with a build system. I thought the concensus was headed toward epydoc + ReST with LaTeX markup, so I was planning to adopt FiPy's nice model for using this. I still hadn't ruled out doxygen and I was planning on looking at it too to see if it has advantages. My guess is that either would be fine, although people here might be more comfortable with epydoc.
David, if this sounds like a good plan to you, perhaps you can move ahead with this, as it'll be a few days before I can start.
Gary R.
David Huard wrote:
Hi all,
I followed this thread with much interest and I wouldn't like it to die without some kind of concensus being reached for the Scipy documention system. Am I correct to say that Epydoc + REST/Latex seems the way to go ?
If this is the case, what's next ? I'm not familiar with any of this, but I'd be great if someone knowledgeable could define a roadmap and create a couple of tickets so that people like me could contribute small steps.
Cheers,
David
Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
On Mon, 18 Dec 2006, David Huard wrote:
I got epydoc running on plain docstrings and the latex-math role from Jens running. However, I havent't looked at how to use both together.
I'm in the same position. But I guess this should be simple for anyone familiar with docutils, so I'll copy this to docutils-users. Cheers, Alan Isaac
Le Mon, 18 Dec 2006 13:14:22 -0500, Alan Isaac a écrit :
On Mon, 18 Dec 2006, David Huard wrote:
I got epydoc running on plain docstrings and the latex-math role from Jens running. However, I havent't looked at how to use both together.
I'm in the same position. But I guess this should be simple for anyone familiar with docutils, so I'll copy this to docutils-users.
Cheers, Alan Isaac
If I understand correctly, I'd have to add the latex-math role to rst/roles.py and the latex-math directive to the rst/directives directory and register it in the __init__. However, the directives defined in Jens' sandbox are writer specific, so I'm a bit lost. A little bit of context: The SciPy and NumPy folks are looking at the various documentation systems out there to build the API documentation and tutorials. Up to now, the combination epydoc+reST seems to most powerful. However, Latex formulas are a must for those packages and the raw role is a bit low-level for our needs, hence the interest in including the latex-math role and directive in the trunk so that epydoc can run smoothly using it. Thanks a lot, David
On Tue, 2006-12-19 at 15:34 -0500, David Huard wrote:
If I understand correctly, I'd have to add the latex-math role to rst/roles.py and the latex-math directive to the rst/directives directory and register it in the __init__. However, the directives defined in Jens' sandbox are writer specific, so I'm a bit lost.
You will have to merge the code from rst2mathml.py and rst2latexmath.py. I would start from the code in rst2mathml.py, where the node class (latex_math) will have to be changed to something like this: class latex_math(nodes.Element): tagname = '#latex-math' def __init__(self, rawsource, mathml_tree, latex): nodes.Element.__init__(self, rawsource) self.mathml_tree = mathml_tree self.latex = latex Also the latex_math_role function and the latex_math_directive class will have to be modified a bit to use the new node class. And then the visit/depart methods should be added to the writers. Hope that helps, Jens Jørgen
A little bit of context: The SciPy and NumPy folks are looking at the various documentation systems out there to build the API documentation and tutorials. Up to now, the combination epydoc+reST seems to most powerful. However, Latex formulas are a must for those packages and the raw role is a bit low-level for our needs, hence the interest in including the latex-math role and directive in the trunk so that epydoc can run smoothly using it.
Thanks Jens, For those who would like to try it out, here is the patch made on a recent svn checkout of docutils. The patch adds the latex-math role and directive to docutils. Using it, you should be able to run epydoc and generate pdf output from the docstrings of any python module (correctly formated, that is). The html output doesn't work very well for me, but that may be due to my browser not supporting MathML fonts. The next step would be to add support for the :input:, :ouput:, :example: and :see also: tags. It shouldn't be difficult, but I'll leave that to someone else : ) To see it in action, try epydoc --pdf epytest.py Cheers, David 2006/12/20, Jens Jørgen Mortensen <jensj@fysik.dtu.dk>:
On Tue, 2006-12-19 at 15:34 -0500, David Huard wrote:
If I understand correctly, I'd have to add the latex-math role to rst/roles.py and the latex-math directive to the rst/directives directory and register it in the __init__. However, the directives defined in Jens' sandbox are writer specific, so I'm a bit lost.
You will have to merge the code from rst2mathml.py and rst2latexmath.py. I would start from the code in rst2mathml.py, where the node class (latex_math) will have to be changed to something like this:
class latex_math(nodes.Element): tagname = '#latex-math' def __init__(self, rawsource, mathml_tree, latex): nodes.Element.__init__(self, rawsource) self.mathml_tree = mathml_tree self.latex = latex
Also the latex_math_role function and the latex_math_directive class will have to be modified a bit to use the new node class.
And then the visit/depart methods should be added to the writers.
Hope that helps, Jens Jørgen
A little bit of context: The SciPy and NumPy folks are looking at the various documentation systems out there to build the API documentation and tutorials. Up to now, the combination epydoc+reST seems to most powerful. However, Latex formulas are a must for those packages and the raw role is a bit low-level for our needs, hence the interest in including the latex-math role and directive in the trunk so that epydoc can run smoothly using it.
On Wed, 20 Dec 2006, David Huard apparently wrote:
The html output doesn't work very well for me, but that may be due to my browser not supporting MathML fonts.
It is XHTML+MathML. So you need to: 1. give the document a .xml extension (or otherwise ensure that it is served as XML) 2. Use FireFox, since IE support for XML is nil (through version 6, anyway) Cheers, Alan Isaac
Hi David, I applied your patch, but I've missed something because I get an error when it tries to import parse_latex_math latex_math from latexparser. i.e. latexparser is nowhere to be found. Is this in some other file which Jens has or do I need to rename something? I wasn't able to work this out from reading others' posts to this thread. thanks, Gary David Huard wrote:
Thanks Jens,
For those who would like to try it out, here is the patch made on a recent svn checkout of docutils. The patch adds the latex-math role and directive to docutils. Using it, you should be able to run epydoc and generate pdf output from the docstrings of any python module (correctly formated, that is). The html output doesn't work very well for me, but that may be due to my browser not supporting MathML fonts.
The next step would be to add support for the :input:, :ouput:, :example: and :see also: tags. It shouldn't be difficult, but I'll leave that to someone else : )
To see it in action, try
epydoc --pdf epytest.py <snip>
Hi Gary, No, it's my mistake. The svn diff didn't catch one file because it wasn't versioned. I've attached a new patch including it. Sorry I missed that. David 2006/12/22, Gary Ruben <gruben@bigpond.net.au>:
Hi David, I applied your patch, but I've missed something because I get an error when it tries to import parse_latex_math latex_math from latexparser. i.e. latexparser is nowhere to be found. Is this in some other file which Jens has or do I need to rename something? I wasn't able to work this out from reading others' posts to this thread. thanks, Gary
David Huard wrote:
Thanks Jens,
For those who would like to try it out, here is the patch made on a recent svn checkout of docutils. The patch adds the latex-math role and directive to docutils. Using it, you should be able to run epydoc and generate pdf output from the docstrings of any python module (correctly formated, that is). The html output doesn't work very well for me, but that may be due to my browser not supporting MathML fonts.
The next step would be to add support for the :input:, :ouput:, :example: and :see also: tags. It shouldn't be difficult, but I'll leave that to someone else : )
To see it in action, try
epydoc --pdf epytest.py <snip>
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Thanks David, It works now. Actually, epydoc did break at the LaTeX point on my system because it wants fancyheadings.sty, which is a deprecated LaTeX package not package managed by MikTeX - I just replaced it in the .tex output with fancyhdr.sty, because I couldn't be bothered installing fancyheadings, and it worked. This is a problem in epydoc and not a problem with your patch. We probably don't need to use the epydoc preamble anyway. thanks, Gary David Huard wrote:
Hi Gary, No, it's my mistake. The svn diff didn't catch one file because it wasn't versioned. I've attached a new patch including it.
Sorry I missed that.
David
Thanks for moving on this David, I'm still getting a machine set up to work on this, but I'm closer now. I like your doc attributes idea as it provides a very obvious way to access the extra docs via ipython, helper functions etc. Depending on how epydoc supports docs outside of the docstring, that may also be a good solution. I think we should avoid small examples in their own modules and reserve external modules for large, detailed examples a'la FiPy. If the 'dynamically accessing a wiki for docs' idea were implemented, it would have to be as an adjunct to completely self-contained documentation, so I'd keep it as an idea, but forget about implementing it now. I also think that docstrings should also have at least one minimal, typical-case call example. There's a compromise between bloat and help and I always find examples an enormous help. Gary R. David Huard wrote:
Gary,
I got epydoc running on plain docstrings and the latex-math role from Jens running. However, I havent't looked at how to use both together.
In any case, its not yet clear to me how the information should be organized. The first post on this thread suggested:
""" 1-2 sentences summarizing what the function does.
INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple)
EXAMPLES: a *bunch* of examples, often a whole page.
NOTES: misc other notes
ALGORITHM: notes about the implementation or algorithm, if applicable
AUTHORS: -- name (date): notes about what was done """
But I think this is maybe too heavy for doctrings. Anyone using IDLE will see large yellow docs pop up. I think docstrings should limit themselves to:
""" 1-2 sentences summarizing what the function does.
INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple)
SEE ALSO: """
The question now is: where too put all the other stuff, namely the examples, notes, algorithm and authorship. Unless I'm mistaken, epydoc has support for documentation outside of the docstring, so we could use that. Another idea is to assign "documentation attributes" to functions. For instance:
def func(a,b,c): """definition :Input: ... :Output: ... :See also: ... """ func.__examples__=[ """ >>> func(1,2,3) [4,5,6] ... """, """...""", ...] func.__author__='Bobby' func.__notes__=' Those attributes could be accessed by an example function def example(function, i=None): """Run example i from function. If i is None, run all examples. """ and eventually an ipython magic command (func?!).
The other solution is to put the examples in completely different files, or fetch them from a wiki page (but I don't think this is implemented yet in epydoc).
I think we should dig more into what epydoc offers before rushing into this. Then we could propose a couple of different layouts, and once everyone agrees on which is best, go ahead and apply it at large.
I started looking at FiPY's documentation and their setup is impressive. I'll look more deeply into it.
David
On Dec 8, 2006, at 7:30 PM, Gary Ruben wrote:
Can you suggest something existing for the extra markup? I've never used anything fancier than plain docstrings. The other existing options seem to be (from a brief look)
reST/docutils epydoc pythondoc doxygen happydoc
Right. These all exist. They all have their issues and their benefits. I just don't see a need for yet another one. We happen to use the epydoc tool to process reST markup.
I like the idea of supporting LaTeX markup, cross references.
Alan addresses this.
I like the idea of supporting matplotlib examples (opinions?).
Supporting how? You certainly can have matplotlib code in doctest lines and you can use reST's ..image:: declaration to include output in your documentation.
I like Mark's idea of a wiki page per function/class/module if it can be done. I also don't mind William's YADMUS.
It's not that I mind his markup; it's that I don't think Python benefits from having at least a half-dozen of them. If the energy that went into developing reST, epydoc, pydoc, doxygen for Python, happydoc, and SAGE-YADMUS had gone into improving any one of them, I think the entire Python community would be better off.
The problem I see with plain doctests is that you can't easily choose to run, say, just the 3rd example out of 5 without specifying some sort of delineation between them, but this would be easy; something like: # Ex1: or Ex2 etc. between each doctest.
I'm not sure exactly what you mean by this. We have full examples of different applications of FiPy. Those are each in individual files where the entire thing is just a big docstring, including doctest lines, with a little bit of code at the end to take care of loading the doctest lines and running them. Those individual scripts can be run on their own (although simply as scripts, not as doctests). We also document the usage of individual methods with specific doctests. I can't imagine why I'd want to just run one of them.
Supporting how? You certainly can have matplotlib code in doctest lines and you can use reST's ..image:: declaration to include output in your documentation.
That's fine. I just raised it because I like graphical examples but I'm concerned about advocating reliance on any separate package, although matplotlib has probably gained enough acceptance that it's OK to assume it will be available on most numpy users' systems.
The problem I see with plain doctests is that you can't easily choose to run, say, just the 3rd example out of 5 without specifying some sort of delineation between them, but this would be easy; something like: # Ex1: or Ex2 etc. between each doctest.
I'm not sure exactly what you mean by this. We have full examples of different applications of FiPy. Those are each in individual files where the entire thing is just a big docstring, including doctest lines, with a little bit of code at the end to take care of loading the doctest lines and running them. Those individual scripts can be run on their own (although simply as scripts, not as doctests). We also document the usage of individual methods with specific doctests. I can't imagine why I'd want to just run one of them.
I was envisaging several small examples in docstrings in the code modules with the ability to run any of them interactively. I think this is how it would be used in SAGE and in ipython. I've now downloaded FiPy to take a look. I think it's very nicely documented. My impression is that the in-code examples have limited mark-up and just a couple of simple, non-graphical examples. This allows them to be uncluttered and remain comprehensible when getting the docstring interactively. Detailed examples, although heavily marked-up, are in separate modules. I'd be happy if numpy/scipy adopted this approach of limiting and separating the mark-up. The LaTeX source is available. I'd like to know what others think about following FiPy's example. What else needs to be considered? I think that to build a skeleton reference document, the numarray document could be a starting point as its source is available, except it seems, for the LaTeX class file. I couldn't find the Numeric doc source. I'm not sure whether it's OK to just include relevant bits of these, but it looks like it's OK, provided suitable licence stuff and attribution is included. A skeleton based on FiPy's manual and the available numarray LaTeX with a mini-spec for docstrings would be something I could have a go at. Gary R.
On Dec 9, 2006, at 4:26 AM, Gary Ruben wrote:
I'm concerned about advocating reliance on any separate package, although matplotlib has probably gained enough acceptance that it's OK to assume it will be available on most numpy users' systems.
We actually provide a set of abstraction classes for that reason, and our examples all just say things like "viewer = fipy.viewers.make (...)" to return something acceptable as long as at least one of our supported viewers is installed. Particularly after what we saw at SciPy'06, though, we're moving toward only supporting matplotlib (at least for 1D and 2D).
I was envisaging several small examples in docstrings in the code modules with the ability to run any of them interactively. I think this is how it would be used in SAGE and in ipython.
You can restrict doctests down to one module vs. another, but I don't know if there's any way to restrict it to only running the tests of a particular method (although I've never tried), and there's certainly no mechanism to run only some tests of a method but not others. I haven't missed that ability, but I can see how other uses might call for it.
I've now downloaded FiPy to take a look. I think it's very nicely documented.
Thank you.
My impression is that the in-code examples have limited mark-up and just a couple of simple, non-graphical examples.
The *NoiseVariable classes have some graphical examples, but other than that, we either haven't found the time or the need to do it. I'd be happy to hear of cases where you think graphics would be helpful in our API documentation. If anybody else wants to follow along at home, you can download our manuals from <http://www.ctcms.nist.gov/fipy/download/> without having to get the rest of the package. fipy.pdf is the user guide, with examples. reference.pdf is the API documentation.
This allows them to be uncluttered and remain comprehensible when getting the docstring interactively. Detailed examples, although heavily marked- up, are in separate modules.
We provide a mechanism, via our setup.py script, to strip the markup if people want to adapt our examples without figuring out how to write docstring/doctest markup. On the other hand, I've come to writing all of my research codes in docstring/doctest. The ability to interlace math and code makes it a lot easier for me to figure out my codes when I come back to them later and clearer to explain to other people. There's an emacs mode for writing doctest out there somewhere, and I've cobbled together some modifications of Alpha's Python mode that simplify writing doctest; I really need to finish that up and check it back in to AlphaTcl.
I'd be happy if numpy/scipy adopted this approach of limiting and separating the mark-up. The LaTeX source is available.
Be aware that you might have trouble building our docs. We used to try to ensure that anybody could build it, but eventually decided that we were more concerned with our ability to produce manuals that looked the way we want than we were with the rare cases of anybody else trying to build it. It's not a huge deal, but you'll need a couple of LaTeX classes and you'll need to adjust your paths to find our hacked epydoc. Anyway, if you want help, write me directly and I'll try to get it working for you.
participants (12)
-
Alan G Isaac -
Alan Isaac
-
Charles R Harris -
Darren Dale -
David Cournapeau -
David Huard -
Fernando Perez -
Gary Ruben -
Jens Jørgen Mortensen -
Jonathan Guyer -
Mark Koudritsky -
Neal Becker