Mailman 3 December 1999 - Doc-SIG

docstring grammar
by David Ascher Jan. 27, 2000

Jan. 27, 2000

Proposed format for docstrings: The whitespace at the beginning of a docstring is ignored. Paragraphs are separated by one or more blank lines. For compatibility with Guido, IDLE and Pythonwin (and increasing the likelihood that the proposal will be accepted by GvR), the docstrings of callables must follow the following convention established in Python's builtins: >>> print len.__doc__ len(object) -> integer Return the number of items of a sequence or mapping. In other words, the first paragraph must fit on a line, repeat the name of the callable, with a 'wordy' signature, the ' -> ' string, and the type of the return value. The second paragraph must be a one-sentence description of the callable. It is also allowed to have those two bits separated by a " -- " string: >>> print [].pop.__doc__ L.pop([index]) -> item -- remove and return item at index (default last) and functions which don't return anything can omit the " -> foo" bit: L.append(object) -- append object to end Each paragraph is either 'text' or a 'keyword-tagged block'. A keyword is a case-sensitive element of [a-zA-Z_]+ followed by two colons (with optional whitespace between the keyword and the colons, but no whitespace allowed between the two colons). A paragraph which doesn't start with a keyword is 'text'. Characters between # signs and the end of the line are stripped by the docstring parser. A 'keyword-tagged block' is nested much like Python code. Just like in Python, the block can either be on the same line as the keyword if it is one-line long (I'll refer to such blocks as 'text' blocks even though they aren't in visual paragraphs), or needs to be indented relative to the keyword. Examples: Author:: Guido van Rossum # comments are stripped Date_of_release :: 1/1/1999 # The key is "Date_of_release" and the # whitespace before the : is stripped Contributors:: # The value is a block of lines. John Doe Ronald Reagan Francois Mitterand Some keywords can have special parsing rules, as the block of text which the keyword designates is well-specified by the rules above. The first example of such a keyword-specific parsing rule is for Arguments: Arguments:: self -- instance input (sequence) -- the sequence which is being processed (the specific syntax of Arguments:: is left for a later discussion). Other candidates which can impose specific parsing rules are: ReturnType, Date, Version, etc. Text blocks can be followed by indented blocks as well -- those are 'children' blocks of the outdented block. 'text' blocks which start with * or - are tagged as 'bullet items' for rendering. The bullet marker has to be consistent within a given level of indentation. Example: * this is one bullet - this is a sub-bullet - this is another sub-bullet * this is another bullet In text blocks, some strings are recognized as links: .foo in the docstring of a class will refer to the foo attribute of the class. In the docstring of a method, it will refer to the foo attribute of the method's class. In the docstring of a module it will refer to a function or class defined in that module foo.bar will refer to the bar attribute of foo, which will be looked up in the following namespaces in order: (to be determined) URL notation is automatically recognized. [foo] refers to the keyword 'foo' in the section 'References' of the current docstring. [..] links cannot span multiple lines or contain whitespaces (as keywords can't). (in other words, if a [ is not matched by a ] in the same line or before a whitespace character is hit, then it is a syntax error. References:: foo:: My Dissertation, University Press, 1902 The set of keywords which are 'officially sanctioned' is: For module docstrings: [see Trove discussion for a good starting set -- this discussion has been had!] For class docstrings: [To be determined] For method docstrings: [To be determined] For function docstrings: [To be determined] Miscellaneous Thoughts: I chose double-colon notation for keywords so that one can have text paragraphs which match the 'word:' notation without having them be interpreted as keywords. Does this proposal make docstrings whitespace-heavy -- the requirement to break each paragraph with a line of whitespace means that a lot of lines are blank, especially when doing 'bulleted lists' The above was (quickly) written with parsing in mind. Is it really easily parseable? If not, what needs to be changed so that it is parseable? I also wanted to make sure that syntax errors could be flagged early and 'localized' for aid in debugging. I'm not sure that I did that carefully enough. Are there normal uses in docstrings where one wants to turn off the automatic link detection? Is there value in having string interpolation? David Arnold mentioned __version__ = "$Revision$[11:-2] __date__ = "$Date$ which raises some issues. I don't think that having [11:-2] evaluated by the docstring parser is a wise idea. However, I can imagine that the module author could do: __version__ = "$Revision$"[11:-2] in the Python code, and then Version:: %(__version__)s in the docstring and that such a simple string interpolation mechanism could have value. I'm not sure it's worth the complication though. What dictionary would be used to do the interpolation? Hopefully constructively, --david PS: It goes without saying that while I railed against design by committee, I am of course hopeful for feedback, for technical reasons (dummy, you forgot special cases X, Y and Z!) and because I realize that a standards proposal needs at least broad agreement if not consensus to be effective in the long run. The sharper-eyed will note that I stacked the deck in my favor in the above proposal by including what Guido does naturally as valid in the proposed grammar.

14 79

Documentation systems for other languages
by Fred L. Drake, Jr. Jan. 8, 2000

Jan. 8, 2000

I've started a list of documentation systems used for other languages: http://www.python.org/sigs/doc-sig/otherlangs.html The list isn't complete, and that's not really my intention. If there are other systems that are interesting, I'll be glad to add them to the list. So far, all of these are for source-embedded docs, but I'd love to find examples that aren't. If someone can send me a meaningful link to POD information, and an example, I'll add it. (It's on the list, just not hyperlinked.) This is *not* intended to be a list of general-purpose systems, so things like DocBook and LinuxDoc are not listed. -Fred -- Fred L. Drake, Jr. <fdrake(a)acm.org> Corporation for National Research Initiatives

6 11

Mandoc (I give up temporarily)
by Manuel Gutierrez Algaba Dec. 26, 1999

Dec. 26, 1999

Dear folks, I've been working in a eval-oriented doc application, due to the lack of time I can't keep on building it, so I think the best I can do is release what I've done so It helps to the discussion. You can find it at: http://www.ctv.es/USERS/irmina/mandoc/ In that directory the relevant files are: mandoc.py (specially this one) restrict.py logger.py examples.py mainly. mandoc.tex is rather "green" and the rest doesn't care. Basically the underlying idea is to eval docstrings and supply and ever-extensible-flexible set of keywords and contructs , and paying a bit of so much flexibility in rendering/formatting time. Enjoy it. It pretended to generate linux-sgml , so that we could have .html, .info. , .tex, very easily. Regards/Saludos Manolo www.ctv.es/USERS/irmina /TeEncontreX.html /texpython.htm /SantisimaInquisicion/index.html If little green men land in your back yard, hide any little green women you've got in the house. -- Mike Harding, "The Armchair Anarchist's Almanac"

1 0

Editorial for freshmeat
by Manuel Gutierrez Algaba Dec. 24, 1999

Dec. 24, 1999

The need of documentation. Open Source phase II As more and more programms are made and as more and more information is available in USENET, free electronic books, tutorials, and so on, the bigger the problems to use/handle that information/code are. The main advantage of Open Source stuff: - Use and reuse of information/code (freely) should be or could be greatly improved using wise (marking) documentation. Almost nearly every programm ( even the simplest one) is divided into several parts, each part is related to something very specific (usually) : - sockets - GUI - formatting into html/TeX - i18n .... So every programm does a 'main' task by resolving many other small common tasks. The developer of programms who learns by reading other's code knows how to find those common tasks. Well, sometimes. Firstly, because the code is not always well structured enough. Secondly because he lacks the time , patience to browse in the middle of so much code. The user sometimes has a very hard time trying to setup, identify the components of the programm or simply learn how to use it. This is mainly because of a 'monolithic' info. I'll argue against this "man-page-hacker-like" attitude later. So we have a Open Source programm that can't be fully reused/used by the lack of information or by a deficient structuration. This is very, very common and it'll become more painfully common in the future. Most developers are happy enough when their code runs smoothly (or simply runs), when they've managed to arrange the code so it compiles and does something. In fact, most developers struggle non stop with buggy code, adding new features and so on, they pay little attention to the fact that most of the C/python/perl/java modules solve relatively "general" problems (KDE programming, sockets, i18n, special formatting). Of course, if they're good enough they'll put most of it into libraries and objects, but then those libraries and objects will remain "hidden" behind the "main tasks" of the programm. I wonder: If you spend three days solving small problems, why don't you spend ten minutes marking those solved small problems (GUI, i18n...) somehow. This way your programm would be useful twice: by itself, by the objects and libraries it keeps. Is this marking difficult? How can I do this marking? One answer is using XML, sgml or something like that. [ MUCH MORE TO BE SAID STILL] Is this subject/style appropiated ? Is it worth that I finish this article to be published ? It's unfinished because I'd like the assurance of it's worth the effort. I'm the author of some free programms(some in freshmeat), see URL above. Regards/Saludos Manolo www.ctv.es/USERS/irmina /TeEncontreX.html /texpython.htm /SantisimaInquisicion/index.html Man's reach must exceed his grasp, for why else the heavens?

1 0

doc-sig/types-sig clash?
by Skip Montanaro Dec. 20, 1999

Dec. 20, 1999

One of the proposals regarding typing seems to be inserting stuff into doc strings. It is perhaps worth noting for those who don't subscribe to the doc-sig that that bunch of ne'er-do-wells (I mean esteemed colleagues) also has their eyes on the doc string (imagine that!). Just raising a small flag to make sure people don't assume the doc string is their private sandbox. Apologies if this has already been addressed. I'm still wading through all the recent Python activity. Skip

2 2

IPC8 Developers Day
by Fred L. Drake, Jr. Dec. 16, 1999

Dec. 16, 1999

For anyone who hasn't noticed, there will be a Doc-SIG session on Developer's Dat at IPC8. The session will be from 10:30 to Noon on Thursday, 27 January. Now, you really *HAVE* to attend IPC8 to avoid missing the documentation event of the millenium! Unless of course, you subscribe to the media's definition of "millenium", in which case there was nothing to miss. ;) I hope to have the discussion paper and updated Doc-SIG web pages up next week. -Fred -- Fred L. Drake, Jr. <fdrake(a)acm.org> Corporation for National Research Initiatives

2 1

update on docstring grammar discussion pending
by David Ascher Dec. 16, 1999

Dec. 16, 1999

I've been super busy in the last couple of weeks, so haven't had a time to polish up the docstring proposal. I hope to do so in the next couple of days, after I review all of the discussion on the sig (and catch up on the types-sig to see if there's relevant stuff there). Thank you for your patience, --david ascher

1 0

Sorry!
by Paul Prescod Dec. 14, 1999

Dec. 14, 1999

...for spamming you guys all night. I want to make sure that everybody's concerns get addressed. I'll slow down tonight. One issue that Greg raised was the difficulty of checking builtin types (in addition to the hairy parameterized types stuff). I've been thinking about this and I think that the doc-sig and the types-sig have the same problem. How do we sniff out the parameters and docstrings for methods without running dangerous binary code. I think that Java (and many other languages) has the right plan with "shadow libraries." The CORBA guys already use IDL as a static library syntax. I think that we should support both IDL and a strongly-typed Pythonic syntax. It might work something like this: def Int foo(a, b): "Foo, defined in module" pass def String foo(c, d, *args ): pass "Foo, defined in module" pass import _foo locals().update( _foo.__dict__ ) Maybe we would have some kind of a keyword instead of the locals() hack. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Three things to be wary of: A new kid in his prime A man who knows the answers, and code that runs first time http://www.geezjan.org/humor/computers/threes.html

1 0

Re: [Doc-SIG] docstring grammar
by Jack Jansen Dec. 3, 1999

Dec. 3, 1999

Now I'm getting confused. Confused about what the intention of docstrings is, that is: as I see them they're quick-reference documentation to be used by IDE's and class browsers and such to help you while you're busy hacking. But more and more terms like extraction of docstrings and such pops up, so I'm wondering whether people are envisioning a tangle/weave/web-like system where the full documentation is included in the source... And, yes, I know I happily joined in in the latter, with the multilingual discussion and more, but I think I would definitely prefer the former scheme, possibly even combined with the-latter-in-reverse, where the Real Documentation was structured in such a way that there could be tools that you would feed the documentation and the source, and that would add docstrings to the source. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

XML Conversion Update
by Fred L. Drake, Jr. Dec. 3, 1999

Dec. 3, 1999

Last week I promised on the Python list to describe the current status of the conversion to SGML/XML. Here it is! I'm currently able to parse all the LaTeX markup and generate either XML or SGML. The structure of the output is very similar to the input structure, but a number of minor improvements are made. The improvements are mostly very localized and have more to do with keeping the markup from getting to complex and disjointed, and eliminating some bogosities. I am not at all decided on a DTD to use. I see three options: 1. DocBook -- this has been developed and heavily use-tested by a number of corporate users, and is known to be good for technical documentation. There are tools and stylesheets available to convert from DocBook to HTML and printed formats. We'd probably need to specialize it, but it's designed for that. Konrad Hinsen has already developed one customization that he's using to document Python modules, and there's an initiative to create a common extension for documenting OO constructs. I've asked Konrad for some sample documentation so I can see how it actually works out. My concern with DocBook is that the markup may be a bit on the "heavy" side; I don't want the document source to be so markup-heavy that I'm the only one to work on them. 2. Create something similar to what we had in LaTeX, but with fewer warts. This is appealing because the conversion would be done sooner. However, new stylesheets would be needed, slowing down the usefulness of the result. It would also be the easiest to adopt for people already familiar with the current markup. 3. Create something entirely new and specific to Python. Clearly, this offers a lot of work to all the volunteers. We'd need requirements analysis, DTD design, stylesheets, and probably lots of things I haven't thought of. However, it also means we can limit the weight of the markup in the source, which might be a major advantage in getting people to use it. But *everyone* would have to learn it (well, everyone that writes documentation at any rate). This offers a great deal of opportunity to "get it right" for Python, but also a lot of rope. (You know what rope is used for, right?) I'd like to see some discussion on what should be done and what needs to be done. From where I sit, the most important thing is to make sure we can maintain a high level of semantic markup (hopefully making further improvements over what we already have), with generation of hypertext (HTML, info, whatever) being the next most important thing. Typeset documents are a requirement, but aren't as high up the list. I'm not terribly concerned about how XML/SGML-->foo conversion processes are implemented, with the caveat being that I need to be able to understand them without a massive learning curve. Clearly, Python code is a major option for tools (surprised?), but I can easily deal with using Java tools (with or without JPython), DSSSL processors (just don't expect me to maintain Jade/OpenJade!), XSL, CSS, and whatnot. I'd like to get away from having any Perl scripts involved, not because I think Perl is Evil, but because I'm not a Perl hacker. (Don't get me wrong; I make no claim that Perl is not Evil! ;) Comments, suggestions, volunteers? -Fred -- Fred L. Drake, Jr. <fdrake(a)acm.org> Corporation for National Research Initiatives

9 16