Mailman 3 March 2001 - Doc-SIG

Error in pyexpat docs
by uche.ogbuji＠fourthought.com July 9, 2002

July 9, 2002

An excerpt from the Python 2.0 docs for pyexpat: """ ParseFile (file) Parse XML data reading from the object file. file only needs to provide the read(nbytes) method, returning the empty string when there's no more data. [snip] The following attributes contain values relating to the most recent error encountered by an xmlparser object, and will only have correct values once a call to Parse() or ParseFile() has raised a xml.parsers.expat.error exception. ErrorByteIndex Byte index at which an error occurred. [etc.] """ The wrong, "xml.parsers.expat" is the first indicator that there might be a problem, yet I took the docs at their word and wrapped the call to ParseFile in a blanket try/except, only to find that no exception of any sort is ever raised by ParseFile. It turns out that ParseFile actually returns 0 on error, returning 1 otherwise. The first matter is that the code and the docs need to be reconciled. However, I would _much_ rather prefer that things were as in the docs. I think ParseFile should raise an exception rather than return an error flag. Interestingly enough, this is the same argument I had with a colleague just last week. -- Uche Ogbuji Principal Consultant uche.ogbuji(a)fourthought.com +1 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python

3 2

syntax vs semantics: implicit --> explicit
by Goodger, David April 2, 2001

April 2, 2001

I've found that many of our discussions about auto-documentation generators unnecessarily (and confusingly) mix arguments from different levels (syntax vs. semantics, and multi-layered semantics at that). In an effort to further make implicit explicit, and to reduce confusion & frustration, I think it's important to separate our discussions based on individual components. At least, we should be conscious of 'where we're coming from' and make that more explicit. For example, I think it's counterproductive to talk about the syntax of a particular construct (e.g. characters used to delimit literals) in the same breath as talking about a Python-specific concept (e.g. hyperlinks generated from the interpretation of literals in a Python-specific context). If the syntax is right, the semantics should fit. Of course, the syntax discussion is at least partially being driven by semantics. I am proposing that we be more explicit about the motivations behind our suggestions. On to a definition of terms, using block diagrams (useful for a blockhead like me :-): The parser is the basic component which takes raw text as input and produces a data structure as output:: +--------+ text --> | parser | --> parsed data structure +--------+ (internal, e.g. DOM tree) Depending on what we want to do with the data, we'll need output formatters:: +-----------+ structured data --> | formatter | --> formatted data (internal) +-----------+ (XML, HTML, TeX, info, etc.) A simple converter program would just need to link the two:: +------------------------------+ | converter | | +--------+ +-----------+ | text --> | | parser | --> | formatter | | --> formatted data | +--------+ +-----------+ | +------------------------------+ Now, when we get into auto-doc-generators (like HappyDoc, Crystal, pydoc, etc.), we need to add Python-specific knowledge to the mix:: +-------------------------------------------+ | Python Documentation Processor | | | | +---------------------------------------+ | | | operating logic: | | | | knowledge of Python syntax, docstring | | | | conventions and rules | | | +---------------------------------------+ | | | | +-----------------+ +------------+ | | | Structured Text | | output | | | | parser | | formatters | | | +-----------------+ +------------+ | | | Python-specific | | | | extensions | +------------------+ | | +-----------------+ | Python language | | | +--------------+ | services | | | | (potentially | | (parser.py, xml, | | | | other input | | inspect, etc.) | | | | parsers) | | | | | +--------------+ +------------------+ | +-------------------------------------------+ I don't know about others on this list, but I would like to use an ST-like markup language for more than just Python docstrings. I'd like to use it for documentation of all kinds, from how-to manuals to web pages (to books even, for crazies like me). When discussing Python docstrings, section hierarchy features (section titles) are less important than for writing a magazine article. This forum, of course, is specifically geared toward Python documentation. But am I unreasonable in thinking that this markup scheme has broader applications? See the Setext specification (http://www.bsdi.com/setext/) for its history; basically, it was used for a pre-web electronic newsletter, TidBits, whose texts were quite long. (Last year I wrote a chapter on Python for Wrox Press' "Professional Linux Programming". I would have been much happier using a complete ST-like markup than futzing around in MSWord.) I believe that the operating logic/rules/conventions ought to be separated conceptually and code-wise from the parser. The parser itself should be separated into generic and Python-specific parts. These things should not be tied together, at least not so strongly. Opinions? Flames? I've got my asbestos suit on! Thanks for reading my idle ramblings! /DG

2 1

(no subject)
by Lucas Bruand April 2, 2001

April 2, 2001

In Documenting Python, it is written page 11 : > \url{url} > A URL (or URN). The URL will be presented as text. In the HTML and PDF formatted versions, the URL will >also be a hyperlink. This can be used when referring to external resources. Note that many characters are special >to L A T E X and this macro does not always do the right thing. In particular, the tilde character () is mis-handled; >encoding it as a hex-sequence does work, use %7e in place of the tilde character. I don't understand what I should exactly write instead of tilde. ( because %7e counts as a remark nor does \symbol{"7e}) Thank in advance for helping a beginner in latex, Lucas Bruand

3 2

docstring signatures
by Edward D. Loper March 31, 2001

March 31, 2001

Guido, On doc-sig, we're trying to put together some standards/conventions for writing documentation strings, to propose in a PEP. (These conventions could then be used by all manner of docstring-related tools). Tibs said he thought that you wanted to require that such conventions include a "signature" for docstrings of callable objects, such as:: def primes(n): """ primes(n) -> lst -- Return a list of all primes in the range [2,n]. <more detailed description...> """ ... or:: def primes(n): """ primes(n) -> lst Return a list of all primes in the range [2,n]. <more detailed description...> """ ... However, it was unclear to me whether that would be affected any by the introduction of tools like inspect.py and pydoc.py's help function. In particular, much of the "signature" information can be obtained by calls to inspect methods; and there is a question of what to do if the "signature" disagrees with inspect. When designing our docstring conventions, should we include signatures, like the one given? Or can we feel free to put information about what is returned by the function, etc., in other places (e.g., under a "Returns: " section)? If you do want us to include signatures, is there somewhere where what they should look like is defined (e.g., whether you should say "primes(n) -> lst" or "primes(int) -> list")? -Edward

8 23

Re: [Doc-SIG] formalizing StructuredText
by Edward D. Loper March 31, 2001

March 31, 2001

> Well, prepare to be well miffed (ST has never supported differing > starting and ending quotes). So hey. Although now we have [...] (or "..."[...] or "...":[...] or whatever it really is). > It should (eventually) in the '"..."' text, but not in the URL itself. > This is actually a good reason to forbid apostrophe in URLs, Of course, you can't reasonably forbid '#' in URLs, so you'll have to put URL recognition before inline recognition *anyway*.. :) > and may > mean I need to put the URL recognition *before* literal recognition - > no, that won't work, 'cos then I couldn't say > > 'http://www.literal.org/' > > Hmm. This is a no-win situation, I'm afraid. Ah - no it's not, because > I'm requiring the user to escape spaces in a URL, and not to end with > "funny" characters, so it *should* actually come out in the wash - we'll > need to make some careful test cases... I think there's a serious problem here if we are allowing URLs to appear in arbitrary places. For example, consider:: foo://no#good bar://parse#for this. It seems perfectly reasonable for #good bar...# to be a literal.. But then it also seems reasonable for those to be urls.. Possible ways out: 1. Say that the opening '#' must have whitespace to its left, and the closing '#' must have whitespace to its right. Of course, that forbids saying things like #Object#s, but I guess I could live with that 2. Use some special demarkation for URLs! :) I'm for this, but am worried about trying to convince the STNG people, esp. if we're proposing using <..>.. Since they're currently saying that such things should be ignored. Of course, they're clearly wrong on that point, too, but it means that I'll have to argue 2 different points at once. :) Also, if we do this, we have to be sure to stress in the PEP/ST docs that math must go in literals like: 'x*y>z'. (Of course, we'll probably want to stress that anyway). Are there any objections in principle for using <...> to delimit URLs? (Other than that it will be hard to convince STNG people). If not, I think we should start trying to convince STNG people to use <...> for URLs, and to give up on ignoring <...> tokens. -Edward

3 9

docstring markup: assorted thoughts..
by Edward D. Loper March 30, 2001

March 30, 2001

I haven't been able to spend much time on doc-sig this week (and probably won't be able to spend much time this weekend, either). I've been reading, but having to hold myself back from responding, because otherwise I'd never get my other work done. :) But I did want to to make my positions on a few things clear... All that follows is just my opinion; feel free to disagree (preferably vocally rather than silently :) ). 1. I strongly believe that docstring markup should not include *any* "heuristic" rules. Put another way, I believe that if the markup will distinguish case A from case B, there should be a single, simple mechanism for distinguishing them. Examples of heuristic rules are "paragraphs that don't have normal indentation are treated as literals," and "ordered lists should be detected by one of umpteen cases, and only recognized when 2 elements with consective numbers occur in a row". These rules work on the principle of making misuse "improbable." They are a Bad Idea, because inevitably they will bite you. What should you do instead? Say things like: "literal paragraphs are marked with form XYZ; non-literal paragraphs must have normal indetation; otherwise, it's an error, and tools should complain".. (There's nothing wrong with defining syntax errors in a markup language!) 1.a. As a sidenote, this applies to URL detection, too. In particular, I think that we should take 1 of two courses: either give an explicit markup for URLs, or have the markup language say *nothing* about how to detect URLs, except maybe that tools may try to do it if they want. In particular, the *markup language specification* should not say things like "A url is anything matching big-ugly-regexp XYZ." I *believe* from what Guido's been saying that he would agree with me on this (that the markup should not have heuristic rules).. Care to confirm that? For me, context-sensitive use of punctuation is a borderline case.. For example, saying that '*' is a delimiter if it has whitespace one one side and non-whitespace on the other, but that it's an asterisk otherwise.. The most reasonable case for this can be made for apostrophes, which are used for 'quoting' and for contractions; 'but it's presumably easy to tell them apart.' I'm not *terribly* happy with context-sensitive punctuation, but I could certainly live with it. 2. There's often an attitude of "let's start off with something simple, and then add rules/heuristics later". This is a Bad Idea. Once you establish rules, people start using them. That makes it much harder to change them. For example, it's not nice to first tell people that '#' is not a markup character, let them write lots of docs, and then later tell them that you've decided that it's a markup character after all.. When we design this markup language, we should do it with the future in mind.. 3. As I said in <http://mail.python.org/pipermail/doc-sig/2001-March/001594.html>, I believe that the most fundamental feature that the markup language must have is the ability to distinguish natural language text from other text. The way I have been envisioning ST fitting into this is: ANYTHING that's not natural language should be quoted. Thus, the fact that < and > and * etc. are used for markup is not a problem, because they're never really used in natural language. If you want to use them, you quote them, like: 'x*y>z'. However, I get the impression that most people (including Guido?) think that quoting everything that's not natural language is too difficult. (There's also the somewhat orthoganal issue of how to escape your quote character, but let's ignore that for now). That's an opinion I can respect, although I *personally* would be quite willing to quote all non-natural-language text when writing docstrings.. And I *personally* really don't care what character we use to do that quoting (apostrophy, backquote, hash, etc.), as long as the contexts it's used in are not contexts that that character would ever be used in for NL (=natural langauage). Perhaps most people would argue that there are really 3 categories here: natural language text; "literal text"; and everything else.. Where "everything else" is not NL, but should still be rendered in non-monospace, etc. With this view, it *does* make more sense to try to severly limit how many "special" characters there are, since "everything else" is likely to use every character you can think of, and then some. So I think that we should do one of the following: * decide that we don't mind quoting all non-natural-language text, and pick a quote character. Try to keep markup characters to a minimum, but not worry too much if we end up with more than 2 or 3 total. * decide that we want the 3 categories, and much more carefully pick a quote character, or maybe one or two "markup characters" that will be used for all inline markup (e.g., '<>' in POD). Note that, for the most part, we all agree that using '::' to mark the beginning of literal blocks, and indentation to mark their end, is acceptable.. So really we just need to worry about how to mark inline non-natural-language text and/or inline literal text. 4. Once we figure out how to mark inline non-NL text, we can think some about my second most fundamental feature: the ability to flag semantic fields, like descriptions of specific parameters, or of the return value, or of exceptions that can be raised. Obviously, these semantic fields will be very useful to tools. Javadoc does this with forms like:: @param p A planet @return The size of the given planet Most people think we should use trailing colons instead of leading at-signs, so we might have something like:: param q: The question of life, the universe and everything return: The answer to life, the universe, and everything (or 'arg' or 'argument' or whatever we decide) Then there's just the question of marking the scope of such expressions.. I believe that the most reasonable thing to do is to use indentation, since we're using "python style colons" anyway. So you can say either:: param x: description of x... or:: param x: description of x... more description of x.. * maybe even a bulleted list. of course, we'll have to make some decisions about where blank lines are required, etc., and how to distinguish this from the use of ':' in natural language.. (and no, I won't be happy with any heuristic rules for doing this. :) ). But I believe that this is on the right track. Another alternative, if we decide that we like description lists, is something like:: Parameters: x -- description of x y -- description of y But I think that we should *only* do that if you can also do it in text, because otherwise people will get confused. 5. Currently, I'm leaning towards making the markup pretty lightweight.. I.e., it wouldn't contain too many other features. Features it might contain inlcude lists, some sort of sectioning mechanism, some sort of emphasis mechanism, some way of doing endnotes, and some way of marking URLs. ----- In case people (incl Guido?) are interested, the following page contain some essays/etc. I wrote on formatted doc strings.. www.cis.upenn.edu/~edloper/pydoc Also, the PEP I've been writing gives (what I hope is) a clear definition of what we have been coming up with over the last month.. Unfortunately, it's not complete (it's currently about 700 lines, and probably about 1/2 complete..), and I've put it on hold for now, both because I've become busy, and because I think we may end up wanting to change a lot of it. But, as I've said before, if anyone wants a copy of what it says so far, send me email and I'll be happy to send you a copy.

4 5

Where to go from here
by Edward D. Loper March 30, 2001

March 30, 2001

Guido's input has raised some questions about whether we're going in the right direction... But he's made it clear that he'll keep an open mind, and seriously consider any real specs we come up with. So I propose we do the following: - continue trying to come up with a concrete, formal spec - drop the idea of maintaining compatibility with STNG, for now. Once STNG sees how much cooler our markup language is, we can convert them. ;) Maybe we should even come up with a new name, so that other people who have become embittered with STclassic won't take it out on us. :) - STminus will focus purely on coming up with a formal description, and drop its goals of unifying STNG/STpy. - Focus on the goal of making a *real* markup language that is *lightweight* and simple to read/write. - Once we have a real specification (hopefully in a couple weeks), we can talk to Guido/others about whether it's acceptable. It's unreasonable to expect Guido to make judgements when the ST stuff is in the state of flux it's in now. Dropping STNG compatibility will allow us to consider a number of options that I hadn't brought up before.. For example, I think we might want to replace '--' with '---' as the description list indicator, since people *do* use '--' in text (I know I do, and apparently Guido does too). And I think we should drop 'o' as a bullet character. etc.. As for colorization, java-mode does just fine colorizing javadoc comments, so I don't see how it's a problem *in principle*, just a problem of someone figuring out what to tell emacs (I'm sure emacs could be told to colorize tripple-quoted-strings correctly if someone really wanted to figure out how to.. I've just been using the work-around of backslashing all double quotes in tripple-quoted-strings, which doesn't affect their value, and makes them colorize correctly) -Edward

4 8

Re: [Doc-SIG] Re: POD (resend)
by Ken Manheimer March 30, 2001

March 30, 2001

Whoops - somehow that last message got sent before i was done, as a result of switching desktops. I wanted to point out that while i'm not crazy about the processed, plain-text pod output (*.pod), i really dislike the stuff in the source. Take a look in particular at the last paragraph in my excerpt - such markup may be a necessary evil, but i'm really hoping we don't have to put something of the sort in function docstrings. Ken 8<---------------------------------------------------------------->8 =head1 NAME Apache - Perl interface to the Apache server API =head1 SYNOPSIS use Apache (); =head1 DESCRIPTION This module provides a Perl interface the Apache API. It is here mainly for B<mod_perl>, but may be used for other Apache modules that wish to embed a Perl interpreter. We suggest that you also consult the description of the Apache C API at http://www.apache.org/docs/. =head1 THE REQUEST OBJECT The request object holds all the information that the server needs to service a request. Apache B<Perl*Handler>s will be given a reference to the request object as parameter and may choose update or use it in various ways. Most of the methods described below obtain information from or updates the request object. The perl version of the request object will be blessed into the B<Apache> package, it is really a C<request_rec*> in disguise. =over 4 =item Apache->request([$r]) The Apache->request method will return a reference to the request object. B<Perl*Handler>s can obtain a reference to the request object when it is passed to them via C<@_>. However, scripts that run under B<Apache::Registry>, for example, need a way to access the request object. B<Apache::Registry> will make a request object available to these scripts by passing an object reference to C<Apache-E<gt>request($r)>. If handlers use modules such as B<CGI::Apache> that need to access C<Apache-E<gt>request>, they too should do this (e.g. B<Apache::Status>).

6 9

RE: [Doc-SIG] using the same delimiter on the left and right..
by Goodger, David March 30, 2001

March 30, 2001

Guido van Rossum writes: > In many fonts, backtick is hard to distinguish from apostrophe! In many fonts, '1' (one) is hard to distinguish from 'l' (el), or 'O' (upcase oh) from '0' (zero). Programmers (& other docstring writers) ought to choose a programmer-friendly font. ;-> /DG

3 2

RE: [Doc-SIG] New document - pytext-fat
by Goodger, David March 30, 2001

March 30, 2001

Guido van Rossum wrote: > - The example with a paragraph consisting of *just* "::" should render > that as a single colon, to be consistent. If you think this should > be special-cased, you need to explain why It's required to begin a docstring (or section) with a literal block:: """:: s = "This is a literal block" That's how you assign a string... """ I inserted it in my reStructuredText spec for completeness, and it is consistent: A paragraph which which ends with two colons ('::') signifies that all following **indented** text blocks are code blocks. ... When '::' is immediately preceeded by whitespace, both colons will be removed from the output. When text immediately preceeds the '::', *one* colon will be removed from the output, leaving only one (i.e., '::' will be replaced by ':'). When '::' is alone on a line, it will be completely removed from the output; no empty paragraph will remain. The difference between 'text::' and 'text ::' is because sometimes we don't want a colon at the end of the preceeding paragraph. '::' all by itself in a paragraph is simply a degenerate case of '::' preceeded by whitespace. /DG

2 1