From martin@loewis.home.cs.tu-berlin.de Thu Nov 2 05:04:22 2000 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Thu, 2 Nov 2000 06:04:22 +0100 Subject: [Doc-SIG] Re: [XML-SIG] Error in pyexpat docs In-Reply-To: <200010291916.MAA16420@localhost.localdomain> (uche.ogbuji@fourthought.com) References: <200010291916.MAA16420@localhost.localdomain> Message-ID: <200011020504.GAA00898@loewis.home.cs.tu-berlin.de> > The wrong, "xml.parsers.expat" is the first indicator that there might be a > problem, yet I took the docs at their word and wrapped the call to ParseFile > in a blanket try/except, only to find that no exception of any sort is ever > raised by ParseFile. > It turns out that ParseFile actually returns 0 on error, returning 1 > otherwise. > The first matter is that the code and the docs need to be reconciled. > However, I would _much_ rather prefer that things were as in the docs. Thanks for the report. What do you mean with 'the wrong, "xml.parsers.expat"'? There really is an exception xml.parsers.expat.error, and it really is raised from xml.parsers.expat (aka pyexpat). It turns out that it is only raised for Parse, not for ParseFile, which is a bug introduced when the return value of these functions was changed from a number to an exception. In the PyXML CVS, I have corrected that error. As a result, PyXML 0.6.2 will build its own pyexpat.c even for Python 2.0, which will implement the documentation. Regards, Martin From akuchlin@mems-exchange.org Fri Nov 3 19:45:16 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 03 Nov 2000 14:45:16 -0500 Subject: [Doc-SIG] Documentation from docstrings Message-ID: I haven't been following this area recently, so... what are the current tools used to generate HTML (or TeX/XML/whatever) from docstrings? There's pythondoc, ?!ng's htmldoc... any others? What do most people use for this task? --amk From mal@lemburg.com Fri Nov 3 20:06:53 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 03 Nov 2000 21:06:53 +0100 Subject: [Doc-SIG] Documentation from docstrings References: Message-ID: <3A031ADD.BE30C99@lemburg.com> Andrew Kuchling wrote: > > I haven't been following this area recently, so... what are the > current tools used to generate HTML (or TeX/XML/whatever) from > docstrings? There's pythondoc, ?!ng's htmldoc... any others? What do > most people use for this task? There's my old doc.py. It produces the classical two frames HTML output. Good for quick shot Python project documentation... I've turned away from these tools though and now use a combination doc-string extractor -> StarOffice import -> GhostScript PDF export to do documentation. This allows better hand-editing of the generated output, since doc-strings usually don't suffice when it comes to documenting code. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jhauser@ifm.uni-kiel.de Fri Nov 3 20:31:28 2000 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Fri, 3 Nov 2000 21:31:28 +0100 (CET) Subject: [Doc-SIG] Documentation from docstrings In-Reply-To: <3A031ADD.BE30C99@lemburg.com> References: <3A031ADD.BE30C99@lemburg.com> Message-ID: <20001103203128.15101.qmail@lisboa.ifm.uni-kiel.de> There is also HappyDoc. One big difference is, that it does not import the modules and extracts the docstring but parses the actual code. So it is not suited to document extension modules. Another thing is, that one can write new backend to format the doc strings. http://happydoc.sourceforge.net/ Then there is also Crystal at http://www.btinternet.com/~tratt/comp/python/crystal/ These two are relative mature. I always wonder, why especially new python users find the pythondoc module, which seems to have problems. Is it linked in a popular place? HTH, __Janko M.-A. Lemburg writes: > Andrew Kuchling wrote: > > > > I haven't been following this area recently, so... what are the > > current tools used to generate HTML (or TeX/XML/whatever) from > > docstrings? There's pythondoc, ?!ng's htmldoc... any others? What do > > most people use for this task? > > There's my old doc.py. It produces the classical two frames > HTML output. Good for quick shot Python project documentation... > > I've turned away from these tools though and now use a combination > doc-string extractor -> StarOffice import -> GhostScript PDF export > to do documentation. This allows better hand-editing of the generated > output, since doc-strings usually don't suffice when it > comes to documenting code. > From Mike Miller" >These two are relative mature. I always wonder, why especially new >python users find the pythondoc module, which seems to have >problems. Is it linked in a popular place? Yes, it's linked off the doc-sig page on python.org. Happydoc isn't. Although I would argue happydoc is better overall, or at least better maintained. No offence to pythondoc's author intended :-) - Mike From fdrake@acm.org Sat Nov 4 05:35:06 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 4 Nov 2000 00:35:06 -0500 (EST) Subject: [Doc-SIG] Documentation from docstrings In-Reply-To: <200011032234.OAA21811@plxw0015.pdx.intel.com> References: <200011032234.OAA21811@plxw0015.pdx.intel.com> Message-ID: <14851.40970.45467.948137@cj42289-a.reston1.va.home.com> Mike Miller writes: > Yes, it's linked off the doc-sig page on python.org. Happydoc isn't. > Although I would argue happydoc is better overall, or at least better > maintained. No offence to pythondoc's author intended :-) Those pages haven't been updated much lately, in part due to all the moves and politics and crud. I'll try and get this fixed shortly. We expect it will be easier to maintain before too long. Sorry for not getting HappyDoc listed sooner! If it isn't there by Monday, feel free to email me a reminder. -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From Moshe Zadka Mon Nov 6 10:46:54 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 6 Nov 2000 12:46:54 +0200 (IST) Subject: [Doc-SIG] Docstrings Format Message-ID: There's a new version of PEP-0216 (docstring format) in the CVS repository. I would love to hear comments. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From klm@digicool.com Mon Nov 6 17:38:27 2000 From: klm@digicool.com (Ken Manheimer) Date: Mon, 6 Nov 2000 12:38:27 -0500 (EST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: <20001106170227.4D63F1D0BA@dinsdale.python.org> Message-ID: On Mon, 6 Nov 2000, my doc-sig digest included this message from Moshe Zadka : > There's a new version of PEP-0216 (docstring format) in the CVS > repository. I would love to hear comments. We seem to have made little progress on any fronts on this docstring stuff. I'm exasperated that StructuredText was dismissed, because i think it was the best candidate as the basis for the docstring format. Several of pep-0216's docstring format goals are chief among StructuredText's motivating principles, and i think it does a better job of satisfying them, as is, than any other format i've encountered: 1. It must be easy to type with any standard text editor. 2. It must be readable to the casual observer. 4. It must contain sufficient information so it can be converted to any reasonable markup format. It seems that a 1/2 year or so ago (?), david ascher was on the verge of endorsing the StructuredText approach as the markup, provided some tailoring could be done, but moshe dismissed it, in pursuit of a new markup that failed to be accepted by many folks. Now, even if StructuredText were to be put back on the table, it's not clear if i or someone else at dc would have time to adapt it to the purpose. (It so happens i'm going to be working up a simple cheat sheet, and probably fixing some problems, for wiki use, so that may be helpful.) Out of curiousity, is it worth talking about StructuredText at this point, or would it still be considered unacceptable? If it does seem unacceptable, could you reiterate the objections? I don't promise a quick reply, but i think it would be a major win, and would be willing to expend some effort to see it find its place. -- Ken klm@digicool.com From Moshe Zadka Mon Nov 6 18:19:39 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 6 Nov 2000 20:19:39 +0200 (IST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: On Mon, 6 Nov 2000, Ken Manheimer wrote: > We seem to have made little progress on any fronts on this docstring > stuff. I'm exasperated that StructuredText was dismissed, because i think > it was the best candidate as the basis for the docstring > format. Several of pep-0216's docstring format goals are chief among > StructuredText's motivating principles, and i think it does a better job > of satisfying them, as is, than any other format i've encountered: > > 1. It must be easy to type with any standard text editor. > 2. It must be readable to the casual observer. > 4. It must contain sufficient information so it can be converted > to any reasonable markup format. I disagree with "4": I cannot mark up things like "this is a Python variable". If you show me how to extend StructuredText for these kind of things, it would definitely get a chance. Please have a look at the PEP and see the exact markup goals I've outlined. Of course, if you do not agree with the goals, you can say that too... > Now, even if > StructuredText were to be put back on the table, it's not clear if i > or someone else at dc would have time to adapt it to the purpose. Well, if these are just tailorings and we have the proposal ready, it might be that people outside DC can help. > Out of curiousity, is it worth talking about StructuredText at this point, > or would it still be considered unacceptable? Nothing is "unacceptable", but as I've explained earlier, StructuredText as is has not enough strength to do arbitrary Python-specific markup. > i think it would be a major win, and would be willing to expend > some effort to see it find its place. I'm happy to hear that. I'm not against any markup. Though of course I originally had in mind something closer to my original proposal, I'm aware of the feelings that it generated, and if there is something better the rest of the world likes, I'll live with it. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From klm@digicool.com Mon Nov 6 19:16:02 2000 From: klm@digicool.com (Ken Manheimer) Date: Mon, 6 Nov 2000 14:16:02 -0500 (EST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: On Mon, 6 Nov 2000, Moshe Zadka wrote: > On Mon, 6 Nov 2000, Ken Manheimer wrote: > > > 1. It must be easy to type with any standard text editor. > > 2. It must be readable to the casual observer. > > 4. It must contain sufficient information so it can be converted > > to any reasonable markup format. > > I disagree with "4": I cannot mark up things like "this is a Python > variable". If you show me how to extend StructuredText for these > kind of things, it would definitely get a chance. Please have > a look at the PEP and see the exact markup goals I've outlined. Of > course, if you do not agree with the goals, you can say that too... StructuredTextNG was specifically developed to be extensible, and has some substantial documentation (i hear) to get you going. (The documentation covers extending the expressions that are recognized, but not yet creating outputters, responsible for rendering whatever presentation you want.) See: http://www.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG > Nothing is "unacceptable", but as I've explained earlier, StructuredText > as is has not enough strength to do arbitrary Python-specific markup. See whether StructuredTextNG's extensibility would give you what you want. -- Ken klm@digicool.com From Juergen Hermann" Message-ID: On Mon, 6 Nov 2000 20:19:39 +0200 (IST), Moshe Zadka wrote: >I'm happy to hear that. I'm not against any markup. Though of course >I originally had in mind something closer to my original proposal, I'm >aware of the feelings that it generated, and if there is something >better the rest of the world likes, I'll live with it. Does anyone of you know Doxygen? It uses extended Javadoc markup for C, = C++ and IDL. By using a preprocessor, some other languages already were enabled to use Doxygen. Bye, J=FCrgen From MarkH@ActiveState.com Mon Nov 6 23:05:05 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 7 Nov 2000 10:05:05 +1100 Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: > > 4. It must contain sufficient information so it can be converted > > to any reasonable markup format. > > I disagree with "4": I cannot mark up things like "this is a Python > variable". I am not with you here. Whether you agree or not about (4) seems quite independent of the capability to add certain markups? I agree with (4), but also agree with your requirement for that markup capability. > If you show me how to extend StructuredText for these > kind of things, it would definitely get a chance. I hate to tell you this Moshe, but I would speculate that the general feeling on this SIG (assuming it hasnt changed over the last 12 months) would be that StructuredText definitely has a chance regardless ;-) > Please have a look at the PEP and see the exact markup goals > I've outlined. Of course, if you do not agree with the goals, > you can say that too... I agree with the goals in general, but agree with Ken's new goal. As a nit-pick: 3. It must not contain information which can be deduced from parsing the module. maybe needs to be reworded slightly, as docstrings generally _do_ include redundant information such as the function and parameter names, even though these are obviously able to be deduced. However, your PEP as it stands doesnt actually _say_ anything! As the PEP itself says, you have rehashed what as already been discussed ad-nauseam right here. Without a specific markup proposal it doesn't seem to add much of value, and as soon as you try and extend it to a specific markup proposal, we are back in the exact same position as 6 months ago. I would vote that we explicitly state a goal is to use StructuredTextNG if possible, and that the focus of the PEP then become two-fold: * Identifying and fixing limitations in StructuredTextNG, as relevant to Python docstrings. * A specific, concrete markup proposal for Python docstrings (ie, narrow the flexibility of StructuredTextNG to a reasonable subset for Python that docstring tools can rely on.) Pity-noone-asked-me-to-vote-though ly, Mark. Mark. From Moshe Zadka Tue Nov 7 03:27:37 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Nov 2000 05:27:37 +0200 (IST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: [Moshe Zadka] >I'm happy to hear that. I'm not against any markup. Though of course >I originally had in mind something closer to my original proposal, I'm >aware of the feelings that it generated, and if there is something >better the rest of the world likes, I'll live with it. [Juergen Hermann] > Does anyone of you know Doxygen? It uses extended Javadoc markup for C, > C++ and IDL. By using a preprocessor, some other languages already > were enabled to use Doxygen. On the face of it, JavaDoc is not an option because it's too married to HTML. However, if you have a link for Doxygen, I'd like to see it. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Tue Nov 7 03:33:06 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Nov 2000 05:33:06 +0200 (IST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: On Tue, 7 Nov 2000, Mark Hammond wrote: > I agree with the goals in general, but agree with Ken's new goal. As a > nit-pick: > 3. It must not contain information which can be deduced from parsing > the module. > > maybe needs to be reworded slightly, as docstrings generally _do_ include > redundant information such as the function and parameter names, even though > these are obviously able to be deduced. Well, yes, a rewording is in order. What I meant that it must not be the case that you *have* to include information which can be deduced from parsing the module. > However, your PEP as it stands doesnt actually _say_ anything! As the PEP > itself says, you have rehashed what as already been discussed ad-nauseam > right here. That's right -- but I seem to remember we had some discrepencies about the *goals* of the docstring format. As long as we're sure we agree on the goals, we can move on with the format. > I would vote that we explicitly state a goal is to use StructuredTextNG if > possible, and that the focus of the PEP then become two-fold: > * Identifying and fixing limitations in StructuredTextNG, as relevant to > Python docstrings. > * A specific, concrete markup proposal for Python docstrings (ie, narrow the > flexibility of StructuredTextNG to a reasonable subset for Python that > docstring tools can rely on.) Well, in the time-honoured Guido tradition of not listening to votes but to arguments, I have to say yours makes a lot of sense. That's two people who already suggested ST-NG, so I'll have to take a look at it. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Tue Nov 7 03:59:33 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Nov 2000 05:59:33 +0200 (IST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: On Mon, 6 Nov 2000, Ken Manheimer wrote: > StructuredTextNG was specifically developed to be extensible, and has some > substantial documentation (i hear) to get you going. (The documentation > covers extending the expressions that are recognized, but not yet creating > outputters, responsible for rendering whatever presentation you > want.) OK, I've had a look around the ZWiki site, and ST-NG seems very nice. Now we need to decide about how to meet markup requirements "a" and "b". To save you the time of going to the PEP: marking up "this is a Python ``something''", and "this is a Python class/function/....". In addition, there must be some escape mechanism to stop ST-NG from recognizing constructs. I think that was a big part of my problem with ST, and is related to goal "5". -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From fdrake@users.sourceforge.net Tue Nov 7 04:05:34 2000 From: fdrake@users.sourceforge.net (Fred L. Drake) Date: Mon, 6 Nov 2000 20:05:34 -0800 Subject: [Doc-SIG] [development doc updates] Message-ID: <200011070405.UAA32052@orbital.p.sourceforge.net> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ From tim_one@email.msn.com Tue Nov 7 07:55:03 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 7 Nov 2000 02:55:03 -0500 Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: [Moshe Zadka] > ... > That's two people who already suggested ST-NG, so I'll have to take > a look at it. I don't know anything about ST-NG. I do know that my current employer likes it a lot, and they can buy my vote without even asking for it . Seriously, ST has been in production use at DC for years, so I'm with the seeming consensus that it wins by default. It's a pity that David Ascher had to drop his work on tweaking it in order to devote his life to finishing rich comparisons . nothing's-trivial-ly y'rs - tim From Moshe Zadka Tue Nov 7 08:26:31 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Nov 2000 10:26:31 +0200 (IST) Subject: [Doc-SIG] Docstrings Format In-Reply-To: Message-ID: On Tue, 7 Nov 2000, Tim Peters wrote: [Moshe Zadka] > ... > That's two people who already suggested ST-NG, so I'll have to take > a look at it. [Tim Peters] > I don't know anything about ST-NG. I do know that my current employer likes > it a lot, and they can buy my vote without even asking for it . Is a vote for ST-NG a vote for Bush too? [Tim Peters] > Seriously, ST has been in production use at DC for years, so I'm with the > seeming consensus that it wins by default. OK, as soon as my home computer finishes upgrading dpkg and I can connect to it for more then 1 Byte/sec, I'll add references to ST-NG in the PEP. Note that this means that we have to address Mark H's comments: -- extending ST-NG for Python's use (see goals "a" and "b" in the PEP) -- enforcing meta-ST-NG rules, so the collection of all docstrings will form a valid ST-NG document (or perhaps not. Is this the next doc-flamewar?) documentation's-fun!-ly y'rs, Z. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Tue Nov 7 09:12:14 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Nov 2000 11:12:14 +0200 (IST) Subject: [Doc-SIG] PEP-0216 Message-ID: A new version is in CVS archive, and should soon be in the SF pages. Please comment. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Juergen Hermann" Message-ID: On Tue, 7 Nov 2000 05:27:37 +0200 (IST), Moshe Zadka wrote: >On the face of it, JavaDoc is not an option because it's too married to= >HTML. However, if you have a link for Doxygen, I'd like to see it. I meant the Javadoc markup, not the tool. Doxygen can be found at http://www.stack.nl/~dimitri/doxygen/index.html and a sample output at http://xml.apache.org/xalan-c/apidocs/index.html It can produce HTML, LaTeX and RTF. Ciao, J=FCrgen -- J=FCrgen Hermann, Developer (jhe@webde-ag.de) WEB.DE AG, http://webde-ag.de/ From fdrake@acm.org Tue Nov 7 16:05:38 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 7 Nov 2000 11:05:38 -0500 (EST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: References: Message-ID: <14856.10322.381862.588662@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > A new version is in CVS archive, and should soon > be in the SF pages. Please comment. I think the "concensus" for StructuredText isn't right; I'm aware of no such decision. I think Tim's intent was that we still consider StructuredText (NG or otherwise). My own inclination is that it isn't quite strong enough, but a variant would probably work just fine. (Speaking strictly of the markup language, not the support code. I think the code is irrelevant at this point.) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From MarkH@ActiveState.com Tue Nov 7 22:33:31 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Wed, 8 Nov 2000 09:33:31 +1100 Subject: [Doc-SIG] PEP-0216 In-Reply-To: <14856.10322.381862.588662@cj42289-a.reston1.va.home.com> Message-ID: > Moshe Zadka writes: > > A new version is in CVS archive, and should soon > > be in the SF pages. Please comment. > > I think the "concensus" for StructuredText isn't right; I'm aware of > no such decision. There has been no such decision, but it seems apparent that DA some time ago, and over the last few days myself, Ken and Tim have come out in favour of it, with no real dissentions. A few of those mails stated that there "appeared to be a consensus" on this, and no one argued. > I think Tim's intent was that we still consider > StructuredText (NG or otherwise). I was saying something similar, but a little stronger: "we should prove that StructuredText(NG) is _not_ suitable before looking at alternatives". > My own inclination is that it isn't quite strong enough, but a > variant would probably work just fine. Agreed! But let's just move forward with _something_. I first started looking at documentation tools about 3 years ago, and wrote a number of docstrings for the purpose. In that entire 3 year period I have not been able to generate anything. Sure, I could invent yet another solution of my own, but I don't want to - that is one of the main points of this SIG. We have to start somewhere, and soon. I could not sit through another doc-sig meeting at the next conference arguing the same things we did 3 years ago, 2 years ago, and early this year. Moshe is to be commended to taking ownership of this PEP, now we need to follow it through. Mark. From Moshe Zadka Wed Nov 8 06:39:22 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 8 Nov 2000 08:39:22 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: On Wed, 8 Nov 2000, Mark Hammond wrote: > There has been no such decision, but it seems apparent that DA some time > ago, and over the last few days myself, Ken and Tim have come out in favour > of it, with no real dissentions. A few of those mails stated that there > "appeared to be a consensus" on this, and no one argued. Let's put it this way (a paraphrase of what Mark said): ST-NG is the current strawman. If we knock it over, we'll revert to something like my original proposal, only changed to cause less antagonism. Current questions about ST-NG: - Does it support proper escaping of meta-characters? If I want to type *, or ', can I? - Does my proposal about Python symbols identified as [:] make sense? How hard would it be to extend the current code to do just that? - What high level rules do we wish to impose on each docstring? In particular, how to document methods that don't exist? (E.g., __getattr__ or instance variables that look like methods) > We have to start somewhere, and soon. I could not sit through another > doc-sig meeting at the next conference arguing the same things we did 3 > years ago, 2 years ago, and early this year. You're so right. The meeting was a total failure -- we got absolutely nothing. If there are any objections/questions other then the ones I outlined, please let me know -- either by private e-mail or on this sig. (As mentioned, I prefer to be CCed on mail to the sig about the PEP -- I'll read it sooner that way) -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tim_one@email.msn.com Wed Nov 8 20:20:52 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 8 Nov 2000 15:20:52 -0500 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: [Mark Hammond] > ... > I was saying something similar, but a little stronger: "we should > prove that StructuredText(NG) is _not_ suitable before looking at > alternatives". Exactly. It's been in use for years, the last time around its creators expressed eagerness to compromise (if needed) to make everyone happier, yet nothing happened. Neither has a credible alternative appeared. Enough is enough. preparing-to-celebrate-the-10-year-anniversary-of-docstring- uselessness-ly y'rs - tim From tony@lsl.co.uk Thu Nov 9 10:06:27 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Thu, 9 Nov 2000 10:06:27 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <001b01c04a34$b95d2ce0$f05aa8c0@lslp7o.int.lsl.co.uk> Tim Peters wrote: > [Mark Hammond] > > ... > > I was saying something similar, but a little stronger: "we should > > prove that StructuredText(NG) is _not_ suitable before looking at > > alternatives". > > Exactly. It's been in use for years, the last time around its > creators expressed eagerness to compromise (if needed) to make > everyon happier, yet nothing happened. Neither has a credible > alternative appeared. Enough is enough. My feelings exactly. We *nearly* got something going last time round, which got derailed for whatever reason (partly excessive pedantry, I suspect, mea culpa amongst others). We can guarantee that no solution will satisfy everyone (I can be awkward on demand if it looked like it would), so an existing solution that gets most of the way and that we have some chance of modifying if we're willing to help sounds like a Good Thing to me. So I say "go for it". Let's make a statement. Meanwhile, if people can be a *little* patient, I'm writing up a summary of what the (in general, pedantic) worries about StructuredText were (NG addresses some of them already), and also trying to firm up the *definition* of StructuredText (the definition as written is a Bit Vague!). This is likely to take another couple of days, because I have about one-and-a-half hours free time a day (we have two small children and my partner commutes two hours or more each way each day), and some of that time gets used for things like ironing, reading, collapsing in front of the TV, and so on. I would hope that this would act as a basis for "now we've got a tool/specification, where would we like to take it". BTW, it's not as if we're short of tools: StructuredText itself pythondoc (when Starship is up) Crystal Happy-Doc all of which support at least *some* variant of StructuredText, I believe. And we mustn't forget Ka-Ping Yee's wonderful htmldoc, for coping with software that isn't marked up. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Are we there yet? My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From Moshe Zadka Thu Nov 9 10:15:18 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 9 Nov 2000 12:15:18 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <001b01c04a34$b95d2ce0$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: On Thu, 9 Nov 2000, Tony J Ibbs (Tibs) wrote: > We can guarantee that no solution will satisfy > everyone (I can be awkward on demand if it looked like it would), so an > existing solution that gets most of the way and that we have some chance of > modifying if we're willing to help sounds like a Good Thing to me. Right on! > Meanwhile, if people can be a *little* patient, I'm writing up a summary of > what the (in general, pedantic) worries about StructuredText were (NG > addresses some of them already), So please leave those worries out! We're going with ST-NG unless 1) Someone finds a *big* problem with it. 2) Guido dislikes it, and his coworkers don't physically beat him into silence At the moment, there are problems with ST-NG, which I've already written about in a previous e-mail. > and also trying to firm up the *definition* > of StructuredText That would help a lot! I would like to include a definitive reference inside the PEP itself, so it could be integrated into Fred's "Documenting Python". BTW, the current output format should be th Pythonish LaTeX which is already used. A future additional output format is whetever is used for external documentation (probably XML/SGML application). > BTW, it's not as if we're short of tools: > > StructuredText itself > pythondoc (when Starship is up) > Crystal > Happy-Doc We're short on *one* tool, which is strong enough to be "the" answer. In short, I'd love to see some definitive Python-extension of ST-NG, which would let us write documentation! If you can do a specification, that would be great! patient-is-no-different-then-lazy-here-ly y'rs, Z. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tony@lsl.co.uk Thu Nov 9 10:48:03 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Thu, 9 Nov 2000 10:48:03 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <001c01c04a3a$88e5e420$f05aa8c0@lslp7o.int.lsl.co.uk> I wrote: > > Meanwhile, if people can be a *little* patient, I'm writing > > up a summary of what the (in general, pedantic) worries about > > StructuredText were (NG addresses some of them already), Moshe Zadka wrote: > So please leave those worries out! We're going with ST-NG unless > > 1) Someone finds a *big* problem with it. > 2) Guido dislikes it, and his coworkers don't physically beat him into > silence I still want a summary of what the perceived problems have been, since I see this as useful for the following reasons: a. It is history. History is a Good Thing, since we can say "been there, done that, already decided to ignore it." (and, btw, here is *why*). b. It is useful as a potential basis for future developments - if we embrace StructuredTextNG, I assume that we are going to want to join the development/maintainence effort, and it would be useful to have a "group consensus" to start that off from. BUT it is an aside, and it is not as important as the "definition" of current NG: > > and also trying to firm up the *definition* > > of StructuredText > > That would help a lot! I would like to include a definitive reference > inside the PEP itself, so it could be integrated into Fred's "Documenting > Python". By the way, it would help me a lot if someone with CVS could extract the current state of StructuredTextNG from the Zope2 archive for me (see http://www.zope.org//Members/jim/StructuredTextWiki/NGReleases for details) so that I could look at what it currently does. > BTW, the current output format should be th Pythonish LaTeX which is > already used. A future additional output format is whetever is used for > external documentation (probably XML/SGML application). This is the sort of thing we should worry about *after* adopting a mechanism (i.e., we take what it gives us, and if that ain't enough, *then* we add more functionality). > > BTW, it's not as if we're short of tools: > > > > StructuredText itself > > pythondoc (when Starship is up) > > Crystal > > Happy-Doc > > We're short on *one* tool, which is strong enough to be "the" answer. But precious close... > In short, I'd love to see some definitive Python-extension of ST-NG, > which would let us write documentation! If you can do a specification, > that would be great! Heh, pedantry-are-us (it's *loads* more fun than standards work, except when there's only two of you doing the standard!). > patient-is-no-different-then-lazy-here-ly y'rs, Z. I like that... Tibs (whose sanity can be judged by the following signature) -- BikeCode0.2 http://www.tibsnjoan.co.uk/bikecode.html P: [Tibs] Tc B10 K:++ i29:30" h1.65m n1960 H+:~ v~ A+ M+ Rg- B: [AnthroTech] 3tRu U1c w37" Wr19:406 Mfr SAf bDh[Sachs]:C G3x7 8s Lrr1B Cb[Michael] VjsX col[MidnightBlue] T: [BurleyD'Lite] 2c2[Thomas] f++ VsX My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From klm@digicool.com Thu Nov 9 18:12:21 2000 From: klm@digicool.com (Ken Manheimer) Date: Thu, 9 Nov 2000 13:12:21 -0500 (EST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <20001109170120.CB9501D13B@dinsdale.python.org> Message-ID: Well, it's remarkable to see StructuredText really considered after all this time. I'm sorry that i probably won't have time to do more than kibitz. Moshe wrote: > At the moment, there are problems with ST-NG, which I've already > written about in a previous e-mail. Which i understand to be: | Current questions about ST-NG: | - Does it support proper escaping of meta-characters? If I want to | type *, or ', can I? Not as it stands. We will need for it to do so sometime soon, to interoperate with, eg, conversion back from other formats (eg, xml) to StructuredText. What we haven't done is arrive at a representation for escaping that conforms to the StructuredText principle of being "natural" when viewed in source form, as well as when rendered. (This is a very central principle, and i think one that can enable StructuredText to be used to transcend the impedence mismatch between rendered and source text.) One idea i'm toying with is to have (") double quotes escape simple constructs they contain. This is obvious for, eg, the code construct:: One expresses code snippets by containing text within single quotes, like "'def halting_test(code):'". For me, at least, it makes sense in the raw text passage. I think there would be a problem with, eg, code that contains double quotes. Perhaps it's ok to have them bind tightly, and use them closer together:: One expresses code snippets by containing text within single quotes, like "'"def halting_test(code):"'". A bit ugly, but i think the intention is apparent and credible. Likewise, for bullets:: "*" A non-bullet sentence following an asterisk. Some names in foreign languages can have funny characters, like !xabu and "*"dingbop"*" (Note - i *believe* that *example* text - blocks introduced by a paragraph ending in a '::' - should have all ST interpretation disabled, so the above examples would show the double quotes as well as the things they are quoting - if the double quote convention existed.) In any case, it will take some coding to implement some escape mechanism. | - Does my proposal about Python symbols identified as [:] | make sense? How hard would it be to extend the current code to | do just that? I don't understand the question, and don't have the time to follow back a second level of indirection to track down your proposal. (And i don't think i'll have time to do the legwork to investigate, if i *did* understand the question:-( ) | - What high level rules do we wish to impose on each docstring? In | particular, how to document methods that don't exist? (E.g., | __getattr__ or instance variables that look like methods) I also don't quite understand this intent of this question, but i suspect it's not a StructuredText issue, per se. ? Ken klm@digicool.com From jim@digicool.com Thu Nov 9 19:05:00 2000 From: jim@digicool.com (Jim Fulton) Date: Thu, 09 Nov 2000 14:05:00 -0500 Subject: [Doc-SIG] PEP-0216 References: Message-ID: <3A0AF55C.5D5BB090@digicool.com> Ken Manheimer wrote: > > Well, it's remarkable to see StructuredText really considered after all > this time. I'm sorry that i probably won't have time to do more than > kibitz. > > Moshe wrote: > > > At the moment, there are problems with ST-NG, which I've already > > written about in a previous e-mail. > > Which i understand to be: > > | Current questions about ST-NG: > | - Does it support proper escaping of meta-characters? If I want to > | type *, or ', can I? Why do you need this? Coud you give some examples? Please don't respond here though. See below. (snip solution to problem I don't quite understand. ;) > (Note - i *believe* that *example* text - blocks introduced by a paragraph > ending in a '::' - should have all ST interpretation disabled, so the > above examples would show the double quotes as well as the things they are > quoting - if the double quote convention existed.) Yes, no ST processing is done in examples. > | - Does my proposal about Python symbols identified as [:] > | make sense? How hard would it be to extend the current code to > | do just that? > > I don't understand the question, and don't have the time to follow > back a second level of indirection to track down your proposal. (And > i don't think i'll have time to do the legwork to investigate, if i > *did* understand the question:-( ) I know how you feel. It would help us (Ken and I), and I suspect others, alot if the Moshe's problem's with structured text could be documented in a more durable form than email messages. I took the liberty of creating a page in the StructuredText Wiki: http://dev.zope.org/Members/jim/StructuredTextWiki/DocumentationStrings to collect ideas for and comments on using StructuredText for doc strings. I'd appreciate it if people could add proposals and comments there. It's a Wiki page, meaning it can be edited through the web. To make changes, you need to be logged in to zope.org. Membership on zope.org is free. The editing format is, of course, StructuredText. :) Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From Moshe Zadka Fri Nov 10 00:08:32 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 02:08:32 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <3A0AF55C.5D5BB090@digicool.com> Message-ID: On Thu, 9 Nov 2000, Jim Fulton wrote: > Why do you need this? Coud you give some examples? See last version of PEP (already in CVS, soon to be on the SF pages) > > I don't understand the question, and don't have the time to follow > > back a second level of indirection to track down your proposal. (And > > i don't think i'll have time to do the legwork to investigate, if i > > *did* understand the question:-( ) > > I know how you feel. Everything is in the PEP now. My bad for not putting it there earlier. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tony@lsl.co.uk Fri Nov 10 10:44:13 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Fri, 10 Nov 2000 10:44:13 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: <3A0AF55C.5D5BB090@digicool.com> Message-ID: <002d01c04b03$2adee650$f05aa8c0@lslp7o.int.lsl.co.uk> (Even if Jim and Ken can't contribute other than by listening, I personally still feel knowing they're there is a Good Thing.) Jim Fulton wrote: > I took the liberty of creating a page in the StructuredText Wiki: > http://dev.zope.org/Members/jim/StructuredTextWiki/DocumentationStrings > > to collect ideas for and comments on using StructuredText for > doc strings. > > I'd appreciate it if people could add proposals and comments > there. It's a Wiki page, meaning it can be edited through the > web. To make changes, you need to be logged in to zope.org. > Membership on zope.org is free. The editing format is, of > course, StructuredText. :) Well, I've just used my Zope membership (knew it would be useful one day) to flesh out a little the pages http://dev.zope.org/Members/jim/StructuredTextWiki/DocumentationStrings http://dev.zope.org/Members/jim/StructuredTextWiki/MosheZadkasProblemsWithSt ructuredText (those are *bound* to wrap). I think for sanity at *this* end we need to conduct our conversation in parallel - I'll make sure that significant material I produce gets placed within the Wiki as I can, or at least referenced from there (as a non-Wiki user, please realise that it takes *time* to get to grips with the paradigms, and that's not something I have much to spare - the nonlinearity of the medium can be a significant problem if one is in a hurry! and the urge to fill in needless links is one I ...just...can't...resist...). Personally, I would like to think we'll end up with: * StructuredTextNG, provided by Digicool/Zope * StructuredText??, a subclass of the above, containing extensions needed for Python Doc strings (from what I've seen of the *documentation* of StructuredTextNG, this is how one is *meant* to do things). * A single BDFL-approved tool for producing documentation from Python modules, using the Zope StructuredText.py module (subclassed as appropriate, of course), included in the Python distribution. * Possible feedback from the DOC sig into the development of StructuredTextNG, via the "normal" methods for such things (whatever they may be) - having the BDFL and his cohorts at Digicool can't but help with our learning how to do this sort of thing. Note that this clause means we don't have to get everything we want into StructuredText at the first go... * As many *other* tools as people want to provide, just as normal, which parse faster, or produce more output formats, or do other clever things - after all, there's more than one way to do things (umm, no, rewind that, something's wrong there...) The question is, can we have it by Python 2.1 (hmm, first beta February 2001, final release March 2001 - let's go for it!) Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ (hmm - Zope - better have a Zope compatible quote) ZODB - "Nyah! Nyah! *My* variables last longer than *your* variables." My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From MarkH@ActiveState.com Fri Nov 10 12:48:57 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 10 Nov 2000 23:48:57 +1100 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: > Everything is in the PEP now. My bad for not putting it there > earlier. OK - looking good. However, I don't really get your concerns. Can you expand? > a. A tag that means "this is a Python ``something'', guess what" > b. Tags that mean "this is a Python class/module/class var/instance var..." I'm not sure what you are getting at here. From the PEP, it seems you may be thinking about situations like: """ ... the POP3() class is passed as a parameter, and the result is an rfc822.Message() instance ...""" and """Params:: size -- size of result If ``size`` is negative, the result is ... """ Am I on the right track? Do you have better examples or what you are getting at? Thanks, Mark. From Moshe Zadka Fri Nov 10 13:02:02 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 15:02:02 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <002d01c04b03$2adee650$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: On Fri, 10 Nov 2000, Tony J Ibbs (Tibs) wrote: > The question is, can we have it by Python 2.1 (hmm, first beta February > 2001, final release March 2001 - let's go for it!) Let me state my own view for the Python time line, and how the doc-sig and PEP-0216 fits into it. The distutils are already finished, and the catalog-sig is starting to come up with a CPAN-like module repository. There are good chances both of these will exist in time for 2.1. None of this will really take off, if after people install a module by just wishing it so (which can be made pretty close to the truth) they cannot see how to use it. So, what has to be for 2.1 is a. A way to have all Python documentation inside the modules, so people will have good examples to follow. For that, we'll need b. A way to generate Python-LaTeX documentation from docstrings. With a little luck, we'll use (b) and something completely unlrelated c. An XML format for Python documentation, and a set of tools to convert it to other formats (HTML, manpages and LaTeX is a must) will replace the current hardly-manageable doc tool-chain To create d. A tool in standard Python 2.1 distribution, which displays a module's documentation, using whatever means available. PEP-0216 is responsible for a and b, and converting b into d when Fred finally finds the correct XML format. we-only-have-four-months-ly y'rs, Z. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Fri Nov 10 13:19:37 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 15:19:37 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: On Fri, 10 Nov 2000, Mark Hammond wrote: > OK - looking good. However, I don't really get your concerns. Can you > expand? Sure. I tried to give examples in my PEP which should be commited. As usual, the SF pages take a while to get updated (slow connection) > > a. A tag that means "this is a Python ``something'', guess what" > > b. Tags that mean "this is a Python class/module/class var/instance > var..." > > I'm not sure what you are getting at here. > > >From the PEP, it seems you may be thinking about situations like: > > """ ... the POP3() class is passed as a parameter, and the result > is an rfc822.Message() instance ...""" Yes, something like that. Please have another look at the PEP and see if I managed to make myself clearer this time. Thanks for the comments. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From MarkH@ActiveState.com Fri Nov 10 13:42:35 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Sat, 11 Nov 2000 00:42:35 +1100 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: > Sure. I tried to give examples in my PEP which should be commited. As > usual, the SF pages take a while to get updated (slow connection) Yep - my mistake was following that hyperlink rather than looking at my file system. The cost of convenience :-) > Yes, something like that. Please have another look at the PEP and see if > I managed to make myself clearer this time. For our other readers: --- a. A tag that means "this is a Python ``something'', guess what" Example: In the sentence "The POP3 class", we need to markup "POP3" so. The parser will be able to guess it is a class from the contents of the poplib module, but we need to make it guess. b. Tags that mean "this is a Python class/module/class var/instance var..." Example: In the sentence "This can be a regular file or a StringIO object" we need to markup "StringIO" as a Python class (the guesser might guess it to be a module by mistake). --- (a) I buy, but think it reasonable a convention could be to use parameters around callable objects, and fully qualified names. Thus "module.variable", "module.class", "local_class()", "local_function()" are all reasonable, and within the structured text spirit (IMO). It only leaves "local_variable" (or param, etc) requiring special treatment. (b) I don't really buy - it appears the exact same problem to me. The docstring should say "a regular file or a StringIO.StringIO() object". If the user took you literally, and tried to create a "StringIO" object, it would fail - presumably your module defines no such object. Further, I believe it would be a reasonable policy decision for the formatting tool to choose to *omit* both the module and the parens in the final document if a link can be made - with the link there can be no ambiguity. This gives you very pretty documentation, readable, clear plain-text, and a pattern that will be correct the vast majority of cases. Mark. From tony@lsl.co.uk Fri Nov 10 13:49:24 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Fri, 10 Nov 2000 13:49:24 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <003001c04b1d$08eb7f80$f05aa8c0@lslp7o.int.lsl.co.uk> Mark Hammond worries about Moshe's (a) and (b) points in the PEP... > > >From the PEP, it seems you may be thinking about situations like: > > > > """ ... the POP3() class is passed as a parameter, and the result > > is an rfc822.Message() instance ...""" It has been said for a long time that we need to be able to generate cross-references from the docstring to the actual code, which is what this is about. HOWEVER. It seems clear to me that we deal with the things we can do now (more or less!) first (i.e., points d and e). Point c is debatably covered (via the Example or :: conventions). We can live with it for the moment. Points a and b are desirable, but we can live without them in the first instance if it means we get DOCUMENTATION started (which is, after all, the *most* important thing). It looks like StructuredTextNG is meant to be designed so it can be "subclassed" for other purposes - at least that's a stated aim. So I would expect the Zopistas to be willing to help us get such a subclassing going, and that's clearly the way to handle the additional points. And if it's done via subclassing, we can clearly roll it in later on. BTW, I commend the following page to people: http://www.zope.org/Members/jim/StructuredTextWiki/StructuredTextZen I'm not entirely sure I agree, but then I wouldn't be... Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Fri Nov 10 13:50:17 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Fri, 10 Nov 2000 13:50:17 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <003101c04b1d$28dab310$f05aa8c0@lslp7o.int.lsl.co.uk> Moshe Zadka elucidate what he saw us needing to produce for 2.1: > a. A way to have all Python documentation inside the modules, so > people will have good examples to follow. > ...etc... My only worry is a little vagueness round the edges of the description whenever you refer to "a module's documentation". I remember Eddy saying, yonks back: > There are three kinds of `documentation' I want associated > with a tool: > > comments -- which I read when I'm modifying the tool > manuals -- which I read before trying to write code which uses > the tool > docstrings -- which I read when I find the tool in my hands Whilst one might argue with the *detail* of those descriptions, I believe that there is a very distinct diffference between manuals and docstrings (in much the same way as there is between books and memos).[1] I assume that when you say "documentation" you *don't*, in fact, mean "manuals", but I have learnt to be cautious, because I *have* seen people conflating the two in the past, and I want to make sure we know what we aren't doing... I think I'm particularly worried by the statement "A way to have all Python documentation inside the modules"... (of course, if you just mean that installing a new module (in the Catalog sense) just ensures that its documentation (of all forms) also gets installed, somewhere, then I am not worried - but that's a different issue) I think I'm also upset if you expect people writing manuals to have to use the same format designed for docstrings, since the AIMS are different - docstrings must be pleasant to read "as is", by people inside the code, or people using a class browser, but very few people (um, well) read HTML or XML or TeX raw, and most people don't even *write* them "raw". And in "real" documentation you often want to have closer control over what is going on than StructuredText will allow. [1] I cite the parlous state of much of the Microsoft technical documentation, which has clearly been cobbled together from all of the notes and memos and (indeed) documentation strings that could be found. This gives a large body of text, with no real organisation, occasional contradiction, and no *flow* - there is, in general, no way one can navigate through a particular problem space without use of the index. Tibs, feeling he should be worried *just in case* (but not, noway, worried enough to stop us getting something done, honest guv) -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ .. "equal" really means "in some sense the same, but maybe not .. the sense you were hoping for", or, more succinctly, "is .. confused with". (Gordon McMillan, Python list, Apr 1998) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From jhauser@ifm.uni-kiel.de Fri Nov 10 14:10:04 2000 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Fri, 10 Nov 2000 15:10:04 +0100 (CET) Subject: [Doc-SIG] PEP-0216 In-Reply-To: References: Message-ID: <20001110141004.665.qmail@lisboa.ifm.uni-kiel.de> Mark Hammond writes: > (a) I buy, but think it reasonable a convention could be to use parameters > around callable objects, and fully qualified names. Thus "module.variable", > "module.class", "local_class()", "local_function()" are all reasonable, and > within the structured text spirit (IMO). It only leaves "local_variable" > (or param, etc) requiring special treatment. > > (b) I don't really buy - it appears the exact same problem to me. The > docstring should say "a regular file or a StringIO.StringIO() object". If > the user took you literally, and tried to create a "StringIO" object, it > would fail - presumably your module defines no such object. Do you mean this should also be done for deeper package structures? These names can become quite long and are all over the place if one makes references to other documentstrings in a package itself. __Janko From Moshe Zadka Fri Nov 10 16:51:17 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 18:51:17 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: On Sat, 11 Nov 2000, Mark Hammond wrote: > Yep - my mistake was following that hyperlink rather than looking at my file > system. The cost of convenience :-) Sorry... > (b) I don't really buy - it appears the exact same problem to me. The > docstring should say "a regular file or a StringIO.StringIO() object". If > the user took you literally, and tried to create a "StringIO" object, it > would fail - presumably your module defines no such object. Perhaps it's a bad example, but I remember some good cases being made the other time around we discussed it: there *must* be a way of overriding the guesser. Your ideas about how to make the guesser better deserve a mention in the PEP, as well as some other ideas which were raised in the IPC8 meeting and later in the doc-sig > This gives you very pretty documentation, readable, clear > plain-text, and a pattern that will be correct the vast majority of cases. In case this wasn't clear, the guesser is supposed to be right 99% of the cases. For the 1% in which Python trickery so deep is employed, the author should override the guesser. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Fri Nov 10 16:59:16 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 18:59:16 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <003001c04b1d$08eb7f80$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: On Fri, 10 Nov 2000, Tony J Ibbs (Tibs) wrote: > HOWEVER. It seems clear to me that we deal with the things we can do now > (more or less!) first (i.e., points d and e). Point c is debatably covered > (via the Example or :: conventions). We can live with it for the moment. > Points a and b are desirable, but we can live without them in the first > instance if it means we get DOCUMENTATION started (which is, after all, the > *most* important thing). I disagree: until we settle points "a" and "b", it will be better to document the standard library (and other modules) in LaTeX, rather then ST-Py (tentative name). Until we redo the standard library, there is little hope that others will pick this up. > It looks like StructuredTextNG is meant to be designed so it can be > "subclassed" for other purposes - at least that's a stated aim Indeed, that's one of the things which pushed me over to ST. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Moshe Zadka Fri Nov 10 17:07:51 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 10 Nov 2000 19:07:51 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <003101c04b1d$28dab310$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: On Fri, 10 Nov 2000, Tony J Ibbs (Tibs) wrote: > > There are three kinds of `documentation' I want associated > > with a tool: > > > > comments -- which I read when I'm modifying the tool > > manuals -- which I read before trying to write code which uses > > the tool > > docstrings -- which I read when I find the tool in my hands > > Whilst one might argue with the *detail* of those descriptions, I believe > that there is a very distinct diffference between manuals and docstrings (in > much the same way as there is between books and memos).[1] > > I assume that when you say "documentation" you *don't*, in fact, mean > "manuals", but I have learnt to be cautious, because I *have* seen people > conflating the two in the past, and I want to make sure we know what we > aren't doing... If you read the PEP carefully, you'd notice that I'm thinking of docstrings in two related ways: parse time and run time. The parse time docstring is a superset of the runtime docstring. The runtime docstring is the memo, the parse time is the book. > I think I'm also upset if you expect people writing manuals to have to use > the same format designed for docstrings, since the AIMS are different - > docstrings must be pleasant to read "as is", by people inside the code, or > people using a class browser, but very few people (um, well) read HTML or > XML or TeX raw, and most people don't even *write* them "raw". And in "real" > documentation you often want to have closer control over what is going on > than StructuredText will allow. I don't believe that is true: as you say, nobody wants to write HTML or XML or TeX. > Tibs, feeling he should be worried *just in case* Not really: I'm not saying documentation for a module *must be* in the module, I'm saying the format must be flexible enough for it to be able to. What people will do is their own business. In my last proposal after IPC8, there was an elabortate template/docstring/xml trinity. At the time it was dismissed by Fred as irrelevant: it seems there are less then 5 people on the face of the planet interested in documentation outside the modules, so he can decide on the format himself. I told him I *am* interested, and if you are, you should tell him so too. In any case, the relationship between a manual outside and a docstring inside is yet to be defined, and not by PEP-0216. Let me just note that as a Perl hacker, the fact that the documentation (up to and including the tutorial and manual) being *inside* the code is very convinient. E.g., knowing that "perldoc HTML::Mason" just works, and gives me everything I need. Indeed POD is a large part of my inspiration. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From Juergen Hermann" Message-ID: On Fri, 10 Nov 2000 15:02:02 +0200 (IST), Moshe Zadka wrote: >c. An XML format for Python documentation, and a set of tools > to convert it to other formats (HTML, manpages and LaTeX is a must) > will replace the current hardly-manageable doc tool-chain Convert the StructuredText markup to DocBook, and then use 4XSLT to generate HTML (XSLT stylesheets for DocBook are there). Being able to generate HTML with a pure Python tool chain is a minimum requirement I think. Manpages and LaTeX might require a Jade etc. installation still, but it's certainly possible, and we're on Linux/FreeBSD documentation projects ground here, i.e. the tools DO exist. They just have to be installed in addition to Python. So, the major work would be to convert ST-NG to DocBook, and that is doable within the timeframe. Ciao, J=FCrgen -- J=FCrgen Hermann, Developer (jhe@webde-ag.de) WEB.DE AG, http://webde-ag.de/ From tony@lsl.co.uk Mon Nov 13 12:00:54 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 13 Nov 2000 12:00:54 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <000001c04d69$603b2e90$f05aa8c0@lslp7o.int.lsl.co.uk> First, status of my weekend work: I had less time than I hoped, due in part to our youngest not being very well, so all I've done at the moment is work out more-or-less what is going on in STNG as it stands. I've got a document that describes what I've learnt so far - although I don't guarantee much beyond its existence, it's at http://www.tibsnjoan.demon.co.uk/STNG-format.html in case anyone wants to criticise. Next thing to do is to actually produce something a little more formal, and easier to read! Second, worries about documentation as a whole... ((NOTE that this is what I would regard as pedantry - we are going to use STNG for docstrings, possibly subclassed, and everything else is frills. So continue at your peril...)) I was worried that Moshe wanted to do ALL the documentation using STNG (or whatever tool we decided on), but assumed that I was probably actually mistaken. Moshe Zadka wrote: > If you read the PEP carefully, you'd notice that I'm thinking of > docstrings in two related ways: parse time and run time. The parse time > docstring is a superset of the runtime docstring. The runtime docstring > is the memo, the parse time is the book. Oh dear, I was right. (I would point out I find the terms "parse time" and "run time" to be most misleading - there's documentation derived from the docstrings (be it at run time in an IDE, or on paper/the web statically), and there's documentatation written as such. OK, in the literate programming world there's an attempt to unify these, but we're not in that world. If Moshe is just assuming that *all* documentation for a module will be in docstrings, then I'm very confused.) I continued: > > I think I'm also upset if you expect people writing manuals to > > have to use the same format designed for docstrings, since the > > AIMS are different - docstrings must be pleasant to read "as is", > > by people inside the code, or people using a class browser, but > > very few people (um, well) read HTML or XML or TeX raw, and most > > people don't even *write* them "raw". > > And in "real" documentation you often want to have closer control > > over what is going on than StructuredText will allow. To which Moshe replied: > I don't believe that is true: as you say, nobody wants to write HTML or > XML or TeX. Firstly, some of us (ME!) do like writing using HTML, XML, TeX or whatever. What I *said* was that I had learnt that *most* people don't, and that if one wants documentation in docstrings, you cannot expect them to use "heavyweight" documentation methods, or you don't get the documentation. So STNG is perfect for docstrings. There is also an argument to say that, if one wants (external) documentation for a module, then a neutral format usable by everyone can also be useful - at the moment that tends to be either a README file in ASCII text, or an HTML page. For these, STNG can also be a win, as has (presumably) been proven by the Zope world. (But note that I do NOT regard these two as interchangable, nor do I *want* them to be.) Moshe wrote: > Not really: I'm not saying documentation for a module *must be* in the > module, I'm saying the format must be flexible enough for it to be > able to. What people will do is their own business. which is OK so far as it goes. He continued: > In my last proposal after IPC8, there was an elabortate > template/docstring/xml trinity. At the time it was dismissed by Fred > as irrelevant: it seems there are less then 5 people on the face of the > planet interested in documentation outside the modules, so he can > decide on the format himself. I told him I *am* interested, and if > you are, you should tell him so too. In any case, the relationship > between a manual outside and a docstring inside is yet to be defined, > and not by PEP-0216. Hmm. Fred is producing the Python documentation set, to a professional standard. He is using professional quality tools for this purpose. STNG is *not* adequate for this job. It cannot be so without getting *way* too complicated, because producing a BOOK (which is what the Python documentation set is composed of, even if the books are also viewable as web pages) is a different and more complicated thing. I believe he is right when he says very few people are interested in doing this, and I also trust him to get it right (he's done a wonderful job so far). This **is** a different issue than what we are discussing for STNG. It *should* be handled by a different mechanism. Full stop. Not a point to be argued about. Ask Fred. As to docstrings and per-module documentation, some of us (me, me!) will feel constrained by the simple formatting that STNG allows, but will put up with it in docstrings because it does a good job at what it does, and one has to have a standard. The question of documentation *outside* the module is still open, so far as I'm concerned, and STNG is just one tool that will get used for that (I'm not saying there *should* be more than one tool, I'm saying that in practice there *will* be more than one tool, although I expect that if STNG is easy enough to use it will be the most prevalent, which would be a Good Thing on the whole.). > Let me just note that as a Perl hacker, the fact that the documentation > (up to and including the tutorial and manual) being *inside* the code > is very convinient. E.g., knowing that "perldoc HTML::Mason" just > works, and gives me everything I need. Indeed POD is a large part > of my inspiration. What does "inside the code" mean in this context? If it means "inside my single .py file", then no, it's a Bad Thing, because it makes my .py file too long, and also makes it hard to distribute the documentation WITHOUT the code (or to amend one without changing the other!). If it means "inside the directory of my module" then it's not so bad, but on installation I damn well want the documentation putting in the Appropriate Place (so I can find it!) - on systems without such a standard place, then inside the module directory is still good, of course. And note that a decent piece of documentation is itself often file-structured (i.e., made up of more than one thing that one would normally keep in a separate file). I *think* we're probably still talking at cross purposes, and could resolve matters a bit better with some concrete examples - lets promise to do that at a later date. But luckily, regardless, the initial stage of getting STNG based python documentation in docstrings off the ground is still something we all want... Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Well we're safe now....thank God we're in a bowling alley. - Big Bob (J.T. Walsh) in "Pleasantville" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Mon Nov 13 12:05:52 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 13 Nov 2000 12:05:52 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <000101c04d6a$117e83f0$f05aa8c0@lslp7o.int.lsl.co.uk> Juergen Hermann [mailto:jh@web.de] wrote: > Convert the StructuredText markup to DocBook, and then use 4XSLT to > generate HTML (XSLT stylesheets for DocBook are there). Being able to > generate HTML with a pure Python tool chain is a minimum requirement I > think. > > Manpages and LaTeX might require a Jade etc. installation still, but > it's certainly possible, and we're on Linux/FreeBSD documentation > projects ground here, i.e. the tools DO exist. They just have to be > installed in addition to Python. First inspection indicates that generation of HTML and DocBook are already provided by STNG. Adding the backend to generate something else "similar" (e.g., some LaTeX variant) should be very easy. Indeed, the main way of accessing the structured text appears to be via DOM calls (which I believe is what you're really after?). Heh, those Digicool people might just know what they're doing... Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From Moshe Zadka Mon Nov 13 12:24:15 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 13 Nov 2000 14:24:15 +0200 (IST) Subject: [Doc-SIG] PEP-0216 In-Reply-To: <000001c04d69$603b2e90$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: I hope your youngest got better. On Mon, 13 Nov 2000, Tony J Ibbs (Tibs) wrote: > Second, worries about documentation as a whole... > > ((NOTE that this is what I would regard as pedantry - we are going to use > STNG for docstrings, possibly subclassed, and everything else is frills. So > continue at your peril...)) I've braved bigger Per[i]ls > I was worried that Moshe wanted to do ALL the documentation using STNG (or > whatever tool we decided on), but assumed that I was probably actually > mistaken. I depends on how you define "Moshe wants", probably. > Oh dear, I was right. (I would point out I find the terms "parse time" and > "run time" to be most misleading - there's documentation derived from the > docstrings (be it at run time in an IDE, or on paper/the web statically), > and there's documentatation written as such. OK, in the literate programming > world there's an attempt to unify these, but we're not in that world. If > Moshe is just assuming that *all* documentation for a module will be in > docstrings, then I'm very confused.) As I said, the issue of documentation outside docstrings, and it's relationship with the documentation in the docstrings are outside the scope of my PEP. Which is not to say they do not interest me, just that I don't want to deal with them right now. So let's put all issues of "inside" vs. "outside" to rest -- my PEP does not deal with external documentation, and so has no opinion what to write where. However, I am concerned if the format is too weak to document the whole module. If you think this concern is baseless, please let me knw which precise documentation requirement in the PEP causes STNG to be too heavy. I've cruelly snipped all further references to that issue from your e-mail. > ... > I believe he is right when > he says very few people are interested in doing this, and I also trust him > to get it right (he's done a wonderful job so far). But Fred can't do that job, and he's not. He's documenting the Python core moudles just fine, but what about the mass of third party modules? Someone has to document them, and it's up to the doc-sig to supply standard, easy, tools. All the documentation within the module in STNG is a great solution for those. Automatic tools which turn these into Python-LaTeX would make documenting Python modules much easier. That's what I want. > What does "inside the code" mean in [the Perl documentation] context? It means inside the code. *Inside*. In the same file. Good distributions (ActivePerl for Windows, or Debian) format the documentation to a suitable format (HTML and man pages, respectively) before installation. Since they usually byte-compile (which gets rid of all strings but the first -- please read my PEP carefully) this means there is no penalty at run time either. In any case, again, this is a non-issue, there might be a format for external docs (in fact, there already is) -- butthis is outside the scope of my PEP. So we may fight later, but currently there is nothing (yet) to fight about. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tony@lsl.co.uk Mon Nov 13 12:46:05 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 13 Nov 2000 12:46:05 -0000 Subject: [Doc-SIG] PEP-0216 In-Reply-To: Message-ID: <000201c04d6f$b03cd820$f05aa8c0@lslp7o.int.lsl.co.uk> Moshe Zadka was (very quickly after I sent my email) terribly reasonable and finished by saying: > In any case, again, this is a non-issue, there might be a format for > external docs (in fact, there already is) -- but this is outside > the scope of my PEP. > > So we may fight later, but currently there is nothing (yet) to fight > about. On that we agree - and I don't think it would be a *fight* later on, more a long discussion (doubtless with forays into me not understanding things) and an eventual agreement of the state of the world. I'd certainly learn things on the way as well. But for now, on with (paid) work, and in the gaps thinking about STNG. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "Bounce with the bunny. Strut with the duck. Spin with the chickens now - CLUCK CLUCK CLUCK!" BARNYARD DANCE! by Sandra Boynton My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Tue Nov 14 10:19:05 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Tue, 14 Nov 2000 10:19:05 -0000 Subject: [Doc-SIG] PEP-0216 (sort of) In-Reply-To: Message-ID: <001001c04e24$5160a010$f05aa8c0@lslp7o.int.lsl.co.uk> Oh dear, I really shouldn't rant at Moshe, particularly given (a) he's got a history of knowing what he's talking about, and (b) he has a key to the time machine... As I was so aggressively saying, Fred would certainly agree with me and not Moshe. Oh yes... http://dev.zope.org/Wikis/DevSite/Proposals/StructuredDocument Apart from that, it's also another useful pointer to where StructuredText may be going (there, that's a bit of face saving). Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) [I've read it twice. I've thought it over. I'm sending it anyway.] From tony@lsl.co.uk Tue Nov 14 13:47:00 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Tue, 14 Nov 2000 13:47:00 -0000 Subject: [Doc-SIG] New(er) version of STNG-format In-Reply-To: Message-ID: <001401c04e41$5cfd7890$f05aa8c0@lslp7o.int.lsl.co.uk> I've just put up a revised version of http://www.tibsnjoan.co.uk/STNG-format.html It's still not finished, but I've at least reached something closer to a useful state. It now documents what I presume to be meant by each of the "special terms" in STNG, and also the RE actually used (and a sort-of translation of the RE for those like me who can't read them without the accompanying manual - this may or may not be correct!). I'll carry on work as I can, but any comments are very welcome (although I shan't see them before tomorrow at the earliest). (at least, the above URL *should* work, but it's having problems locally. If you can't follow it to the page, try http://www.tibsnjoan.demon.co.uk/STNG-format.html which is where that should redirect to) Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Feet first on 3 wheels... My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Wed Nov 15 16:30:56 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Wed, 15 Nov 2000 16:30:56 -0000 Subject: [Doc-SIG] New(erer) version of STNG-format In-Reply-To: <001401c04e41$5cfd7890$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <000501c04f21$6e752560$f05aa8c0@lslp7o.int.lsl.co.uk> I've just put up a revised version of http://www.tibsnjoan.co.uk/STNG-format.html (or possibly http://www.tibsnjoan.demon.co.uk/STNG-format.html) It now has my "assumptions" about what each type of colorization ("markup") should mean, decorating the text from STNG's own description. I've also mapped out the rest of the document, so it might now begin to be of some use. I also rewrote my bit of http://dev.zope.org/Members/jim/StructuredTextWiki/DocumentationStrings to be a bit neater, and to refer to said page. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "Bounce with the bunny. Strut with the duck. Spin with the chickens now - CLUCK CLUCK CLUCK!" BARNYARD DANCE! by Sandra Boynton My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From robin@alldunn.com Wed Nov 15 16:40:42 2000 From: robin@alldunn.com (Robin Dunn) Date: Wed, 15 Nov 2000 08:40:42 -0800 Subject: [Doc-SIG] New(erer) version of STNG-format References: <000501c04f21$6e752560$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <008c01c04f22$cc038900$3225d2d1@ARES> > I've just put up a revised version of > > http://www.tibsnjoan.co.uk/STNG-format.html > > (or possibly http://www.tibsnjoan.demon.co.uk/STNG-format.html) > I have been unable to connect to these websites since Tibs first posted them. Has anybody else been able to get to it? Could the page be moved to Starship or someplace more accessible? -- Robin Dunn Software Craftsman robin@AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython! From tony@lsl.co.uk Wed Nov 15 16:58:47 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Wed, 15 Nov 2000 16:58:47 -0000 Subject: [Doc-SIG] New(erer) version of STNG-format In-Reply-To: <008c01c04f22$cc038900$3225d2d1@ARES> Message-ID: <000601c04f25$525d14b0$f05aa8c0@lslp7o.int.lsl.co.uk> Robin Dunn wrote: > I have been unable to connect to these websites since Tibs first posted > them. Has anybody else been able to get to it? Could the page > be moved to Starship or someplace more accessible? I've been able to connect with the second URL (the "demon" one) OK, although it looks as if some funny DNS stuff has been happening locally (but then our intranet is being improoooved as well, so that might be coincidence). Although I have an account on Starship, I haven't "logged on" for ages, and it would take me significant effort to copy it there, I think. As an alternative, I've copied it to: http://homepage.ntlworld.com/tibsnjoan/STNG-format.html which *might* be more visible (again, *I* can see it from here, but that obviously proves nothing). Alternatively, I'm quite happy for someone else to mirror it elsewhere. Must dash - children to collect... Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From robin@alldunn.com Wed Nov 15 17:04:05 2000 From: robin@alldunn.com (Robin Dunn) Date: Wed, 15 Nov 2000 09:04:05 -0800 Subject: [Doc-SIG] New(erer) version of STNG-format References: <000601c04f25$525d14b0$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <009c01c04f26$10a14590$3225d2d1@ARES> > Although I have an account on Starship, I haven't "logged on" for ages, and > it would take me significant effort to copy it there, I think. As an > alternative, I've copied it to: > > http://homepage.ntlworld.com/tibsnjoan/STNG-format.html > > which *might* be more visible (again, *I* can see it from here, but that > obviously proves nothing). > Thanks! I'm able to reach that site. -- Robin Dunn Software Craftsman robin@AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython! From Moshe Zadka Thu Nov 16 07:04:36 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 16 Nov 2000 09:04:36 +0200 (IST) Subject: [Doc-SIG] STNG-format Mirror Message-ID: If you can't access the sites Tibs mentioned, please try http://www.moshez.org/STNG-format.html. It has a bit of occasionaly downtime, but it raises the odds of something working. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tony@lsl.co.uk Thu Nov 16 12:08:35 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Thu, 16 Nov 2000 12:08:35 -0000 Subject: [Doc-SIG] Even new(erer) version of STNG-format In-Reply-To: <000601c04f25$525d14b0$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <000101c04fc5$f20ace80$f05aa8c0@lslp7o.int.lsl.co.uk> OK - today's version of the document is at http://www.tibsnjoan.co.uk/STNG-format.html http://www.tibsnjoan.demon.co.uk/STNG-format.html or http://homepage.ntlworld.com/tibsnjoan/STNG-format.html Moshe - if you're keeping the mirror up, could you update yours as well, please? I'm now at the stage where some questions about what STNG does or should do *at the moment* need answering, so I'll be back to reading code for a bit. Comments are thus welcome on the document as it stands, in all its mess (including if people think the way it is done is not directly useful, in which case comments on how it should be organised/directed would be welcome). Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "Bounce with the bunny. Strut with the duck. Spin with the chickens now - CLUCK CLUCK CLUCK!" BARNYARD DANCE! by Sandra Boynton My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From dgoodger@bigfoot.com Fri Nov 17 02:34:34 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Thu, 16 Nov 2000 21:34:34 -0500 Subject: [Doc-SIG] A favor... Message-ID: I'm very interested in the docstring/auto-documentation/StructuredText discussion that's going on (and has been going on, off and on, for some time...). I think StructuredText is definitely the way to go, but with some rethinking; I am reworking my notes (for a related project, stalled) and will post them here shortly. I have a favor to ask of some kind soul out there in docsigland. With MacOS at home and being behind a very restrictive firewall at work, I have been unable to access Digital Creation's CVS repository. I would very much like to examine the StructuredTextNG code. Could someone please make a .tgz or .zip of the package? Put it up on the web, or mail it to me and I'll make it available to all. Thank you! (10 points to anyone who can discover the name of the stalled project.) -- David Goodger dgoodger@bigfoot.com Open-source projects: - The Go Tools Project: http://gotools.sourceforge.net (more to come!) From dgoodger@bigfoot.com Tue Nov 21 03:22:00 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Mon, 20 Nov 2000 22:22:00 -0500 Subject: [Doc-SIG] Re: A favor... In-Reply-To: Message-ID: Never mind, I got it via my SourceForge shell account. Thanks SourceForge! If anybody would like a copy of the current StructuredTextNG, please let me know. From fdrake@users.sourceforge.net Wed Nov 22 12:51:00 2000 From: fdrake@users.sourceforge.net (Fred L. Drake) Date: Wed, 22 Nov 2000 04:51:00 -0800 Subject: [Doc-SIG] [development doc updates] Message-ID: <200011221251.EAA16696@orbital.p.sourceforge.net> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ From dgoodger@bigfoot.com Sat Nov 25 03:57:10 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Fri, 24 Nov 2000 22:57:10 -0500 Subject: [Doc-SIG] A Plan for Structured Text Message-ID: All the activity over PEP 216 and StructuredTextNG has spurred me to complete a project I left off many months ago: a plan, an analysis, and a revised specification for Structured Text. You may be saying, "Not again!", because this topic has come up many times in Doc-SIG and elsewhere. But I believe in the idea, and I believe that it is the best hope for Python to get its own superior equivalent to Perl's POD and Java's JavaDoc. I am posting these documents to the Doc-SIG to ask for everyone's input. I will soon put them (or revised versions after discussion) in a more permanent home at http://structuredtext.sourceforge.net ('restructuredtext' was too long for a SourceForge project name, unfortunately). Please rest assured that this is not another argument without code to back it up. I will commit to writing a new implementation, or assisting in modifying an existing implementation. This is a serious proposal. I want to see this get *done*! -- David Goodger dgoodger@bigfoot.com Open-source projects: - The Go Tools Project: http://gotools.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net ============================ A Plan for Structured Text ============================ David Goodger (mailto:dgoodger@bigfoot.com) 2000-11-24 Structured Text is useful for quickly creating simple web pages and for in-line program documentation (in comments any programming language, or in documentation strings in languages that support them, such as Python and Emacs Lisp). Structured Text can be used in systems which extract in-line comments and docstrings to create API documentation. A Structured Text implementation should be designed with the goal of enabling further extension and modification for specific application domains, and be coded to be readable and understandable. The following block diagram sketches out a hypothetical 'DocXtor' system, showing how Structured Text fits in:: +-----------------------------------------------------------------+ | DocXtor: Python Documentation Extractor | | | |--------------------------------+ | | Python-specific | | | Structured Text extensions | +---------------------| |--------------------------------+-------+ | Python language | | reStructuredText: | | services | | Revised Structured Text processor | | (parser.py, etc.) | +----------------------------------------+--+---------------------+ History ======= StructuredText.py was developed by Digital Creations (http://www.digicool.com) and first released in 1996. It is now released as a part of the open-source 'Z Object Publishing Environment' (ZOPE, http://www.zope.org). Structured Text itself is based on the earlier Setext specification (http://www.bsdi.com/setext). I discovered Structured Text while searching for a solution to my need to document the Python modules in my SGF parser & summarizer project (see http://gotools.sourceforge.net). Version 1.1 of StructuredText was included in the 'pythondoc' project (http://starship.python.net/crew/danilo/pythondoc/). Although I have yet to get pythondoc to work for me, I found Structured Text to be almost ideal for my needs. I joined the Python Doc-SIG (Documentation Special Interest Group, http://www.python.org/sigs/doc-sig/) mailing list and found an ongoing discussion of the shortcomings of the Structured Text 'standard'. I decided to modify the original module with my own extensions and some suggested by the Doc-SIG members. I soon realized that the module was not written with extension in mind, so I embarked upon a general reworking, including adapting it to the 're' regular expression module (I was more used to 're', it was more powerful, and the regular expressions in 'regex' and 'regsub' were nearly unintelligible with their excess of backslashes). Soon after I completed the modifications, I discovered that StructuredText.py was up to version 1.23 in the ZOPE distribution. Implementing the new syntax extensions from version 1.23 proved to be an exercise in frustration, as the complexity of the module had become overwhelming. I decided that a complete rewrite was in order, and even started a SourceForge project, reStructuredText_. Unfortunately I was sidetracked (or, if you ask my wife: fortunately I became employed) and stopped working on this project. Recently, development on StructuredTextNG (Next Generation) has begun at Digital Creations. It seems to have many improvements, but still suffers from many of the problems of classic StructuredText (ST-TOS, StructuredText: The Original Spec?). .. _reStructuredText: http://structuredtext.sourceforge.net Thus I recently made the time to enumerate the problems and possible solutions, and complete the first draft of a revised Structured Text specification. My motivations are as follows: - I need a standard format for inline documentation of the programs I write. I believe many others have the same need. Structured Text could be that format. - I believe in the Structured Text idea and want to help formalize the standard. However, I feel it has flaws that desperately need fixing. - Perl has POD, Java has JavaDoc. Neither of these mesh with the Pythonic worldview. Structured Text could form the foundation for a documentation extraction system (cool name: DocXtor, the Python Documentation Extractor) that Python needs and could greatly benefit from. There have been many attempts to write such a system, with varying success. A 'best of breed' system should be chosen and/or developed and included in Python's standard library. - Structured Text is only a foundation. It should not aspire to be the entire system. In fact, for Python docstring extraction, the Structured Text syntax would probably have to be extended to allow for higher-level semantic constructs (keyword-tagged values and the like). I don't want Structured Text or the hypothetical Python DocXtor to die because of overcomplication. - Most of all, I want to help ease the documentation chore, the bane of many a programmer. Structured Text Goals ===================== 1. To allow people to create richly structured documents using an ordinary text editor, without having to think about the markup. 2. The markup shall be intuitive, minimal, and unobtrusive. A. Intuitive -- Almost any plaintext document should be valid structured text. Structured text takes its cues from the kind of ad-hoc markup used in plaintext email messages and newsgroup postings. Wherever possible, structure should be inferred from such naturally occurring markup. B. Minimal -- The markup that is used must be as sparse as possible. HTML/XML-type tags are too cumbersome for manually edited, often embedded, multipurpose texts. C. Unobtrusive -- A document in structured text format shall read equally well in raw form as in processed form. Please keep in mind a basic goal of structured text, which is to keep the raw text as readable as possible. **If you don't buy into this idea, you're probably wasting your time.** (http://dev.zope.org/Members/jim/StructuredTextWiki/DocumentationStrings) 3. Multiple output formats shall be possible. - internal DOM-based data structures - XML/SGML: multiple DTDs - HTML - TeX/LaTeX - Structured Text - others (extensible) 4. The markup shall be modular and extensible for use in specific application domains. 5. The implementation shall be well documented, both internally (doc strings and comments) and externally (users manual, API guide, tutorial). StructuredTextZen ----------------- (from http://dev.zope.org/Members/jim/StructuredTextWiki/StructuredTextZen) What attracted me to StructuredText in the first place was the fact that it's the only structured document format that I could UseWithoutThinking. No matter how much of a DocBook wizard or HTML wizard you are, you still have to constantly think about elements, entities, and what not. By contrast, you can largely use StructuredText... and concentrate on your text rather than your formatting. Obviously this makes it very easy to use for experienced authors, but it also has tremendous benefits for folks who want to use it as part of a content delegation solution. StructuredText can be picked up in a very short time by non-techies and is probably the only solution available where people who don't necessarily think about document structure can author documents that are semantically parsable... From dgoodger@bigfoot.com Sat Nov 25 04:08:03 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Fri, 24 Nov 2000 23:08:03 -0500 Subject: [Doc-SIG] Problems With StructuredText Message-ID: ============================== Problems With StructuredText ============================== David Goodger (mailto:dgoodger@bigfoot.com) 2000-11-24 StructuredText_ is a great idea, however it does have flaws. Many of these flaws go back to the original Setext_ (structure enhanced text) specification (interesting reading!). .. _StructuredText: http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage .. _Setext: http://www.bsdi.com/setext There are several problems, unresolved issues, and areas of controversy within StructuredText. In order to resolve all these issues, I'd like to bring them all out into the open, enumerate all the alternatives, and propose solutions. Problems below are labelled C for Classic StructuredText, NG for Next Generation. 1. No formal specification_. The code *is* the standard. (C, NG) 2. Difficult to [understand and extend]_. (C, NG) 3. Block/section structure via indentation_. (C, NG) 4. No [escaping mechanism]_. (C, NG) 5. Awkward [bullet list markup]_: 'o'. (C, NG) 6. Problematic [enumerated list markup]_. (C) 7. Ambiguous markup for [code blocks]_. (C, NG) 8. Tables_. (C, NG) 9. Awkward [inline code]_ markup. (C, NG) 10. Awkward [hyperlink markup}_. (C, NG) 11. Markup must start with whitespace_. (C, NG) 1. Formal Specification ======================= .. _specification: The description in the original StructuredText.py has been criticized for being vague. "The code *is* the standard." Tony "Tibs" Ibbs has been working on deducing a detailed description from the documentation and code of StructuredTextNG_. His notes are available at: http://www.tibsnjoan.demon.co.uk/STNG-format.html .. _StructuredTextNG: http://dev.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG The specification should always preceed the code. Otherwise, StructuredText is a moving target which can never be adopted as a standard. 2. Understanding and Extending the Code ======================================= .. _understand and extend: The original StructuredText is a dense mass of sparsely commented code and inscrutable regular expressions. It was not designed to be extended and is very difficult to understand. StructuredTextNG has been designed to allow input (syntax) and output extensions, but its documentation (both internal [comments & docstrings], and external) is inadequate. I would like to see Structured Text become truly useful, perhaps even joining Python's standard library. Therefore it must have clear, understandable documentation and implementation code. 3. Structure via Indentation ============================ .. _indentation: Setext_ required that body text be indented by 2 spaces. The original StructuredText_ and StructuredTextNG_ require that section structure be indicated through indentation, as 'inspired by Python'. For certain structures (lists, code blocks, block quotes) indentation naturally indicates structure/hierarchy. For section structure, indentation is unnatural, wasteful of horizontal space, and awkward. Rather, the style of the section title usually indicates its structure. In the original StructuredText, sections consist of title paragraphs followed by indented paragraphs and other body elements. Using indentation is: - Unnatural -- Most published works use title style (type size, face, weight, and position) rather than indentation to indicate hierarchy. When indentation is used, it is usually the formatted end-result and is there for aesthetic rather than structural purposes. - Wasteful -- As the left indent is increased, the amound of horizontal space available for text decreases, unnecessarily extending the vertical length of a document. - Awkward -- One must think about the formatting as the text is keyed in. And when structural changes are made (it is very common during the composition of a document to rearrange sections and their hierarchy) we must use block-indent and -unindent functions. In order to input documents using indentation, relatively advanced text editors must be used. Python's significant whitespace is a wonderful innovation (even if it wasn't original to Python), however applying indentation to ordinary written text is overgeneralization. Instead, section structure through title style (as exemplified by this document) is far more natural. In fact, it is already in widespread use in plain text documents. 4. Escaping Mechanism ===================== .. _escaping mechanism: StructuredText needs a mechanism to treat markup-significant characters as the characters themselves. Currently there is no such mechanism (although ZWiki uses '!'). What are the candidates? 1. ! (http://dev.zope.org/Members/jim/StructuredTextWiki/NGEscaping) 2. \ 3. ~ 4. any others? I believe the best choice for this is the backslash (\). It's the single most popular escaping character in the world, therefore familiar. Since characters only need to be escaped under special circumstances, which are typically those explaining technical programming issues, the use of the backslash is natural and understandable. Python docstrings can be raw (prefixed with an 'r', as in 'r""'), which would obviate the need for gratuitous doubling-up of backslashes. The rule would be: A backslash followed by any character escapes the character. The escaped character represents the character itself, and is prevented from playing a role in any markup interpretation. The backslash is removed from the output. A literal backslash is represented by two backslashes in a row. XXX Allow backslashes preceeding non-markup characters to remain in the output? That might make describing regexes much easier. 5. Bullet List Markup ===================== .. _bullet list markup: StructuredText includes 'o' as a bullet character. This is dangerous and counter to the language-independent nature of the markup. There are many languages in which 'o' is a word. For example, in Spanish: Llamame a la casa o al trabajo. (Call me at home or at work.) And in Japanese (when romanized): Senshuu no doyoubi ni tegami o kakimashita. ([I] wrote a letter on Saturday last week.) If a paragraph containing an 'o' word wraps such that the 'o' is the first text on a line, it could be misinterpreted as a bullet list. I recommend omitting 'o' as a bullet character. '+' could be used instead. 6. Enumerated List Markup ========================= .. _enumerated list markup: StructuredText enumerated lists are allowed to begin with a number (sequence of digits) followed by whitespace. This could have consequences for line wrapping and writing styles:: "That bird wouldn't *voom* if you put 10000 volts through it!" 1 is all I need. I recommend requiring something after the number, a period ('.'), a colon (':'), a dash ('-'), a space and a dash (' -'), a right-parenthesis (')'), or surrounded with parentheses ('()'). Perhaps this list is excessive. But forgiving is better than restrictive. Should the digits/letters/numerals themselves be interpreted, allowing nested enumerated lists to be created without indentation? How about nested enumerated lists without indentation via compound enumerators? Simply count the 'length' (number of sub-enumerators) of the compound enumerator:: 1. one 1.a. two 1.a.I. three 2.a. two 2.b.I. three 7. Code Blocks ============== .. _code blocks: The StructuredText specification has example code blocks indicated by 'example', 'examples', or '::' ending the preceeding paragraph. STNG only recognizes '::'; 'example'/'examples' are not implemented. This is good; it fixes a language-dependent feature. The problem is what to do with the '::'. I propose that '::' at the end of a paragraph indicate that subsequent *indented* blocks are treated as example code. No further markup interpretation is done within code blocks (not even backslash-escapes). If the '::' is preceeded by whitespace, '::' is omitted from the output; if '::' was the sole content of a paragraph, the entire paragraph is removed (no 'empty' paragraph remains). If '::' is preceeded by a non-whitespace character, '::' is replaced by ':' (i.e., the extra colon is removed). Thus, a section could begin with a code block as follows:: Section Title ------------- :: print "this is example code" One possible variation is for meta-documentation (perhaps an extension?): use triple-colons (':::') to indicate 'take the following code block, mark it up as a code block, then copy it and mark it up as if it weren't a code block'. The implementation may insert text in-between, such as 'Marked up as:', or may alter the formatting (different font, set in a colored box, whatever). 8. Tables ========= .. _tables: The table markup scheme in classic StructuredText was horrible. Its omission from StructuredTextNG is welcome, and I will not dignify the markup by repeating it here. However, tables themselves are useful in documentation. Alternatives: 1. This format is the most natural and obvious. I came up with it (no great feat of creation!), and later discovered that it is the format supported by the [Emacs table mode]_:: +------------+------------+---------------------------+ | Column 1 | Column 2 | Column 3 & 4 span (Row 1) | +------------+------------+------------+--------------+ | Column 1 & 2 span | Column 3 | - Column 4 | +------------+------------+------------+ - Row 2 & 3 | | 1 | 2 | 3 | - span | +------------+------------+------------+--------------+ Tables are described with a visual outline made up of the characters '-', '|', and '+'. The hyphen ('-') is used for horizontal lines (row separators), the vertical bar ('|') for vertical lines (column separators), and the plus sign ('+') for intersections of horizontal and vertical lines. Row and column spans are possible simply by omitting the column or row separators, respectively. Each cell contains body elements, and may have multiple paragraphs, lists, etc. .. _Emacs table mode: ftp://archive.cis.ohio-state.edu/pub/emacs-lisp/archive/table.el 2. Below is a minimalist possibility. It may be better suited to manual input than alternative #1, but there is no Emacs editing mode available. One disadvantage is that it resembles section titles; a one-column table would look exactly like section titles. It could be a directive-driven (extra syntax) extension. :: Column 1 Column 2 Column 3 & 4 span (Row 1) ============ ============ =========================== Column 1 & 2 span Column 3 - Column 4 ------------------------- ------------ - Row 2 & 3 1 2 3 - span ============ ============ ============ ============== Each row is underlined. The head row is underlined with '=', with spaces at column boundaries. If there is no head row, the table begins with a top border of equals signs with spaces at column boundaries. Internal row separators are underlines of '-', with spaces at column boundaries. Column spans have no spaces. Row spans simply lack an underline at the row boundary. The bottom boundary of the table consists of '=' underlines. A blank line is required following a table. 9. Inline Code ============== .. _inline code: The current markup for inline code (text left as-is, verbatim, usually in a monospaced font; HTML ) is single quotes ('code'). The problem with single quotes is that they are too often used for other purposes, like apostrophes, quoting text, and string literals. Alternatives:: 'code' \'code\' ''code'' "code" \"code\" ""code"" #code# @code@ `code` ^code^ ``code'' The examples below contain inline code, quoted text, and apostrophes. Each example should evaluate to the following HTML::

Some code, with a 'quote', "double", ain't it grand?

0. Some code, with a quote, double, ain't it grand? 1. Some \'code\', with a 'quote', "double", ain't it grand? 2. Some 'code', with a \'quote\', "double", ain\'t it grand? 3. Some ''code'', with a 'quote', "double", ain't it grand? 4. Some "code", with a 'quote', \"double\", ain't it grand? 5. Some \"code\", with a 'quote', "double", ain't it grand? 6. Some ""code"", with a 'quote', "double", ain't it grand? 7. Some #code#, with a 'quote', "double", ain't it grand? 8. Some @code@, with a 'quote', "double", ain't it grand? 9. Some `code`, with a 'quote', "double", ain't it grand? 10. Some ^code^, with a 'quote', "double", ain't it grand? 11. Some ``code'', with a 'quote', "double", ain't it grand? A more complicated piece of inline code::

Does a[b] = 'c' + "d" + `2^3` work?

0. Does a[b] = 'c' + "d" + `2^3` work? 1. Does \'a[b] = 'c' + "d" + `2^3`\' work? 2. Does 'a[b] = \'c\' + "d" + `2^3`' work? 3. Does ''a[b] = 'c' + "d" + `2^3`'' work? 4. Does "a[b] = 'c' + "d" + `2^3`" work? 5. Does \"a[b] = 'c' + "d" + `2^3`\" work? 6. Does ""a[b] = 'c' + "d" + `2^3`"" work? 7. Does #a[b] = 'c' + "d" + `2^3`# work? 8. Does @a[b] = 'c' + "d" + `2^3`@ work? 9. Does `a[b] = 'c' + "d" + \`2^3\`` work? 10. Does ^a[b] = 'c' + "d" + `2\^3`^ work? 11. Does ``a[b] = 'c' + "d" + `2^3`'' work? Backquotes (#9) seem to be the best choice. They are unobtrusive and relatviely rarely used (more rarely than ' or ", anyhow). Backquotes have the connotation of 'quotes', which other options (like carets, #10) don't. When used within code, they can be escaped (\`). Alternative choices are carets (#10) and TeX-style quotes (#11). For examples of TeX-style quoting, see: http://www.zope.org/Members/jim/StructuredTextWiki/CustomizingTheDocumentPro cessor The only uses of backquotes I know are: (A) As a synonym for repr() in Python. (B) For command-interpolation in shell scripts. (C) Used as open-quotes in TeX code (and carried over into plaintext by TeXies). The backslash-escape mechanism would allow A & B inside inline code. TeX quotes outside inline code (``like this'') could be a special case, interpreted and marked up as proper quotes. That leaves TeX quotes inside inline code, which (although ugly) could be handled by escaping with backslashes:: line `\`\`this''`! Let's face it, no mechanism for inline code is perfect, just as no escaping mechanism is perfect. No matter what we use, complicated expressions will end up looking ugly. We can only choose the least ugly option. 10. HyperLinks ============== .. _hyperlink markup: There are three forms of hyperlink currently in StructuredText_: 1. (Absolute & relative URLs.) Text enclosed by double quotes followed by a colon, a URL, and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:: "Python":http://www.python.org/ 2. (Absolute URLs only.) Text enclosed by double quotes followed by a comma, one or more spaces, an absolute URL and concluded by punctuation plus white space, or just white space, is treated as a hyperlink:: "mail me", mailto:me@mail.com 3. (Endnotes.) Text enclosed by brackets link to an endnote at the end of the document: at the beginning of the line, two dots, a space, and the same text in brackets, followed by the end note itself:: Please refer to the fine manual [GVR2000]. .. [GVR2000] Python Documentation, van Rossum, Drake, et al., http://www.python.org/doc/ The problem with forms 1 and 2 is that they are neither intuitive nor unobtrusive (they break Goal 2). The brackets in form 3 are too common in ordinary text (such as [nested] asides and Python lists like [12]). Alternatives: 0. Have no special markup for hyperlinks. A. Except for #1 below? 1. Interpret and mark up hyperlinks as any contiguous text containing '://' or ':...@' after an alphanumeric word (absolute URL; exact specification to be looked up). To de-emphasize the URL, simply enclose it in parentheses: Python (http://www.python.org/) A. Leave special hyperlink markup as a domain-specific extension. Ordinary Structured Text documents would be required to have inline hyperlinks. Processed hyperlinks (with the URL hidden) may be important for Zope and ZWiki pages, but are they important for general uses? I suspect yes. 2. The original Setext_ introduced a mechanism of indirect hyperlinks. A source link word ('hot word') in the text is given a trailing underscore:: Here is some text with a hyperlink_ built in. The hyperlink itself appears at the end of the document on a line by itself, beginning with two dots, a space, the link word with a leading underscore, whitespace, and the URL itself:: .. _hyperlink http://www.123.xyz This has the advantage of being readable and relatively unobtrusive. Since each source link must match up to a target, the odd variable ending in an underscore can be spared being marked up (no such target). The only disadvantage is that phrase-links aren't possible without some obtrusive syntax. Setext used 'underscores_instead_of_spaces_' for phrase links. We could achieve phrase-links if we enclose the link text in double quotes ('"like this"_') or in brackets ('[like this]_'). We get obtrusive markup, but that is unavoidable. I prefer the bracketed syntax as reminiscent of links on many web pages. The same markup can also be used for footnotes, removing the problem with ordinary bracketed text and Python lists:: Please refer to the fine manual [GVR2000]_. .. _[GVR2000] Python Documentation, van Rossum, Drake, et al., http://www.python.org/doc/ The two-dots-and-a-space syntax was generalized by Setext for comments, which are removed from the processed text. In order to eliminate ambiguity with comments and footnotes, I propose that a colon always follow the target link word/phrase in indirect hyperlinks (denoting 'maps to'). There is no reason to restrict target links to the end of the document; they could just as easily be interspersed. Internal hyperlinks (hyperlinks from one point to another within a single document) can be expressed by a source link as before, and a target link with a colon but no URL. As an added bonus, we now have a perfect candidate for Structured Text directives, a simple extension mechanism: a comment containing a single word followed by two colons and whitespace. The interpretation of subsequent data on the directive line or following is directive- and/or implementation-dependent. To summarize:: .. This is a comment. .. version:: 1 .. The line above is an example of a directive. This internal hyperlink will take us to the footnotes_. Here is a one-word_ indirect hyperlink. Here is [an indirect hyperlink phrase]_. This is a footnote [1]_. .. _footnotes: .. _one-word: http://www.123.xyz .. _an indirect hyperlink phrase: http://www.123.xyz .. _[1] Footnote text goes here. The presence or absence of a colon after the target link differentiates an indirect hyperlink from a footnote, respectively. Brackets around a target link word or phrase are optional as long as the phrase does not contain a colon. The examples below contain links (URLs & references), and bracketed text. In HTML, each example should evaluate to::

A URL, see [eggs2000] (in Bacon [Publisher]). Also see http://eggs.org.

0. A URL http://spam.org, see eggs2000 (in Bacon [Publisher]). Also see http://eggs.org. 1. A "URL":http://spam.org, see [eggs2000] (in Bacon [Publisher]). Also see "http://eggs.org":http://eggs.org. 2. A "URL", http://spam.org, see [eggs2000] (in Bacon [Publisher]). Also see "http://eggs.org", http://eggs.org. 3. A URL_, see [eggs2000]_ (in Bacon [Publisher]). Also see http://eggs.org. The bracketed text '[Publisher]' may be problematic with syntax 1 & 2. Syntax 3 is definitely the most readable. Here is the endnote/footnote itself. In HTML, each example should evaluate to::

[eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam"

0. eggs2000 "Spam, Spam, Spam, Eggs, Bacon, and Spam" 1. .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" 2. .. [eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" 3. .. _[eggs2000] "Spam, Spam, Spam, Eggs, Bacon, and Spam" For style #3, the indirect hyperlink would be entered as follows:: .. _URL: http:/spam.org 11. Whitespace Delimitation of Markup ===================================== .. _whitespace: StructuredText specifies that inline markup begin with whitespace, precluding such constructs as parenthesized or quoted emphatic text:: "**What?**" she cried. (*exit stage left*) The specification for how markup is detected should be refined to allow for such constructs. From dgoodger@bigfoot.com Sat Nov 25 04:15:02 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Fri, 24 Nov 2000 23:15:02 -0500 Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification Message-ID: ========================================================= reStructuredText: Revised Structured Text Specification ========================================================= David Goodger (mailto:dgoodger@bigfoot.com) 2000-11-24 This revised specification is an attempt to refine, standardize, and extend the original Structured Text from Digital Creations' Zope (http://www.zope.org). Structured Text is plain text (i.e., text without tags, control characters, or other embedded formatting information) that uses simple, intuitive, and language-independent constructs to indicate the structure of a document. These constructs are equally easy to read in raw and processed forms. This document is itself an example of Structured Text (raw, if you are reading the text file, or processed, if you are reading an HTML page, for example). Simple symbology is used to indicate special constructs, such as headings, bullet lists, and emphasis. The symbology used is as minimal and unobtrusive as possible. Less often-used constructs and extensions to the basic structured text format may have more elaborate markup. A Structured Text document is made up of body elements, and optionally structured into sections. Sections contain body elements and/or subsections. Body elements consist of: - paragraphs, which contain text and optional inline markup; - lists (enumerated, bullet, descriptive, and option), which contain list items, whose items in turn contain body elements; - code blocks, which contain preformatted text only (spaces and linebreaks are preserved); - block quotes, which contain body elements; and - tables, whose cells contain body elements. Blank lines are used to separate paragraphs and other elements. Blank lines may be omitted when the markup makes element separation unambiguous. Tabs will be replaced by spaces; tab stops are at every 8th column. Indentation is used to indicate, and is only significant in indicating: - nesting within list items, such as nested lists, or multiple paragraphs within a list item, - block quotes, and - the extent of code blocks. Paragraphs may contain inline markup. Inline markup may not be nested. - inline code - strong - emphasis - hyperlinks: - standalone (absolute URLs) - indirect (absolute and relative URLs) - internal (cross-links within a document) - footnotes Below is a block diagram overview of the hierarchy of element types in Structured Text. Elements 'may contain' other elements below them. Element types in parentheses indicate recursive relationships: sections may contain (sub)sections, tables contain further body elements, etc. Footnotes, comments, directives, and hyperlink targets (all starting with '.. ' in column 1) are independent of the hierarchy and may appear at any point. :: +-------------+ +----------------------------------------+ | comments, | | +-------+ | | directives, | | sections (begins with one | title |) | +-----------| hyperlink | | +-------+ | | footnotes | targets | |------------+---------------------------+---+-----------+-------------| | (sections) | body elements: | text | +------------| code | | block | | para- | block | | blocks | lists | quotes | tables | graphs |-------------+ +--------+-------+--------+--------+--------| | (body elements) | inline | +-------------------------| markup | +--------+ Syntax Details ============== Escaping Mechanism ------------------ The character set available in plain text documents is limited. Every non-alphanumeric character has been overloaded with functionality: ordinary written text, mathematics, computer programming, regular expressions, Internet conventions. No matter what characters are used for markup, they will already have multiple meanings in written text. Therefore they *will* appear in text **without being intended as markup**. A serious markup system requires an escaping mechanism to override the default meaning of the characters used for the markup. In Structured Text, we will use the (almost) universal escaping character, the backslash. A backslash followed by any character escapes the character. The escaped character represents the character itself, and is prevented from playing a role in any markup interpretation. The backslash is removed from the output. A literal backslash is represented by two backslashes in a row. Comments and Directives ----------------------- A comment/directive block is a text block: - whose first line begins with '.. ' in column 1, - whose second and subsequent lines are indented relative to the first, and - which ends with a blank or unindented line. This syntax is used for comments, footnotes, indirect hyperlinks, internal hyperlinks, directives, and as an extension mechanism. Footnotes and hyperlinks are described in the section 'Hyperlinks' below. Comments :::::::: Arbitrary text may follow the comment start and will be removed from the processed output. The only restriction on comments is that they not use the same syntax as directives or hyperlinks. It is recommended to put a blank line after a comment, to ensure that subsequent indented text blocks are not accidentally commented out. Directives :::::::::: Directives are indicated by a text block beginning with '.. ', followed by a single word (the directive name, [a-zA-Z][a-zA-Z0-9_-]*), two colons, and whitespace. (Two colons are used to avoid clashes with common comment text like '.. Warning: modify at your own risk!'.) Directive names are case-insensitive. Actions taken in response to directives and the interpretation of data in the directive block or subsequent text block(s) are directive- and implementation-dependent. No directives are defined by the core Structured Text specification. Directives can be used as an extension mechanism for Structured Text. For example, a proposal was made in the Python Doc-SIG for keyword-tagged values. This could be accomplished as follows:: .. keywords:: Author: Anne Elk (Miss) Revision: 1 If an implementation of Structured Text doesn't recognize a directive, the entire directive block will simply be treated as a comment. Any subsequent text blocks will be processed as usual. The implentation may also emit a warning. Section Structure ----------------- Sections are identified through their titles. Titles are marked up with 'underlines' below the title text (and, in some cases, 'overlines' above the title). An underline/overline is a line of non-alphanumeric characters that begins in column 1 and extends at least as far as the title text. In the case of both overlines and underlines, their lengths and characters must match. There may be any number of levels of section titles. Rather than imposing a fixed number and order of section title styles, the order enforced will be the order as encountered. The first style encountered will be an outermost title (like HTML H1), the second style will be a subtitle, the third will be a subsubtitle, and so on. Below are examples of section titles. The first five styles are recommended:: =============== Section Title =============== Section Title ============= Section Title ------------- Section Title ::::::::::::: Section Title ............. Section Title ************* Section Title +++++++++++++ Section Title ~~~~~~~~~~~~~ Section Title ^^^^^^^^^^^^^ Note that the first example title above (overline & underline of '=') is slightly inset, but it doesn't have to be; this is merely aesthetic and not significant. A blank line after a title is optional. All text blocks up to the next title are included in a section (or subsection, etc.). All section/title types need not be used, nor must any specific section/title type be used. However, a document must be consistent in its use of sections/titles: once established, section title types must be used in the outer-to-inner order. Body Elements ------------- Code Blocks ::::::::::: A paragraph which which ends with two colons ('::') signifies that all following **indented** text blocks are code blocks. No further markup processing is done within a code block. It is left as-is, and typically rendered in a monospaced font:: This is a typical paragraph. A code block follows:: for a in [5,4,3,2,1]: # this is some program code, formatted as-is print a print "it's..." # a code block continues until the indentation ends This text has returned to the indentation of the first paragraph, is outside of the code block, and therefore treated as an ordinary paragraph. When '::' is immediately preceeded by whitespace, both colons will be removed from the output. When text immediately preceeds the '::', *one* colon will be removed from the output, leaving only one (i.e., '::' will be replaced by ':'). When '::' is alone on a line, it will be completely removed from the output; no empty paragraph will remain. The minimum leading whitespace will be removed from the code block. In the example code block above, only the second line ('` print a`') will keep its leading whitespace. Block Quotes :::::::::::: A text block that is indented relative to the preceeding text, without markup indicating it to be a code block, is a block quote. All markup processing (for body elements and inline markup) continues within the block quote:: This is an ordinary paragraph, introducing a quote: "It is my business to know things. That is my trade." --Sir Arthur Conan Doyle Bullet Lists :::::::::::: A text block which begins with a '-', '*', or '+', followed by whitespace, is treated as a bullet list (unordered list) item. For example:: - This is the first bullet list item. - This is the first paragraph in the second item in the list. This is the second paragraph in the second item in the list. The blank line above this paragraph is required. - This is a sublist. A code block needs to be indented even more:: print "lemon curry?" - This is the third item of the main list. - This is the fourth item of the main list (no blank line above). The second line of this item is not indented relative to the bullet, which precludes it from having a second paragraph. - A fifth item, whose second line is indented only one space relative to the bullet. A second paragraph for the fifth item. This paragraph is not part of the list. Blank lines before bullet list items are optional; blank lines are only required to separate list items from other types of text blocks, as noted in the example. The indentation of bullet list items takes the bullet itself into account. In the second list item above: - The second paragraph is indented relative to the bullet. The second paragraph must line up with the left edge of the first. - The bullet of the sublist is indented relative to the bullet of the outer list's item. Enumerated Lists :::::::::::::::: A text block which begins with a sequence label is treated as an enumerated list (ordered list) element. Sequence labels can be:: 1. A sequence of digits followed by a period ('1.'), a colon ('1:'), a dash ('1-'), a space and a dash ('1 -'), a right-parenthesis ('1)'), or surrounded with parentheses ('(1)'). B. A single letter (uppercase or lowercase) followed by a period etc. III. A roman numeral (uppercase or lowercase) followed by a period etc. III.a. A sequence of enumerations, separated by periods and ending with a period etc. (III)(b) A sequence of enumerations, each enclosed in paretheses. III(c) A mixture of styles. Nested enumerated lists must be created with indentation (as in the example above). Enumerators are not interpreted. Descriptive Lists ::::::::::::::::: A text block with a first line that contains some text, followed by whitespace, '--', and some more whitespace, is treated as a descriptive list element. The '--' must be on the first line. The leading text is the term, and the text after the '--' is the description:: Type A -- The description may begin immediately after the '--', as long as the description is only one paragraph. Type B -- The description may begin immediately after the '--', and may contain multiple paragraphs if second and subsequent lines are indented relative to the left edge of the first line. Description paragraph 2, indented to the same level. Type C -- Type C is a variation of Type B. Description paragraph 2, indented to the same level. Type D -- The description may also begin below, indented. This is useful for multiple paragraphs, or arbitrary text blocks (lists, etc.). Description paragraph 2, indented to the same level. For type A descriptive list items, the second line of the description paragraph is checked for ' -- '. If present, it is assumed that it is the start of another list item. Example:: Item One -- Description. Item Two -- Description. Option Lists :::::::::::: .. XXX perhaps this should be left as an extension? Option lists are two-column lists of command-line options and descriptions. There are two types of options: short and long. Short options consist of one dash, an option letter, and an optional argument placeholder. Long options consist of two dashes, an option word, and possibly an argument placeholder. There must be at least two spaces between the option and the description. The option acts as a bullet, and description begins a new text block which may contain multiple paragraphs and body elements. For example:: -a Output all -b Output both (this description is quite long) -c arg Output just arg. --long Output all day long. Tables :::::: Tables are described with a visual outline made up of the characters '-', '|', and '+'. The hyphen ('-') is used for horizontal lines (row separators). The vertical bar ('|') is used for vertical lines (column separators). The plus sign ('+') is used for intersections of horizontal and vertical lines. Each cell contains body elements, and may have multiple paragraphs, lists, etc. Example: +------------+------------+---------------------------+ | Column 1 | Column 2 | Column 3 & 4 span (Row 1) | +------------+------------+------------+--------------+ | Column 1 & 2 span | Column 3 | - Column 4 | +------------+------------+------------+ - Row 2 & 3 | | 1 | 2 | 3 | - span | +------------+------------+------------+--------------+ Paragraphs :::::::::: Paragraphs are what's left when all other body element markup is exhausted. They consist of blocks of text with no external markup indicating any other body element. Blank lines separate paragraphs from each other and from other body elements. However, when unambiguous due to markup, blank lines may be omitted. An alternate style of indented-first-line paragraphs is as follows: This is a paragraph with an indented first line. Here is a second such paragraph. Inline Markup ------------- Inline markup is the markup of text within a text block. Inline markup cannot be nested. Inline Code ::::::::::: Text enclosed by backquotes (with whitespace or punctuation to the left of the first backquote and to the right of the second backquote) is treated as `example code`. Inline code is typically set in a monospaced typeface. Strong :::::: Text surrounded by '**' characters (with whitespace or puctuation to the left and to the right) is **emphasized strongly**, typically displayed as boldface. Emphasis :::::::: Text surrounded by '*' characters (with whitespace or puctuation to the left and to the right) is *emphasized*, typically displayed as italics. Hyperlinks :::::::::: Standalone Hyperlinks ..................... An absolute URL within a text block is treated as a general external hyperlink with the URL itself as the link's text. For example, :: See http://www.python.org for info. would be marked up in HTML as:: See http://www.python.org for info. Indirect Hyperlinks ................... Indirect hyperlinks consist of two parts. In the text body, there is a source link, a name with a trailing underscore:: See the Python_ home page for info. Somewhere else in the document is a target link: two dots, a space, an underscore, the same name used for the source link (no trailing underscore), a colon, whitespace, and a URL (relative or absolute):: .. _Python: http://www.python.org Combined, this is expressed in HTML as:: See the Python home page for info. Phrase-links (a hyperlink whose name is a phrase) can be expressed by enclosing the phrase in brackets and treating the bracketed text as a link name:: Want to learn about [my favorite programming language]_? .. _my favorite programming language: http://www.python.org If a phrase-link name contains any colons, they must be backslash-escaped in the link target. Internal Hyperlinks ................... Internal hyperlinks connect one point to another within a document. They are identical to indirect hyperlinks except that there is no URL in the target link. For example:: .. _target: This is the target point. Clicking on this internal hyperlink will take us back to the target_. Footnotes ......... Footnotes are like internal hyperlinks with text in the targets. Footnotes consist of two parts. In the text body there is a source link: a bracketed name (an alphanumeric string with no spaces), with a trailing underscore:: Please refer to the fine manual [GVR2000]_. Somewhere else in the document (not necessarily at the end) is a target link: two dots, a space, an underscore, the same bracketed name used for the source link (no trailing underscore or colon), whitespace, and the footnote text:: .. _[GVR2000] Python Documentation, van Rossum, Drake, et al., http://www.python.org/doc/ Syntax Diagrams =============== Paragraphs may be separated by a blank line:: +------------------------------+ | paragraph | | | +------------------------------+ +------------------------------+ | paragraph | | | +------------------------------+ First-line-indented paragraphs require no blank line to separate them:: +------------------------------+ | paragraph | | | +--+---------------------------+ | paragraph | +--+ | | | +------------------------------+ Code blocks indicated by '::' at the end of the preceeding paragraph:: +------------------------------+ | paragraph | | '::'$ | +--+---------------------------| | code block | +---------------------------+ List item blocks which are indented relative to the bullet or enumerator may contain multiple body elements (paragraphs, etc.):: +------+-----------------------+ | '- ' | list item | +------| | | (body elements)+ | +-----------------------+ +------+-----------------------+ | '- ' | list item | +--+---+ | | (body elements)+ | +---------------------------+ List item blocks which are not indented relative to the bullet or enumerator contain a single paragraph only:: +------+-----------------------+ | '- ' | paragraph | |------+ | | | +------------------------------+ Block quotes are indented relative to the preceeding text:: +------------------------------+ | current level of | | indentation | +--+---------------------------+ | block quote | | (body elements)+ | +---------------------------+ Comments begin in column 1 with two dots and a space:: +--------+---------------------+ | ^'.. ' | comment block | +--+-----+ | | | +---------------------------+ Directives are comments which begin with a directive name and two colons:: +------------------+-----------+ | ^'.. ' name '::' | directive | +--+---------------+ block | | | +---------------------------+ Footnotes use comment syntax with an underscore footnote name in brackets:: +-------------------+----------+ | ^'.. _[' name ']' | footnote | +--+----------------+ | | (body elements)+ | +---------------------------+ Hyperlink targets use comment syntax with an underscore, link name, and a colon:: +------------------+-----------+ | ^'.. _' name ':' | link | +--+---------------+ target | | | +---------------------------+ (Internal hyperlinks have empty link blocks. Indirect hyperlinks have an absolute or relative URL in their link blocks.) From dgoodger@bigfoot.com Sat Nov 25 04:34:48 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Fri, 24 Nov 2000 23:34:48 -0500 Subject: [Doc-SIG] A Plan for Structured Text In-Reply-To: <20001125040902.41DFA1CD2E@dinsdale.python.org> Message-ID: Mailman seems to have munged up my messages a bit. Wherever I had '..' at column 1, '...' was sent instead. Is this a bug or a feature? :> From dgoodger@bigfoot.com Sat Nov 25 14:22:05 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Sat, 25 Nov 2000 09:22:05 -0500 Subject: [Doc-SIG] Mailman In-Reply-To: <20001125141344.A17017@server.onjapan.net> Message-ID: on 2000-11-25 00:13, Jim Tittsler (jwt@OnJapan.net) wrote: > Your messages had '..' in them (not '...') on my machine. > > My mailman installation doesn't seem to treat that pattern > at the beginning of a line specially. > > Are you sure it isn't your mail client (or receiving MTA?)? It appears that you are correct. I've checked the Doc-SIG archives and the messages appear just fine there. Thanks for the feedback! David From dgoodger@bigfoot.com Sat Nov 25 19:16:51 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Sat, 25 Nov 2000 14:16:51 -0500 Subject: FW: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: <00112509193800.06574@argon> Message-ID: ---------- From: Bob Tolbert Reply-To: bob@tolbert.org Date: Sat, 25 Nov 2000 09:19:38 -0500 To: David Goodger Subject: Re: [Doc-SIG] reStructuredText: Revised Structured Text Specification On Friday 24 November 2000 23:15, you wrote: > ========================================================= > reStructuredText: Revised Structured Text Specification > ========================================================= > David Goodger (mailto:dgoodger@bigfoot.com) > 2000-11-24 David, Looks like you given this a bit of thought. I too happened upon the Doc-SIG 'war' while trying to find a decent method to document code. Specifically, I am working on documenting all of wxPython, a monumental task, and I don't want to do it twice. I have been impressed with the concept of ST, but unimpressed with the implementation. I am very happy to write LaTeX, so I find ST just too limited in it's current implementation and it is impossible to comprehend in order to modify. Just as an example of my problems, I find the current table layout mechanism interesting, but too limited. I had an idea that if I could add just a small wart to the layout, that I might be able to introduce the following: and then be able to get around the stylistic limitations by using CSS in the HTML output. A second problem I have is that simple itemized lists are set as complete paragraphs. In HTML this is a terrible waste of vertical space. For example: I might type HTML like:
  1. one
  2. two
  3. three
but in ST if I use the bullet layout, I would get:
  1. one

  2. two

  3. three

I clearly haven't thought about this as much as you, because I have a rather narrow goal for what I want to document. In any event, I need to get something done soon, so I am curious as too how much of this you have coded. Are you still trying to get the spec complete? Do you think it is worth doing even if the Doc-SIG doesn't jump on-board at first? (If you build it, will they come?) If this has a chance of becoming usable in a short while. (at least in a development version) then I would consider it a useful use of my time to help since it will get me to my more important goal sooner. Thanks for the thoughtful discussion, Bob Tolbert From dgoodger@bigfoot.com Sat Nov 25 19:17:50 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Sat, 25 Nov 2000 14:17:50 -0500 Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: <00112509193800.06574@argon> Message-ID: Hi Bob, I think a lot of people are in the same situation as we are, and have the same frustrations. > Just as an example of my problems, I find the current table layout mechanism > interesting, but too limited. I had an idea that if I could add just a small > wart to the layout, that I might be able to introduce the following: > >
This is exactly the kind of thing that extensions to the spec are for. Either subclassing the 'table' class for an implementation-specific quick fix, or implementing a general extension along with a directive, like:: .. table-class:: events or:: .. table:: class="events" The latter could be a useful general way of adding attributes to arbitrary structures. The point is that the core syntax should be kept as minimal as possible. Everyone will have specific extensions they need; an extension-enabling mechanism is the key to making everyone happy. Until now, the proposals have been of the 'all or nothing' variety. Person A will say, "let's add feature X to StructuredText". People who see this as cluttering up the syntax will never agree, Person A gets discouraged, and in the end nobody gets to use feature X. I see a future where there's the core Structured Text, plus a 'menu' of optional add-ons. So we order up a 'Structured Text with table extensions on the side'. (And hope the implementation doesn't ask, 'You want fries with that?' :-) > A second problem I have is that simple itemized lists are set as complete > paragraphs. In HTML this is a terrible waste of vertical space. A simple matter of tweaking the HTML output model for layout efficiency. From a document architecture point of view, having list items containing paragraphs (and other lists, code blocks, etc.) is a much cleaner model. But there's no reason that the HTML module couldn't be smart enough to make its output layout-friendly. > In any event, I need to get something done soon, so I am curious as too how > much of this you have coded. Are you still trying to get the spec complete? Yes, it's in the design/conceptual phase now, finalizing the spec, and thinking about the implementation. I made an aborted attempt to write something (many months ago) based on ST-TOS. At this point, I see two possibilities: 1. Jim Fulton at Digital Creations sees my proposal, thinks it's great, and decides to rewrite ST-NG based on it (or some subset). Of course, I would be happy to help out. 2. A new, from-scratch version is started my myself and anyone else who's interested. > Do you think it is worth doing even if the Doc-SIG doesn't jump on-board at > first? (If you build it, will they come?) Yes, we'll go the distance! People are always skeptical of proposals without code to back them up. I'm hoping to get feedback (of both the 'how about this?' and 'that sucks!' varieties) to firm up the spec so that an implementation will have a better chance of being accepted. > If this has a chance of becoming usable in a short while (at least in a > development version) then I would consider it a useful use of my time to help > since it will get me to my more important goal sooner. A chance? Sure. How much of a chance? You'll have to stay tuned (and continue to contribute, I hope!). Thanks for your input. BTW, would it be OK if I shared your original message and this reply with Doc-SIG? Just had an idea. One of the Python-specific extensions that's obvious is to mark up a block of text that begins with '>>> ' as a code block (and splice in Tim Peters' doctest too). Looking at this message, an email extension that marks up blocks of text where each line begins with '>'+ as block quotes would also be obvious. Oh, the possibilities! -- David Goodger dgoodger@bigfoot.com Open-source projects: - The Go Tools Project: http://gotools.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net (soon) From tony@lsl.co.uk Mon Nov 27 10:43:14 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 27 Nov 2000 10:43:14 -0000 Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: Message-ID: <007501c0585e$d88ec820$f05aa8c0@lslp7o.int.lsl.co.uk> I've still to read David's mammoth work (it's headed to the printer so I can look at it later). So a minor comment on something I *can* comment on quickly: Lists without blank lines between them:: a. A line b. Another line ii. Something else c. And more is something that the Zope STNG would like to address, and it is also something that is relatively easy to do, either in the STNG code that is being worked on, or in the "quick hack" that I've been working on, or (definitely!) in whatever David works on (one assumes). (one can either move some of the "recognition" code from the colouriser into the paragrapher, or else enable the colourised to split existing paragraphs - it's a matter of taste) Maybe that's not what Bob was talking about, since he mentions ST producing::
  1. one

  2. two

  3. three

which I'd regard as "broken" HTML anyway, so I doubt *anyone's* STNG would/should produce it... (and I suspect it's much more likely to start that with '
    ', or whatever the syntax is, anyway) As to coding - Zope people are working on it now. I've got a "test bed" using RE's that I'm working on in my glacial slow time - it's only partial, but it shows parsing with some more of what is wanted than STClassic provides. I'd prefer to use mxTextTools to REs, but REs are in Python now... I think that coding is not a large problem - we can steal stuff from various places, and the bulk of "given a docstring produce a structure you can use to emit interesting stuff" is quick to write. But I'd better stop now - I've got paid work to do, and then some reading... <> Both Zope/STNG and what I was hacking assume a code that one subclases from. Indeed, one can affect the parsing done even without a new subclass. Adding recognition of paragraphs that start with ">>>" (which is *all* one needs to do if paragraphs are delimited by blank lines, of course) is trivial and copes with Tim Peters' neat doc test tool, as said. I'd refrain from worrying about email messages yet! (but, as I believe I said before, the people at Zope aren't entirely silly, you know - I suspect they've seen the possibilities as well!) Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Mon Nov 27 12:50:10 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 27 Nov 2000 12:50:10 -0000 Subject: [Doc-SIG] A promise In-Reply-To: Message-ID: <007901c05870$93a2df00$f05aa8c0@lslp7o.int.lsl.co.uk> We have had several rounds of the Doc-SIG, each of which starts with a proposal, sounds like it might be getting somewhere, and then bottoms out when pedantry about *exactly* what we want sets in. I'm one of the pedants... This time round, we have agreed that something *very like* STNG is exactly what we want, and have now started to worry about the details. (NB: I *still* haven't read any of David Goodger's stuff, even though a quick flick through makes it look as if it has lots of good points, at least two of which I would want to take on board.[1]) You should already be able to see what I'm worried about... OK. Here is my proposal and promise. 1. We are going with something "very like" STNG(the specification). We said that already. Specifically, since STNG(the code) *will* be extendable by subclassing and customisation (heck, *is*), I believe that whatever we choose to do *will* be doable with STNG(the application), when it is finished (and that is a Good Thing). 2. We want to get the solution into Python 2.1, which means we need to get *something* going soon - not least so people can try it and get any bugs out. 3. The something we get going need not be the correct, perfect final solution - it need only cope with *enough* of the problems to get doc strings going (and issues of larger documents and tidying up and even slight incompatibilities can be coped with later on). This is a Very Important Point. 4. We need a tool that runs with bog-standard Python (preferably 1.5.2, since that's "old technology", and also 'cos it's what I've got available at work and at home!) 5. It's trivial to cope with Python "doc test" paragraphs (really, really trivial), and it's not hard to cope with things like:: This is some text 1. This is a list line 2. This is another ii. So is this (i.e., making the list items separate paragraphs, even without blank lines) - and I think those two cover 80 or 90 percent of the "wish it were better" for Python users. (Actually, pedantically, I'm not entirely convinced one needs them to be separate paragraphs, so long as they get to be separate nodes in the tree, but paragraphs are traditional...) 6. The likely solutions ALL provide base classes which can be subclassed and customised to get the parsing and output solutions that are wanted, so we can later on fritter away time on extended versions of the process with complete ease. I keep grumbling about not having any time, but here's a promise: I will have something simple available by Christmas that will: a. find docstrings in a module b. parse them c. emit something useful (which will be HTML initially 'cos it's dead easy) d. be easy to extend and customise 'b.' is partly there already, and I shall aim to get a tidied up version of it on the web by the end of the week (I *do* want the docstrings to tell the truth!). 'a.' will get nicked from somewhere else, or someone else can give it me! 'c.' might well be nicked as well, but is easy enough to do. The solution won't necessarily be elegant, most definitely won't be fast (except by accident), and it will have some severe restrictions (you probably won't be able to do '*This is emphasised and **this is nested strong***', which is what *I* call a severe restriction!), but it will be there, and it *will* explain how to extend itself. Is this a Good Thing To Do? .. [1] David says "the specification must preceed the code". I agree, but in practice specifications are normally vague (and if they're not, they normally get changed as time proceeds). I think it is valuable to get a tool "fleshed out", since the bulk of the tool will not change even if major details of the spec do - once we've agreed to be based around ST[NG] we're already most of the way there. So refining of the spec can proceed in parallel. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) [I've read it twice. I've thought it over. I'm sending it anyway.] From akuchlin@mems-exchange.org Mon Nov 27 15:38:48 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 27 Nov 2000 10:38:48 -0500 Subject: [Doc-SIG] Moving Python HOWTO site Message-ID: I'd like to move the HOWTO Web pages to the SourceForge site http://py-howto.sourceforge.net/, so that other people can help maintain the pages. Does anyone have suggestions about other material that should be on these pages? (Some draft pages are up at that URL currently, along with download links for everything; comments are welcomed. Don't tell me that the sidebar links are messed up; I'm aware of that...) --amk From fdrake@acm.org Mon Nov 27 15:39:54 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 10:39:54 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: <007901c05870$93a2df00$f05aa8c0@lslp7o.int.lsl.co.uk> References: <007901c05870$93a2df00$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <14882.32842.17550.886869@cj42289-a.reston1.va.home.com> Tony J Ibbs (Tibs) writes: > 'a.' will get nicked from somewhere else, or someone else can give it > me! I have some code that needs to be cleaned up, but shouldn't been too hard to use. > 'c.' might well be nicked as well, but is easy enough to do. I just want a data structure; once we have that, the output is not only trivial, but largely irrelevant. On the whole, this sounds like a very nice Christmas gift for everyone! (And if you don't celebrate, that shouldn't matter for this gift! ;) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From fdrake@acm.org Mon Nov 27 15:53:45 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 10:53:45 -0500 (EST) Subject: [Doc-SIG] Moving Python HOWTO site In-Reply-To: References: Message-ID: <14882.33673.990037.482069@cj42289-a.reston1.va.home.com> Andrew Kuchling writes: > I'd like to move the HOWTO Web pages to the SourceForge site > http://py-howto.sourceforge.net/, so that other people can help > maintain the pages. This sounds good to me! -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From tony@lsl.co.uk Mon Nov 27 15:57:57 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 27 Nov 2000 15:57:57 -0000 Subject: [Doc-SIG] A promise In-Reply-To: <14882.32842.17550.886869@cj42289-a.reston1.va.home.com> Message-ID: <008301c0588a$cf8e1420$f05aa8c0@lslp7o.int.lsl.co.uk> I wrote: > 'a.' will get nicked from somewhere else, or someone else > can give it me! and Fred L. Drake, Jr. wrote: > I have some code that needs to be cleaned up, but shouldn't be too > hard to use. to which I reply - Yes please, if you could email it to me (my aim is to have the docstring thingy do a Tim Peters' style doctest on itself, so the earlier I get such code the better) > On the whole, this sounds like a very nice Christmas gift for > everyone! (And if you don't celebrate, that shouldn't matter for this > gift! ;) Thanks. (As background, at the moment my work work is mainly tidying up/bug fixing/testing work. This is *very* valuable, and I do want to be doing it, but it isn't very *nice* or fun, and it means working in C or worse (the "worse" option is a sort of castrated Pascal). So in a sense the docstring stuff is a welcome relaxation/mind stretch. Well, in some sense.) Hmm. A serious point. What do we want to *call* this thing? pyDoc? Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ .. "equal" really means "in some sense the same, but maybe not .. the sense you were hoping for", or, more succinctly, "is .. confused with". (Gordon McMillan, Python list, Apr 1998) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From fdrake@acm.org Mon Nov 27 16:02:48 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 11:02:48 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: <008301c0588a$cf8e1420$f05aa8c0@lslp7o.int.lsl.co.uk> References: <14882.32842.17550.886869@cj42289-a.reston1.va.home.com> <008301c0588a$cf8e1420$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <14882.34216.951538.940039@cj42289-a.reston1.va.home.com> Tony J Ibbs (Tibs) writes: > Hmm. A serious point. What do we want to *call* this thing? That depends on what "this thing" is. ;) I imagine there are at least two components (long term): a tool that people run over their .py files, and a package in the standard library that allows access to useful chunks of documentation "online", with an interface from the interactive interpreter, IDLE, or other IDEs. I think the *first* thing to do is to create the tool; the library can be evolved & refactored from that codebase and additional requirements over time; I don't see this as terribly pressing. The package in the standard library should probably be called "docutils", after Greg Ward's "distutils", but I've not a clue as to what to call the command-line tool (well, not any good clues). -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From laurie@eh.org Mon Nov 27 16:20:15 2000 From: laurie@eh.org (Laurence Tratt) Date: Mon, 27 Nov 2000 16:20:15 +0000 Subject: [Doc-SIG] A promise References: <007901c05870$93a2df00$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <3A2289BF.A8C5DEB4@eh.org> "Tony J Ibbs (Tibs)" wrote: > OK. Here is my proposal and promise. [snip] > 2. We want to get the solution into Python 2.1, which means we need to > get *something* going soon - not least so people can try it and get any > bugs out. In my opinion, trying to rush this for the sake of speed alone is likely to create a bad tool which will alienate people to the point that noone will want to *ever* use the tool - a bad initial release can put people off even when the tool becomes useable. By all means develop *alongside* Python, but wanting to get something - no matter how good or bad - into the distribution isn't the way to get everyone to play along. It'll look unprofessional to include crappy software alongside a main Python release. You might be able to get something out by March (when I wrote Crystal, most of the work was done in 2 months and that includes things that wouldn't be necessary for a tool that is only intended to deal with Python) but there again you might not. When you're relying on peoples spare time - as this likely will - playing with release dates is like playing with fire. > 4. We need a tool that runs with bog-standard Python (preferably 1.5.2, > since that's "old technology", and also 'cos it's what I've got > available at work and at home!) Bad idea. IMHO, the only way to create a professional tool is to get it to use Pythons parse trees ("import"ing the modules is at best mickey mouse even if it solves one or two problems that static analysis can't hope to) and that really means that the tool has to track the latest release. As you seem to want to tie it to the main release anyway, this doesn't seem like a problem . > I keep grumbling about not having any time, but here's a promise: > > I will have something simple available by Christmas that will: > a. find docstrings in a module > b. parse them > c. emit something useful (which will be HTML initially > 'cos it's dead easy) > d. be easy to extend and customise > > 'b.' is partly there already, and I shall aim to get a tidied up version > of it on the web by the end of the week (I *do* want the docstrings to > tell the truth!). > > 'a.' will get nicked from somewhere else, or someone else can give it > me! There's a demo of how to munch the parse trees in the main distribution - it's what I started from. > 'c.' might well be nicked as well, but is easy enough to do. I doubt nicking this component is feasible. > The solution won't necessarily be elegant, most definitely won't be fast > (except by accident), and it will have some severe restrictions (you > probably won't be able to do '*This is emphasised and **this is nested > strong***', which is what *I* call a severe restriction!), but it will > be there, and it *will* explain how to extend itself. > > Is this a Good Thing To Do? Yes and no. If what you're producing will be extendable enough to become the perfect tool, yes. If it's not extendable enough and is going to be another brick on the way to creating the perfect tool, I wouldn't bother - current bricks include gendoc, pythondoc, Crystal, pydoc and happydoc. We have enough "nearly" examples to go on. > .. [1] David says "the specification must preceed the code". I agree, > but in practice specifications are normally vague (and if they're not, > they normally get changed as time proceeds). I think it is valuable to > get a tool "fleshed out", since the bulk of the tool will not change > even if major details of the spec do - once we've agreed to be based > around ST[NG] we're already most of the way there. So refining of the > spec can proceed in parallel. You might run into what I did - people actually seem surprisingly disinterested when a tool is actually produced. I know - I produced such a tool and it's probably extendable enough to cope with all this stuff . "Maybe this time"... but we've seen that several times before and people seem to get less interested as the discussions get increasingly anal and removed from getting anything done which is what's always happened before - a hint maybe for me to wrap this message up . Laurie From tony@lsl.co.uk Mon Nov 27 16:21:50 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 27 Nov 2000 16:21:50 -0000 Subject: [Doc-SIG] A promise In-Reply-To: <14882.34216.951538.940039@cj42289-a.reston1.va.home.com> Message-ID: <008401c0588e$25c917b0$f05aa8c0@lslp7o.int.lsl.co.uk> Fred L. Drake, Jr. wrote: > [...deletia...] I imagine there are at > least two components (long term): > a tool that people run over their .py files, That was the one I was thinking of, and will be the first "product". > and a package in the standard library that allows access to > useful chunks of documentation "online", with an interface > from the interactive interpreter, IDLE, or other IDEs. And that was the one I *wasn't* thinking of, but should have thought of. > I think the *first* thing to do is to create the tool; !! > the library can be evolved & refactored from that codebase and > additional requirements over time; I don't see this as terribly pressing. Agreed. If "refactoring" is what I've always taken it to be, any piece of Python code I work on gets refactored about once or twice a week in the early stages anyway... > The package in the standard library should probably be called "docutils", > after Greg Ward's "distutils", And that is useful input - I've *been* calling it STNG, which I already knew was a Bad Idea (since it's a term Zope should be allowed to keep for themselves). > but I've not a clue as to what to call > the command-line tool (well, not any good clues). OK, I'll call it something I can type and we'll rename it later. I'll avoid bad puns unless I can think of one. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Give a pedant an inch and they'll take 25.4mm (once they've established you're talking a post-1959 inch, of course) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tony@lsl.co.uk Mon Nov 27 16:59:56 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 27 Nov 2000 16:59:56 -0000 Subject: [Doc-SIG] A promise In-Reply-To: <3A2289BF.A8C5DEB4@eh.org> Message-ID: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> Laurence Tratt commented on what I wrote (and Outlook helped mangle the result): > > 2. We want to get the solution into Python 2.1, which means > > we need to get *something* going soon - not least so people > > can try it and get any bugs out. > > In my opinion, trying to rush this for the sake of speed > alone is likely to create a bad tool which will alienate > people to the point that noone will want to *ever* use > the tool - a bad initial release can put people > off even when the tool becomes useable. By all means develop > *alongside* Python, but wanting to get something - no matter > how good or bad - into the distribution isn't the way > to get everyone to play along. It'll look unprofessional to > include crappy software alongside a main Python release. Agreed in so far as it goes... (polite way of saying "aha - obviously I didn't explain myself well enough last time"). Given the history of getting nowhere with this, having a release date planned is a Good Thing. I think that the distutils work shows that. It concentrates the mind. I would *not* advocate accepting something crappy. If the consensus is that whatever I produced was crappy, I would not want it included (of course, I don't *believe* that will be the consensus, but if it were I would defend that stance). One of my earlier goals was to work out what STNG was trying to do. One of the simplest ways for me to do that was to try some things out (that's the way the insides of my head work). I now have the choice of writing more documents based on that, or producing some code and documenting it and using the code to turn said comments into the document I would have written anyway. Oh dear. But you see what I mean. If we don't have a solution for Python 2.1, I will be sad, but not devastated. On the other hand, if we don't have a solution for 2.2, I shall be angry. One other point, though, is that I am also predefending against arguments of the nature "but that is hard to do, so we'll ignore it" - it's easiest for me to do that with working code. I also hope some of it might be of use to the Zope people, who *have* put a lot of work into this. > You might be able to get something out by March (when I wrote > Crystal, most of the work was done in 2 months and that includes > things that wouldn't be necessary for a tool that is only intended > to deal with Python) but there again > you might not. Agreed. And I know how little spare time I have. (Truth to tell, I had considered just trying to modify Crystal, or HappyDoc, but they *are* both trying to be more sophisticated than we need in some ways, and less so in others). > > 4. We need a tool that runs with bog-standard Python > > (preferably 1.5.2, since that's "old technology", and > > also 'cos it's what I've got available at work and at home!) > > Bad idea. IMHO, the only way to create a professional tool is > to get it to use Pythons parse trees ("import"ing the modules > is at best mickey mouse even if it solves one or two problems > that static analysis can't hope to) and that really means that > the tool has to track the latest release. As you seem to want > to tie it to the main release anyway, this doesn't seem like a > problem . On this one you undoubtedly have more experience than me, but I would *like* to be able to use the same tool for 1.5.2 as for later versions of Python, even if in a less sophisticated manner - there are a significant number of firms (read one, the one I work for!) who have only just gone up to 1.5.2 (from 1.3 - can you say "hurray!" - is Eddy watching this?) and aren't likely to go up again soon. But again, I won't weep too much if that isn't feasible. > > Is this a Good Thing To Do? > > Yes and no. If what you're producing will be extendable > enough to become the perfect tool, yes. If it's not extendable > enough and is going to be another brick on the way to creating > the perfect tool, I wouldn't bother - current bricks include > gendoc, pythondoc, Crystal, pydoc and happydoc. We have enough > "nearly" examples to go on. I write simple code, 'cos I don't understand the complicated stuff. I've never been accused of undercommenting (hmm - maybe not always a recommendation). The thing will probably come as an "STNG-clone" supporting class with a pre-built example of extending it to be "our tool" ('cos that's a fairly easy way to do it and to make it possibly more useful for Zope-readers). It will use REs because that's "standard" and I don't want to fight the "getting a parser into Python" battle. I *don't* expect it to be "perfect", but I do expect it to be "useful". And I don't want to go another round of the Doc-SIG docstring tool "game", so I'm motivated to be awkward... > You might run into what I did - people actually seem > surprisingly disinterested when a tool is actually produced. > I know - I produced such a tool and it's probably extendable > enough to cope with all this stuff . "Maybe this > time"... but we've seen that several times before and people > seem to get less interested as the discussions get increasingly > anal and removed from getting anything done which is what's > always happened before - a hint maybe for me to wrap this > message up . Oh, the message is OK and it ends well! I don't expect *other* people to get "excited" about this (but heh, I've had three responses, which is about, ooh, two more than I expected!). Ultimately, the only response that *counts* will be Guido's (and I *suspect* that also means Frank's) when something gets finished and presented as "can we put it into a distribution, please?". Personally, I don't *care* if people *like* the tool - I think it needs to be invisible, just a component. There's enough egoboo in taking part (or being allowed to take part). Definitely more than enough wittering, I need to stop and go collect children Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Give a pedant an inch and they'll take 25.4mm (once they've established you're talking a post-1959 inch, of course) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From fdrake@acm.org Mon Nov 27 17:14:18 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 12:14:18 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> References: <3A2289BF.A8C5DEB4@eh.org> <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> Tony J Ibbs (Tibs) writes: > If we don't have a solution for Python 2.1, I will be sad, but not > devastated. On the other hand, if we don't have a solution for 2.2, I > shall be angry. At this point, I don't think worrying about Python releases is interesting. Until there's something to use, it doesn't matter. I don't see any reason to bundle the tool until it's pretty much "right". There's also no precedent of including executable tools beyond the interpreter itself for Unix, so I don't think this is an issue. Lawrence Tratt sea: > Bad idea. IMHO, the only way to create a professional tool is > to get it to use Pythons parse trees ("import"ing the modules > is at best mickey mouse even if it solves one or two problems > that static analysis can't hope to) and that really means that I agree that parse-tree analysis is required; I don't see why this precludes making the tool a little more general. If 1.5.2 support is required by the author of the tool, it's a requirement I won't argue with! I think it's a fair requirement that the modules being documented be parsable by the version of Python doing the parsing -- if a module uses list comprehensions, it's ok if the doc tool can't handle it when run under 1.5.2. Pulling interesting information out of the parse trees isn't that hard; at most, there may need to be different sets of patterns to match against based on version, but there just aren't that many versions to worry about. Supporting Jython will be harder, but also should not be too hard. (This points to using the tokenize module instead of the parser module, as well.) > It will use REs because that's "standard" and I don't want to > fight the "getting a parser into Python" battle. There's already a Python parser in the interpreter; it doesn't require building one. So this is a freebie. ;) > I don't expect *other* people to get "excited" about this (but heh, I've > had three responses, which is about, ooh, two more than I expected!). > Ultimately, the only response that *counts* will be Guido's (and I > *suspect* that also means Frank's) when something gets finished and So who is this Frank person, and why is he involved in this? My comments on the tool will inform Guido's, if that's what you're getting at. Which is not to say that he'll agree with any conclusions I make. ;) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From laurie@eh.org Mon Nov 27 17:24:45 2000 From: laurie@eh.org (Laurence Tratt) Date: Mon, 27 Nov 2000 17:24:45 +0000 Subject: [Doc-SIG] A promise References: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <3A2298DD.7D361246@eh.org> "Tony J Ibbs (Tibs)" wrote: > If we don't have a solution for Python 2.1, I will be sad, but not > devastated. On the other hand, if we don't have a solution for 2.2, I > shall be angry. Missing 2.1 wouldn't trouble me personally, although I agree 2.2 is a realistic, sensible target and gnashing of teeth would be the result of missing that particular boat. >>> 4. We need a tool that runs with bog-standard Python >>> (preferably 1.5.2, since that's "old technology", and >>> also 'cos it's what I've got available at work and at home!) >> Bad idea. IMHO, the only way to create a professional tool is >> to get it to use Pythons parse trees ("import"ing the modules >> is at best mickey mouse even if it solves one or two problems >> that static analysis can't hope to) and that really means that >> the tool has to track the latest release. As you seem to want >> to tie it to the main release anyway, this doesn't seem like a >> problem . > On this one you undoubtedly have more experience than me, but I would > *like* to be able to use the same tool for 1.5.2 as for later versions > of Python, even if in a less sophisticated manner - there are a > significant number of firms (read one, the one I work for!) who have > only just gone up to 1.5.2 (from 1.3 - can you say "hurray!" - is Eddy > watching this?) and aren't likely to go up again soon. > > But again, I won't weep too much if that isn't feasible. You have three main choices as to how to examine the Python code and work out where the docstrings are (the docstrings themselves are something seperate and probably easiser to deal with): 1) Use Pythons built in parser Pros: 100% accurate; already exists Cons: Tracks the current release (or, at least, the last time the grammar changed) 2) Create another parser with eg Spark Pros: Decouples from Python version Cons: Probably slower; argument inducing over which system to use; creating and maintaining a seperate grammar can be hard 3) Parse using regular expressions or similar Pros: Decouples from Python version; no need for a parsing system Cons: Completely crap, buggy way of doing it (I investigated it for feasibility once. It isn't) In my opinion, with this type of tool "less sophisticated" can too easily mean "not worth using". You have to go the whole hog in nearly every aspect otherwise you'll end up with a half-arsed system and we have enough of those already. Having said that, some things - like creating a sane mechanism to allow the user to denote private variables - are not necessary and if not present won't detract from the overall experience. > The thing will probably come as an "STNG-clone" supporting class with a > pre-built example of extending it to be "our tool" ('cos that's a fairly > easy way to do it and to make it possibly more useful for Zope-readers). I am slightly confused over your initial aim: are you inteding to produce a STNG parsing module on its own, first or do you intend creating a system which contains the STNG module as part of it? Now I'm worried we're talking at cross purposes! Laurie From laurie@eh.org Mon Nov 27 17:37:49 2000 From: laurie@eh.org (Laurence Tratt) Date: Mon, 27 Nov 2000 17:37:49 +0000 Subject: [Doc-SIG] A promise References: <3A2289BF.A8C5DEB4@eh.org> <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> Message-ID: <3A229BED.FD8E731C@eh.org> "Fred L. Drake, Jr." wrote: [Laurie] >> Bad idea. IMHO, the only way to create a professional tool is >> to get it to use Pythons parse trees ("import"ing the modules >> is at best mickey mouse even if it solves one or two problems >> that static analysis can't hope to) and that really means that > I agree that parse-tree analysis is required; I don't see why this > precludes making the tool a little more general. If 1.5.2 support is > required by the author of the tool, it's a requirement I won't argue > with! What one could do - at least a tool like Crystal supports this - is pluggable parsers. In fact, thinking back the default parser in Crystal was named CPython specifically so I could cater for the different interface in (as was then) JPython. However, when the grammar changes other things tend to change too, the sorts of changes that might well affect other parts of the docutils system. I may be being a little too paranoid here... But there again I wouldn't want to be the poor git maintaining parsers for Python 1.5.2, 1.6, 2.0, Jython 1.0, Jython 1.1 etc... That's just multiplying the number of chances for things to go seriously wrong. I'll put my neck on the line: long term, tracking the current Python release (even if not using the built in parser interface) is the way to go. In the short term, you might get away with coping with multiple versions but if eg the type/class dichotomy is solved (did I see a PEP for that? Can't remember), then that might have ramifications beyond the grammar probably ruining easily maintained multiple version support. > Supporting Jython will be harder, but also should not be too hard. > (This points to using the tokenize module instead of the parser module, > as well.) If you only use the tokenize module, you effectively have to write your own grammar (be that for a parser system or implicitly in code)... It just seems another chance for things to go wrong even if it does make the tool more flexible. As evidenced by the size of the tokenize module, tokenisation is the easy part of things! Laurie From akuchlin@mems-exchange.org Mon Nov 27 17:44:04 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 27 Nov 2000 12:44:04 -0500 Subject: [Doc-SIG] A promise In-Reply-To: <3A2298DD.7D361246@eh.org>; from laurie@eh.org on Mon, Nov 27, 2000 at 05:24:45PM +0000 References: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <3A2298DD.7D361246@eh.org> Message-ID: <20001127124404.F25269@kronos.cnri.reston.va.us> On Mon, Nov 27, 2000 at 05:24:45PM +0000, Laurence Tratt wrote: >You have three main choices as to how to examine the Python code and work out >where the docstrings are (the docstrings themselves are something seperate and >probably easier to deal with): 4) Use Jeremy's Tools/compiler (perhaps including a copy with the tool) Pros: Handles both 2.0 and 1.5.2, I believe. Someone is responsible for maintaining it. Cons: Not installed automatically by Python 2.0; must be packaged separately for 1.5.2. We use it for parsing Python syntax in Quixote, and everything ported from 1.5.2 to 2.0 without effort. --amk From fdrake@acm.org Mon Nov 27 17:49:34 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 12:49:34 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: <3A229BED.FD8E731C@eh.org> References: <3A2289BF.A8C5DEB4@eh.org> <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> <3A229BED.FD8E731C@eh.org> Message-ID: <14882.40622.400093.803306@cj42289-a.reston1.va.home.com> Laurence Tratt writes: > However, when the grammar changes other things tend to change too, > the sorts of changes that might well affect other parts of the > docutils system. I may be being a little too paranoid here... But > there again I wouldn't want to be the poor git maintaining parsers > for Python 1.5.2, 1.6, 2.0, Jython 1.0, Jython 1.1 etc... That's > just multiplying the number of chances for things to go seriously > wrong. What my (still unpublished) code does is use a generic tree matcher with some variables specified in the pattern, and return the matching tree and a dictionary mapping the variables to the subtrees that matched. I only needed a few patterns; we don't care about most of the Python grammar, only enough to get docstrings out. > I'll put my neck on the line: long term, tracking the current > Python release (even if not using the built in parser interface) is > the way to go. In the short term, you might get away with coping This is certainly the way to gain support for the current version. I don't think there's likely to be much difficulty with extracting most interesting information; the hardest part will be pulling apart things like parameter lists, which have changed between 1.5.2 and 2.0. There are semantic issues with that sort of information. But that's pulling a lot of things out beyond docstrings. (Though I think *that's* pretty valuable.) > with multiple versions but if eg the type/class dichotomy is solved > (did I see a PEP for that? Can't remember), then that might have > ramifications beyond the grammar probably ruining easily maintained > multiple version support. I don't think solving the type/class dichotomy has grammar implications. > If you only use the tokenize module, you effectively have to write > your own grammar (be that for a parser system or implicitly in Yes, but if we're only extracting docstrings, the grammar can be dirt simple. The difficulties really only appear when pulling out more detailed information than module/class structures and method names/docstrings. -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From laurie@eh.org Mon Nov 27 18:15:31 2000 From: laurie@eh.org (Laurence Tratt) Date: Mon, 27 Nov 2000 18:15:31 +0000 Subject: [Doc-SIG] A promise References: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <3A2298DD.7D361246@eh.org> <20001127124404.F25269@kronos.cnri.reston.va.us> Message-ID: <3A22A4C3.DBACDF3B@eh.org> Andrew Kuchling wrote: >>You have three main choices as to how to examine the Python code and work out >>where the docstrings are (the docstrings themselves are something seperate and >>probably easier to deal with): > 4) Use Jeremy's Tools/compiler (perhaps including a copy with the tool) > Pros: Handles both 2.0 and 1.5.2, I believe. Someone is responsible for > maintaining it. > Cons: Not installed automatically by Python 2.0; must be packaged separately > for 1.5.2. > > We use it for parsing Python syntax in Quixote, and everything ported > from 1.5.2 to 2.0 without effort. That does look promising (the installation thing wouldn't worry me particularly), although would I be an idiot if I said there's no obvious documentation? Laurie From laurie@eh.org Mon Nov 27 18:18:59 2000 From: laurie@eh.org (Laurence Tratt) Date: Mon, 27 Nov 2000 18:18:59 +0000 Subject: [Doc-SIG] A promise References: <3A2289BF.A8C5DEB4@eh.org> <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> <3A229BED.FD8E731C@eh.org> <14882.40622.400093.803306@cj42289-a.reston1.va.home.com> Message-ID: <3A22A593.B7578874@eh.org> "Fred L. Drake, Jr." wrote: >> I'll put my neck on the line: long term, tracking the current >> Python release (even if not using the built in parser interface) is >> the way to go. In the short term, you might get away with coping > This is certainly the way to gain support for the current version. That's another way of looking at it :) > I don't think there's likely to be much difficulty with extracting > most interesting information; the hardest part will be pulling apart > things like parameter lists, which have changed between 1.5.2 and > 2.0. There are semantic issues with that sort of information. But > that's pulling a lot of things out beyond docstrings. (Though I think > *that's* pretty valuable.) I ended up being surprised at how much information in the grammar I made use of in the end... Allowing doc strings for inner classes, class variables etc means you need to move beyond brute strength, which was the first thing I tried. Andrews suggestion of Tools/compiler is certainly something I would suggest be given further thought. Laurie From fdrake@acm.org Mon Nov 27 18:31:23 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 13:31:23 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: <3A22A4C3.DBACDF3B@eh.org> References: <008501c05893$7827b980$f05aa8c0@lslp7o.int.lsl.co.uk> <3A2298DD.7D361246@eh.org> <20001127124404.F25269@kronos.cnri.reston.va.us> <3A22A4C3.DBACDF3B@eh.org> Message-ID: <14882.43131.25487.20897@cj42289-a.reston1.va.home.com> Laurence Tratt writes: > That does look promising (the installation thing wouldn't worry me > particularly), although would I be an idiot if I said there's no obvious > documentation? A quick look reveals at least one docstring! ;) I've filed a feature request for documentation for this, and assigned it to Jeremy. He's always telling me he's forgetting what documentation projects are on his plate; now his own reminder script can send him mail about it! ;) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From klm@digicool.com Mon Nov 27 18:52:39 2000 From: klm@digicool.com (Ken Manheimer) Date: Mon, 27 Nov 2000 13:52:39 -0500 (EST) Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: <20001125170118.CC4591CE0D@dinsdale.python.org> Message-ID: I like that you're nailing down your specs. There are a couple of things you're suggesing changing in structured text that i would like to see changed - most prominent, allowing bullets without intervening white space, etc. However, i much prefer reading text with indentation to indicate section structure! The cost of a 14 or 16 character right margin for level *7* sections seems not so bad, and i would be dubious of extensive level 7 sections, besides. I also don't like use of '\' for quoting. I don't see a hint of intuitive motivation for it, except maybe for programmers familiar with shell/C/programming-in-general quoting - which seems like a different venue. I'd like to stick with having people do literal quoting by using code ('single-quote' bracketed) and "example::" - these are self-evident in the raw form. (If you're trying to cater to existing "natural" text, i doubt you'll find use of '\' - and i think you're imposing an unfeasible requirement, anyway. As i see it, the goal is not to provide for every reasonable convention, but rather to provide a reasonable set of acceptible conventions that people can adopt as the common basis.) About tables - *nobody* likes the current structured text abomination for tables. It betrays the principles (naturalness), and provides woefully inadequate capability, as a bonus.-) Jim has a proposal for a much more comprehensive provision, which looks fairly similar to the one you sketch out: http://www.zope.org/Members/jim/StructuredTextWiki/StructuredTextTables At cursory glance, they look extremely similar - jim's provides for table header rows, which may be nice. Both suffer from requiring a lot of typing from the user for elaborate tables, and maybe some complications in changing the table contents if it bursts the boundaries of what you already have - but considering how hard it is to create elaborate tables in html, maybe that kind of overhead is inevitable. Both are really nice in that they provide for arbitrary (structured) text in the cells - i think this is proper. Overall, i'm really glad to see you paying careful attention to this, and i'm hoping you and tony and whoever else is interested can pool your efforts, make it work... Ken klm@digicool.com From robin@alldunn.com Mon Nov 27 19:54:02 2000 From: robin@alldunn.com (Robin Dunn) Date: Mon, 27 Nov 2000 11:54:02 -0800 Subject: [Doc-SIG] Problems With StructuredText References: Message-ID: <01ac01c058ab$cadc48e0$3225d2d1@ARES> > 1. This format is the most natural and obvious. I came up with it (no great > feat of creation!), and later discovered that it is the format supported > by the [Emacs table mode]_:: > > +------------+------------+---------------------------+ > | Column 1 | Column 2 | Column 3 & 4 span (Row 1) | > +------------+------------+------------+--------------+ > | Column 1 & 2 span | Column 3 | - Column 4 | > +------------+------------+------------+ - Row 2 & 3 | > | 1 | 2 | 3 | - span | > +------------+------------+------------+--------------+ > > .. _Emacs table mode: > ftp://archive.cis.ohio-state.edu/pub/emacs-lisp/archive/table.el > I just played with this emacs table mode and I have to say it is very awesome! It makes editing tables just as easy in plain text as it is in a word processor. What ever is chosen as the standard for structured text markup, I will be very disappointed if it doesn't allow defining tables like this. It is very intuitive, very readable in it's plain text form, and with table.el for emacs very easy to create and edit too. -- Robin Dunn Software Craftsman robin@AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython! From Moshe Zadka Mon Nov 27 21:43:26 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 27 Nov 2000 23:43:26 +0200 (IST) Subject: [Doc-SIG] A promise In-Reply-To: <14882.34216.951538.940039@cj42289-a.reston1.va.home.com> Message-ID: [Warning -- 0.5 Tongue in Cheek] On Mon, 27 Nov 2000, Fred L. Drake, Jr. wrote: > The > package in the standard library should probably be called "docutils", > after Greg Ward's "distutils", but I've not a clue as to what to call > the command-line tool (well, not any good clues). How about python -c 'import docutils;docutils.main()' Sorry, it's 12pm, and I probably need some sleep. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From fdrake@acm.org Mon Nov 27 21:43:16 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 27 Nov 2000 16:43:16 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: References: <14882.34216.951538.940039@cj42289-a.reston1.va.home.com> Message-ID: <14882.54644.247773.488057@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > python -c 'import docutils;docutils.main()' I know how to *spell* it, but how's it *pronounced*? > Sorry, it's 12pm, and I probably need some sleep. Yes, please! ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From Moshe Zadka Tue Nov 28 02:33:09 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 28 Nov 2000 04:33:09 +0200 (IST) Subject: [Doc-SIG] A promise In-Reply-To: <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> Message-ID: On Mon, 27 Nov 2000, Fred L. Drake, Jr. wrote: > There's also no precedent of including executable tools > beyond the interpreter itself for Unix, so I don't think this is an > issue. OK, it's 4am so maybe I'm totally spaced out, but isn't the Tools/ directory *choke full* of executable tools beyond the interpreter itself? They're written in Python, but I doubt docutils is going to be written in Python too. -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From dgoodger@bigfoot.com Tue Nov 28 02:43:47 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Mon, 27 Nov 2000 21:43:47 -0500 Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: Message-ID: on 2000-11-27 13:52, Ken Manheimer (klm@digicool.com) wrote: > However, i much prefer reading text with > indentation to indicate section structure! The cost of a 14 or 16 > character right margin for level *7* sections seems not so bad, and i > would be dubious of extensive level 7 sections, besides. Granted, this may be a personal preference issue. My 'wasted space' argument is minor, compared to the ambiguity and difficulty of using indentation for section structure. It would be painful to write up indented text using SimpleText or NotePad. I think requiring the use of Emacs or VI (or equivalent) would decimate the potential user base of this format. Even with Emacs, I would code my own extension rather than use indentation where indentation isn't warranted. As for ambiguity, what do we do if the 'parent' paragraph is 2 lines long, and the 'child' is indented, ie:: This is the parent paragraph. Here is the child. StructuredText allows for 1-line parents as titles. Would the above example become a block quote? If so, how would one code a block quote with a 1-liner parent paragraph? Again, personal preference is probably the overriding factor. > I also don't like use of '\' for quoting. I indented '\' for *escaping* only, vehemently *not* for quoting. An escaping mechanism is needed, as has been discussed ad nauseum on this mailing list and elsewhere, including http://www.zope.org/Members/jim/StructuredTextWiki/NGEscaping. Escaping the markup will be only rarely needed, but without it certain tasks (like documenting Structured Text!) become very difficult. For most cases, code blocks are better, I agree. But *something* is needed, and I see no better alternative. Single-quotes for inline code markup prevents us from using them for their natural purpose. It is a wart that should be removed. Ever compared a novel published in the US to one published in the UK? Notice the quotes around dialog. > As i see it, the goal is not to provide for > every reasonable convention, but rather to provide a reasonable set of > acceptible conventions that people can adopt as the common basis.) This statement reinforces my rejection of single quotes for anything but their ordinary language purposes. > About tables ... Jim has a proposal for a > much more comprehensive provision, which looks fairly similar to the > one you sketch out: > > http://www.zope.org/Members/jim/StructuredTextWiki/StructuredTextTables This must have been a recent addition; didn't see it last time I visited the site. Yes, it's quite similar. It lacks the use of '+' at intersections, though; '+'s make parsing the table simple, and allow '|' within cell text. Jim's suggestion is slightly easier to type. > Overall, i'm really glad to see you paying careful attention to this, > and i'm hoping you and tony and whoever else is interested can pool > your efforts, make it work... Thank you. I hope so too! -- David Goodger dgoodger@bigfoot.com Open-source projects: - The Go Tools Project: http://gotools.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net (soon) From klm@digicool.com Tue Nov 28 04:14:25 2000 From: klm@digicool.com (Ken Manheimer) Date: Mon, 27 Nov 2000 23:14:25 -0500 (EST) Subject: [Doc-SIG] reStructuredText: Revised Structured Text Specification In-Reply-To: Message-ID: On Mon, 27 Nov 2000, David Goodger wrote: > on 2000-11-27 13:52, Ken Manheimer (klm@digicool.com) wrote: > > However, i much prefer reading text with > > indentation to indicate section structure! The cost of a 14 or 16 > > character right margin for level *7* sections seems not so bad, and i > > would be dubious of extensive level 7 sections, besides. > > Granted, this may be a personal preference issue. My 'wasted space' argument > is minor, compared to the ambiguity and difficulty of using indentation for > section structure. It would be painful to write up indented text using > SimpleText or NotePad. I think requiring the use of Emacs or VI (or I think that neither the ambiguity nor the difficulty are clear-cut cases in either direction. Indentation is a nearly ideal cue for local sectional relationships - it makes it abundantly clear which sections are deeper than which. In fact, i usually *prefer* the raw structured text content to the html-rendered version, because i find the indentation to be a way more self-evident cue than the section header font sizes. (For some more personal-preference territory, i also find it clearer than your underline styles, and i find the underlines clutterful.) Conversely, indentation is harder to compare across large distances in a document, unless you can line up the columns and compare, or your editor has a ruler (i think wordpad does, not sure about notepad). Of course, if the issue at hand is docstrings, you don't have to worry about such large bodies of text. As far as difficulty, i don't quite see it. I often use structured-text style organization in email messages (for memos and such), and edit using pico - which doesn't do fancy indented-block flows like emacs or vim. I spend some time getting things to line up, but not a lot - and for text that's going to be processed, you don't need to worry about the left margin, the text is going to be flowed by, eg, html, anyway! I don't mean to dismiss either of these things, i realize they can be real drawbacks in some situations (large documents and deficient tools). However, when it comes to python docstrings, people are already going to have to be dealing with indenting the python code - if they can deal with that, they can probably deal with indenting paragraphs. > equivalent) would decimate the potential user base of this format. Even with > Emacs, I would code my own extension rather than use indentation where > indentation isn't warranted. As for ambiguity, what do we do if the 'parent' > paragraph is 2 lines long, and the 'child' is indented, ie:: > > This is the parent > paragraph. > > Here is the child. > > StructuredText allows for 1-line parents as titles. Would the above example > become a block quote? If so, how would one code a block quote with a 1-liner > parent paragraph? I may be missing something here. Punctuation at the end of the line, quite suitably, disqualifies it as a header. And i didn't think that structured text presents anything as a blockquote! It's not a bad idea, but probably should be constrained (to avoid the problem you're sketching), maybe to an indented block that begins and ends with '"' double quotes. > Again, personal preference is probably the overriding factor. But keep in mind that there are fine lines here - structured text embodies some difficult-to-attain economies, pretty successfully, and i think it's easier at this point to disrupt those economies than it is to improve them. I don't mean to say there's not room for improvement, by any means - but consider your above example, where structured text was doing the right thing for good reasons, even when you didn't expect it to do so. There's some cruft in it (footnotes and tables come immediately to mind), but it's generally not easy to twiddle it to advantage. > > I also don't like use of '\' for quoting. > > I indented '\' for *escaping* only, vehemently *not* for quoting. An Whoops - i meant escaping, not quoting. Inline code blocks are *supposed* to inhibit other processing, and example:: blocks should do so also - with the one side effect that they turn the text to fixed-width fonts. > escaping mechanism is needed, as has been discussed ad nauseum on this > mailing list and elsewhere, including > http://www.zope.org/Members/jim/StructuredTextWiki/NGEscaping. Escaping the > markup will be only rarely needed, but without it certain tasks (like > documenting Structured Text!) become very difficult. For most cases, code > blocks are better, I agree. But *something* is needed, and I see no better > alternative. > > Single-quotes for inline code markup prevents us from using them for their > natural purpose. It is a wart that should be removed. Ever compared a novel > published in the US to one published in the UK? Notice the quotes around > dialog. I guess at this point the most valuable thing would be to see some motivating examples - reasonable things people want to do where inline code markup and example:: blocks won't suffice. I don't mean to question the need, but think that being specific will probably help identify exactly what the need is - the requirements. > This must have been a recent addition; didn't see it last time I visited the > site. Yes, it's quite similar. It lacks the use of '+' at intersections, > though; '+'s make parsing the table simple, and allow '|' within cell text. > Jim's suggestion is slightly easier to type. According to the change history (not yet exposed to non-manage accounts, should be soon, when i get done w/WikiForNow), nov 17, just 10 days ago... > > Overall, i'm really glad to see you paying careful attention to this, > > and i'm hoping you and tony and whoever else is interested can pool > > your efforts, make it work... > > Thank you. I hope so too! Cool. Ken klm@digicool.com From fdrake@acm.org Tue Nov 28 06:18:04 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 28 Nov 2000 01:18:04 -0500 (EST) Subject: [Doc-SIG] A promise In-Reply-To: References: <14882.38506.702518.287799@cj42289-a.reston1.va.home.com> Message-ID: <14883.19996.807211.823293@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > OK, it's 4am so maybe I'm totally spaced out, but isn't the Tools/ > directory *choke full* of executable tools beyond the interpreter > itself? They're written in Python, but I doubt docutils is going > to be written in Python too. Oops! My fault -- I wasn't clear. None of that stuff is *installed* on Unix. And IDLE is the only thing from that "installed" on Windows. (And I think it *should* be installed on Unix as well, along with whatever other tools make sense, and their man pages.) -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations From Moshe Zadka Tue Nov 28 07:24:58 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 28 Nov 2000 09:24:58 +0200 (IST) Subject: [Doc-SIG] A promise In-Reply-To: <14883.19996.807211.823293@cj42289-a.reston1.va.home.com> Message-ID: On Tue, 28 Nov 2000, Fred L. Drake, Jr. wrote: > > Moshe Zadka writes: > > OK, it's 4am so maybe I'm totally spaced out, but isn't the Tools/ > > directory *choke full* of executable tools beyond the interpreter > > itself? They're written in Python, but I doubt docutils is going > > to be written in Python too. > > Oops! My fault -- I wasn't clear. None of that stuff is > *installed* on Unix. And IDLE is the only thing from that "installed" > on Windows. (And I think it *should* be installed on Unix as well, > along with whatever other tools make sense, and their man pages.) OK, define "UNIX". What you mean is probably "not installed by the default source install". Well, guess what -- many people are not using the default source install, but rather packaging systems. On Debian, for example, "apt-get install task-python-dev" installs IDLE too. And when there are other important tools, I'm guessing they'll be Debianized too. There was a discussion in Python-Dev not long ago about installing more things with the source install -- not that it got anywhere once we realized distutils doesn't support executables yet. A worth goal, but for another sig . -- Moshe Zadka -- 95855124 http://advogato.org/person/moshez From tony@lsl.co.uk Tue Nov 28 10:39:38 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Tue, 28 Nov 2000 10:39:38 -0000 Subject: [Doc-SIG] A promise In-Reply-To: <14882.40622.400093.803306@cj42289-a.reston1.va.home.com> Message-ID: <008e01c05927$823df8f0$f05aa8c0@lslp7o.int.lsl.co.uk> A quick swathe of odd replies to various messages - gosh, everyone was busy after I went home last night. OK. We have three components to this thing: 1. Find the docstrings (and (eventually) provide information for references 'tween them) 2. Parse the docstrings 3. Emit the documentation As said before, it looks like I can "steal" 1, probably from Fred's code initially (not Frank, not Frank - I'll have to write that 1000 times on the whiteboard.). And then maybe Tools/compiler - I'll have a quick look at that. However, as someone else said, which tool is used only *really* becomes important when one wants cross-references to be disambiguated correctly (unless one is willing to make up "labels" and "names" and hope they'll match up in the wash, so to speak - rather like the way Unix handles dynamic linking!). And I think everyone is agreed that "importing" modules just to document them is not on. Parsing of the docstrings is what I've been most concerned with, and is currently using REs, 'cos that's what other things were using. It thus won't be as powerful as it might be (at least at first), 'cos I hate REs and don't want to write the nifty tricks one needs to do to embed strong in italic, etc. If I had my ideal way, I'd parse the docstrings with mxTextTools, 'cos I like them. But since I assume they're not going to be in the standard distribution, that's out. For 3, I'm assuming that STNG's approach of phase 2 producing a DOM structure is sensible, and thus the tool will eventually do that (it doesn't yet) - that should make it very easy to plug a formatter in. As to the format itself - I like the indentation. I think people overestimate how *much* they need to use indentation until they've written ST[NG] a bit. I also think (and stuff seen from Jim and co. confirms me in this - well, actually, I stole the idea) that ST[NG] users find that the problems they come up with *in theory* don't often come up in practise, because of the design of the thing - with the obvious exceptions of *our* perceived need to handle code specially in-line, and thus a need for quoting that has been able to be put off for a while in ST. I'd write more on that if I had time (the things that disturb me about STNG are more things like "coherence of lists" than indentation). *But* what I want to do is produce A Tool which can read docstrings containing STPy (and preferably STNG as well), and produce documentation from it. Soon. Then we can criticise and reform a body of code that already (mostly) does the job we want, *as well as* refining the rest of the documentation standard itself (so in a sense it's a good thing that's already started, *provided* the tool still gets written - although I wish there was another me to argue the format as well). As others have said, if it misses 2.1, so be it - heck, 2.1 may miss 2.1 (so to speak). Tibs - must dash, paid work to do -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From klm@digicool.com Tue Nov 28 17:30:10 2000 From: klm@digicool.com (Ken Manheimer) Date: Tue, 28 Nov 2000 12:30:10 -0500 (EST) Subject: [Doc-SIG] table.el? Message-ID: A few people have mentioned a table.el elisp file for a nifty table mode for emacs. I can't find it - can someone point me to it? Tnx, Ken klm@digicool.com From robin@alldunn.com Tue Nov 28 17:47:39 2000 From: robin@alldunn.com (Robin Dunn) Date: Tue, 28 Nov 2000 09:47:39 -0800 Subject: [Doc-SIG] table.el? References: Message-ID: <00fb01c05963$4dbed400$3225d2d1@ARES> > A few people have mentioned a table.el elisp file for a nifty table mode > for emacs. I can't find it - can someone point me to it? > ftp://archive.cis.ohio-state.edu/pub/emacs-lisp/archive/ -- Robin Dunn Software Craftsman robin@AllDunn.com http://wxPython.org Java give you jitters? http://wxPROs.com Relax with wxPython! From fdrake@users.sourceforge.net Wed Nov 29 06:20:31 2000 From: fdrake@users.sourceforge.net (Fred L. Drake) Date: Tue, 28 Nov 2000 22:20:31 -0800 Subject: [Doc-SIG] [development doc updates] Message-ID: <200011290620.WAA09888@orbital.p.sourceforge.net> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ From tony@lsl.co.uk Wed Nov 29 12:37:36 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Wed, 29 Nov 2000 12:37:36 -0000 Subject: [Doc-SIG] A promise - using Tools/compiler (yay) In-Reply-To: <20001127124404.F25269@kronos.cnri.reston.va.us> Message-ID: <009901c05a01$27f18260$f05aa8c0@lslp7o.int.lsl.co.uk> AMK suggested that it would be good to: > 4) Use Jeremy's Tools/compiler (perhaps including a copy with > the tool) and this indeed appears to be exactly what I want. It's not even too hard to figure out how to use it, given the demo example (the main trick is to work out how one's supplied visitor class can call "visit" without defining it, I think). Oh frabjous days. One minor note - a first attempt at:: import compiler under 1.5.2 gives:: Traceback (innermost last): File "docstring.py", line 17, in ? import compiler File "compiler/__init__.py", line 23, in ? from pycodegen import compile File "compiler/pycodegen.py", line 12, in ? from compiler import pyassem, misc File "compiler/pyassem.py", line 138, in ? class PyFlowGraph(FlowGraph): File "compiler/pyassem.py", line 299, in PyFlowGraph _cmp = list(dis.cmp_op) AttributeError: cmp_op but commenting out the line:: from pycodegen import compile from __init__.py makes the traceback go away, and since I don't *want* to work with bytecodes, this may be OK in my context. It is, perhaps, something that could be "robustified" if it's to be formally supported under 1.5.2 - but does anyone care? Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From fdrake@users.sourceforge.net Thu Nov 30 07:33:55 2000 From: fdrake@users.sourceforge.net (Fred L. Drake) Date: Wed, 29 Nov 2000 23:33:55 -0800 Subject: [Doc-SIG] [development doc updates] Message-ID: <200011300733.XAA12728@orbital.p.sourceforge.net> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Today's changes are primarily in the xml.dom API documentation: http://python.sourceforge.net/devel-docs/lib/markup.html From tony@lsl.co.uk Thu Nov 30 14:34:18 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Thu, 30 Nov 2000 14:34:18 -0000 Subject: [Doc-SIG] docutils status report Message-ID: <00ae01c05ada$9f043f80$f05aa8c0@lslp7o.int.lsl.co.uk> Well, the Tools/compiler hint was just what I needed - it took very little time to start extracting docstrings from a Python file, and it's fairly clear how one would manage a package. Anyway, here is a status report and some "to do" information: I currently have some code that will: 1. Find the docstrings in a Python file (currently itself!) 2. Split the text into paragraphs at blank lines. 3. Identify lines within a paragraph that start like a list item, and split there as well - this allows:: This is a paragraph. 1. So is this fred -- and so is this to produce 3 paragraphs, and I think this is the main place that Python users want/need to be able to have "no blank lines". 4. Identify subparagraphs of a paragraph ending "::" as literal text (note that, as in STNG, the "::" is enough text to constitute "text in a paragraph", so the "empty header" idea will work). 5. Identify paragraphs starting ">>>" (allowing leading whitespace) as Python code (i.e., literal) 6. Recognise bullet list items (as in ST - the use of "o" may go away following David's comments) 7. Recognise numbered list items (but the final dot *is* required, otherwise 3 above will fail on:: My favourite drink is tea, but also I like coffee (thinking the second line to have a Roman numeral at the start!)). Note that one won't be able to do:: And the final number is 1. without a spurious list, but I reckon we can live with that! (We have to pay for apparent simplicity with true complexity.) 8. Recognise descriptive list items (note that markup is allowed in the "title" of the item, so one can do:: ' -- ' -- This is an awkward case 9. Recognise *emphasised* text, **strong** text and 'literal' text (but, except by "accident" nesting of markup does not work (well, you can't *nest* markup in literal, since it won't be seen!)). The emphasised and strong texts may contain any characters (except the terminating sequence, of course), and inline literals may contain anything but "'". Escaping characters is not yet addressed. 10. Emit a "dump" of the datastructure that is built up. Note that the markup done is handled in a way that should be simple to customise - I may well provide the final product as an example of this, with an "STNG-like" core and the pyST extras as "extension". I'd prefer not to make code public until I've got stuff DOM structured (see below), but if anyone *really* wants to, I can make it available for download. Things to do next ----------------- (in no particular order) include: A. Move the datastructure to a DOM model (probably based on Python 2.0's mindom.py). DOM looks sensible because it is (erm) fashionable, and if I use minidom then I get XML output for free. B. Add more markup (I've got a bare minimum for testing at the moment) C. Define what the command line interface is (i.e., how to specify that one wants to parse a file or package, what one wants the output to be, and so on.) D. Document what it does, so that David and company can haggle over the exact syntax supported. This obviously includes making sure the whole thing has nice (correct) docstrings throughout. E. Make nested markup work, so one can do:: *This is **strong and 'literal'** text within emphasised* I do *not* intend to provide support for tables! (that can come later). Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Give a pedant an inch and they'll take 25.4mm (once they've established you're talking a post-1959 inch, of course) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From fdrake@acm.org Thu Nov 30 15:50:48 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 30 Nov 2000 10:50:48 -0500 (EST) Subject: [Doc-SIG] docutils status report In-Reply-To: <00ae01c05ada$9f043f80$f05aa8c0@lslp7o.int.lsl.co.uk> References: <00ae01c05ada$9f043f80$f05aa8c0@lslp7o.int.lsl.co.uk> Message-ID: <14886.30552.62165.180721@cj42289-a.reston1.va.home.com> --KkM6q1xuaq Content-Type: text/plain; charset=us-ascii Content-Description: message body and .signature Content-Transfer-Encoding: 7bit Tony J Ibbs (Tibs) writes: > 7. Recognise numbered list items (but the final dot *is* required, > otherwise 3 above will fail on:: I like this adjustment. > Note that one won't be able to do:: > > And the final number is > 1. > > without a spurious list, but I reckon we can live with that! > (We have to pay for apparent simplicity with true complexity.) Perhaps also require some text after the number in order to break the paragraph and start a numbered list. Seems like a reasonable sanity check. > A. Move the datastructure to a DOM model (probably based on > Python 2.0's mindom.py). DOM looks sensible because it is > (erm) fashionable, and if I use minidom then I get XML > output for free. I'm not sure what you mean a "DOM model"; why not use the DOM with a specific structure? I've attached a short DTD I came up with while thinking about structured text & such. ;) > D. Document what it does, so that David and company can haggle over the > exact syntax supported. This obviously includes making sure the > whole thing has nice (correct) docstrings throughout. There's documenting the code, and the format. Concentrate on the later before worrying about the former. -Fred -- Fred L. Drake, Jr. PythonLabs at Digital Creations --KkM6q1xuaq Content-Type: text/plain Content-Description: potential STNG DTD Content-Disposition: inline; filename="basic.dtd" Content-Transfer-Encoding: 7bit --KkM6q1xuaq-- From tony@lsl.co.uk Thu Nov 30 16:11:27 2000 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Thu, 30 Nov 2000 16:11:27 -0000 Subject: [Doc-SIG] docutils status report In-Reply-To: <14886.30552.62165.180721@cj42289-a.reston1.va.home.com> Message-ID: <00b001c05ae8$315bdbb0$f05aa8c0@lslp7o.int.lsl.co.uk> Fred L. Drake, Jr. wrote (in reponse to my quoted text): > > Note that one won't be able to do:: > > > > And the final number is > > 1. > > > > without a spurious list, but I reckon we can live with that! > > (We have to pay for apparent simplicity with true complexity.) > > Perhaps also require some text after the number in order to break > the paragraph and start a numbered list. Seems like a reasonable > sanity check. Ah - that's the idea I was missing. Thanks. > I'm not sure what you mean a "DOM model"; why not use the DOM with > a specific structure? I *think* that's just my bad terminology showing... > I've attached a short DTD I came up with while > thinking about structured text & such. ;) [..has quick look..] Hmm, insofar as I understand DTDs (and one doesn't need to to read that!) I think that sums up what I want. I'll know more about using the DOM later this week... > There's documenting the code, and the format. Concentrate on the > later before worrying about the former. I'm constitutionally unable to avoid writing comments in code (no, me verbose!), but documenting the format is certainly what I prefer to spend time doing (partly 'cos that *is* the spec, and redrafting the spec in light of experience is a Good Thing, and of course feeds back into the code). I can live with empty docstrings (or even slightly wrong ones) for a while if needs be... Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Well we're safe now....thank God we're in a bowling alley. - Big Bob (J.T. Walsh) in "Pleasantville" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)