[Doc-SIG] Re: docstring signatures (answer to Guido)

[Guido van Rossum]

> Oh dear. Well, thanks to Edward for finally getting the BDFL's opinion
> on the "top of the callable docstring" thing - wish I could find the
> reference where it was claimed to be needed. Maybe I was remembering a
> dream, or living in a parallel world, or something. I'll happily drop
> them from the spec (and stop using them!).

You probably saw the docstrings on some *extension* modules, where
the signature is generally included.

> I *am* a bit disturbed, though, as to whether Guido has decided against
> an ST approach *in all circumstances* (I'm sure he was less negative on
> previous rounds of the Doc-SIG, but I'm not entirely prepared to trust
> my memory at this stage).

That's because now I have actual experience using ST (in ZWiki).

> I do know that *I* want some "light formatting" solution, and so have
> other participants of the SIG - and given this is Python-land, I don't
> believe that's because we're trying to "nanny" people, I think it's
> because we want it *for ourselves*.
> 
> I hope the rest of this email doesn't come across as ranting against
> Guido. But I *do* feel he's being a little bit unfair...
> 
> In response to Edward Loper, Guido wrote:
> > Not true.  Most of the standard library uses the same convention, and
> > even if it's not quite written down, it wouldn't be hard to figure out
> > what it is.
> 
> Hmm. But this is where we *came* from, initially, surely - an attempt to
> figure out what people *actually* write down. Asking people to conform
> to a convention that is *not* evident explicitly somewhere is, well, a
> bit unfair (I include me in "people" here, by the way).

You cut out the part where I pointed out that it *is* explicit -- in
the style guidelines, which haven't been challenged.

> > >     As a result, it is very difficult to write widely-applicable
> > >     tools for processing documentation strings.
> >
> > Again not true.  Ping's pydoc does quite well second-guessing the
> > existing conventions.
> 
> This has been a great source of argument on Doc-SIG in the past - "quite
> well" is not the goal that some of us wanted. But it's still not the
> only reason why many of us want ST<whatever> - we actually want to have
> some markup in the text for all sorts of reasons.

There are only two choices.  Either you have markup or you don't.  If
you design a markup system, it should be complete, and allow full
control over the lay-out -- including full control in cases where you
*don't* want the special characters to have effect.  ST is neither
markup nor "not markup", and that's why it fails, in my view.

> > >         * Safety: No well-formed formatted documentation
> > >           string should result in unexpected formatting.
> >
> > This is a good one.  ST loses big here!
> 
> I *do* feel that it is a *leetle bit* (excuse the sarcasm) unfair to
> judge the STpy and STminus works on the basis of a tool/specification
> that they are not. As far as I can tell, STClassic (the implementation)
> is *not* a very good example of how to do it (and that's meant to be
> english understatement). And that seems to be what Guido is basing this
> statement on.

Sure.  As I'm not a subscriber to this list, I was not aware of those,
and nobody has bothered to forward me a pointer to a specification.
(I typed them into Google but got no useful hits.)

> > The proponents of ST (that I've talked to) seem to believe that it's
> > unnecessary to tell the users what the exact rules are.
> 
> Yes, but that wasn't *us* - we're proponents of (a form of) ST as well.
> But obviously just not *those* proponents.

Then you did a poor job of distinguishing yourself.  Thanks for
clarifying.

> > This, plus numerous bugs in the ST implementation and the context
> > in which it is used, continuously bite me.
> 
> Again, it's surely a bit unfair to say (as this does) "an implementation
> of an ancestor specification sucked, so what you're doing does as well".

Well, you have associated yourself with it by choosing the same
moniker.  I see that you are trying to dissociate yourself now.  OK,
I'll give it a shot.  But show me the specs please!

> > E.g. if a paragraph starts with a word
> > followed by a period, the word is replaced with "1.".
> 
> I agree that's loony. But it's not what is being proposed.
> 
> > If I use "--" anywhere in the first line of a paragraph
> > it is turned into a <DT>...<DD>... style construct.
> 
> Well, ' -- ' in our version - predicated surely on the idea that most
> people don't use double hyphens in plain text (which I happen to believe
> as well), whereas the:
> 
> 	something -- some text about it
> 
> style is fairly easy to spot.

Then we disagree -- I use double hyphens in text *all the time* -- and
I know I'm not alone.  Unless I misunderstand what you propose.

> >  There's no easy way to escape HTML keywords.
> 
> A problem of *that* specification, not of STpy or STminus (and
> *aggressively* not so). We do *not* weld ourselves to HTML as an output
> format, nor indeed XML, and thus '<' and '>' are not treated specially
> at all.
> 
> > In general, when you *don't* want something to have its
> > special effect, there's no way to escape it.
> 
> A problem, agreed - but we've actively been worrying about this, and
> looking for *specific cases* where this causes a problem, to see if we
> can work around it. I'd be very interested to know which cases cause
> Guido problems (and if they're artefacts of the earlier specifications,
> or something we can use as examples of problems for ourselves).

Without knowing your ruleset I can't know what the problems are, of
course.

> > There's no way that I know of to create a bulleted item
> > consisting of several paragraphs.
> 
> This is a lunacy of the implementation Guido's been using, I would say.

I would hope so.

> > The reliance on indentation levels to detect various levels of
> > headings never works for me.
> 
> Well, I don't like it either. It won't stop me writing a PEP, though,
> which does the same thing (and is, of course, pretty close to being
> written in STpy/STminus).

Ah, but the "pretty close" is exactly what's wrong.  PEPs are written
in plain text, and there's not enough information to know when to
interpret characters as markup and when not to.  E.g. a PEP describing
ST would be filled with examples of ST markup -- if the PEP is written
in plaintext, these examples don't need any special quoting, but if it
is written in ST, they must be quoted.  (And don't tell me to put all
examples in literal blocks -- inline examples are essential.)  The
PEP-to-HTML processor uses only one strict rule, and a few heuristics:
it uses unindented text for headings (but it doesn't have multiple
heading levels), and it turns things looking like URLs into
hyperlinks.  But other than that it doesn't use any markup characters,
and even line breaks in the original text are honored exactly in the
HTML.

> For what it's worth, there *are* proposals to fix that (the section =
> indentation thingy), but they're not worth pursuing until we have
> something available to talk about, which is what we've been trying to
> do.

Sorry, you've lost me here.

> > > Any info you can give on what you would like to see come out
> > > of this project (or pointers to info) would be most appreciated.
> >
> > A lot of effort has gone into creating a body of documentation for the
> > standard library *outside* the source code.  It is rich in mark-up,
> > indexed, contains many examples, well maintained, and is generally
> > considered high quality documentation.  I don't want to have to redo
> > the work that went into creating this.
> 
> Of course not. We're not attempting to change that (at least Edward and
> I are not).

OK.  The factions in the doc-sig are hard to keep apart for an
outsider.  At the conference I met some people who wanted to prescribe
that all module documentation be maintained in the source code file.
I find that insanity.

> > It should be easier to combine code and documentation for 3rd party
> > modules, and there should be an easier way to integrate such
> > documentation into (a version of) the standard documentation.  But I
> > disagree with the viewpoint that documentation should be maintained in
> > the same file as source code.  (If you believe the argument that it
> > is easier to ensure that it stays up to date, think again.  This never
> > worked for comments.)
> 
> I'm afraid you (Guido) are conflating two different arguments. The
> argument by Ka-Ping Yee that the *whole* documentation for a module
> should live in the file for that module is (a) a different thread, and
> (b) one I argue strongly against.

I'm so glad to hear that.  See above. :-)

> I hope that Guido isn't already decided on this issue, once and for all.

No, I'm always open to reasonable input.  I'm waiting for your spec so
I can form an opinion on it.

> The consensus of the Doc-SIG, over the years (many of whom have been
> people who Guido knows and has much more reason to respect the opinion
> of than me) has been that we need a way of formatting docstrings, and
> that it should be an etext derivative.

(But I've never agreed so far, and my one prolonged experience with an
ST-based system makes me hate every bit of it.  It could be that
implementation though.)

> I believe that we (on the
> Doc-SIG) have been producing a *better* etext derivative, whilst still
> trying to stay at least partially compatible with the sibling STNG
> project.

I don't think that compatibility with a possibly broken alternative
ought to be constraining you.

> *Should* we have been more radical and just broken with STNG entirely?
> It would have made life somewhat simpler...

I don't know much about STNG either -- I always thought that it was an
idea for a project to fix the problems with ST, but not anything
concrete.  I am not aware of any part of Zope actually using STNG, so
I doubt that interoperability can be a big issue.

--Guido van Rossum (home page: http://www.python.org/~guido/)