On Thu, Dec 3, 2009 at 3:00 AM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

On Dec 3, 2009, at 1:29 AM, Kevin Horn wrote:

On Wed, Dec 2, 2009 at 8:38 PM, <exarkun@twistedmatrix.com> wrote:

As far as the (I'm going to call it) roadmap goes, the thought that's
pushed its way to the fore for me is that I'd like to try this with
something smaller and simpler than Twisted first.  It would be nice if
the Divmod projects would qualify here, but they may not be active
enough for any real experience to accumulate.

I haven't run my lore2sphinx script against the Divmod stuff yet, but I could certainly try it.  The intention is to convert the Divmod stuff as well as the Twisted stuff eventually though.

I've been concentrating on the Twisted docs, since they seemed the highest priority, but I could detour a bit if you like.  Obviously that would slow progress on the Twisted stuff a little.

While converting pyOpenSSL's documentation would be a worthwhile goal in itself, I don't actually like this idea.

Mainly I disagree with the premise that accumulation of experience is necessary for the conversion.  But this is not a strong disagreement, as I don't understand the motivation for saying so in the first place :).

The beauty of this plan, as far as I see it, is that the only person who has to get any significant experience with Sphinx in order for the conversion to happen is Kevin.   Mostly what the rest of us will do is read the documentation and make sure it looks OK.  Obviously we all need to learn ReST *after* that conversion to write documentation, but there is so much documentation of ReST and sphinx available that I'm not really worried about that part.

More importantly, as JP already noted, the pyOpenSSL documentation is in a different format and the conversion would use a different toolchain, so even if we do have to get some practice, it's not a particularly helpful place to start.  If we *do* need practice for some reason, I think Nevow would be the best place to start, but then, I don't see why the activity level of the project makes any difference.

I have a few issues with the roadmap too, though:

"blah, blah": I'm not sure what that's supposed to mean.

Doh!  Forgot to go back and change that...pretend it sayssomething like: "(list of branches here)"

I'll fix it shortly.

"branches containing lore docs changes should be separated into two groups..." - no criteria are specified for deciding which go into which group.  Ideally we could just get all of those changes merged; if you are making progress on the conversion I'm sure we could organize a sprint to evaluate those changes and either abandon them or get them into review.

Well, I'm not entirely certain what criteria should be used until we at least know what branches exist with lore docs in them.
 
If it's only say 2 or 3 branches that are relatively easy to do, we just do all of them.  If it's a whole bunch, and some of them are those "everlasting branches" which have been around for years with no sign of being merged any time soon, well...that's group B. :)



Identifying tickets which propose lore functionality is pretty easy; just look at <http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reopened&component=lore&order=priority>.  Any tickets which have not been classified properly and therefore don't show up in that list can be closed later, as we discover them.  I strongly feel that we do *not* need to conduct an exhaustive review of the entire ticket tracker and get everything perfectly in order in order to do this conversion, as long as it's clear to everyone what is supposed to happen to lore-related tickets in the future.
 
Yeah, I don't expect this to be too difficult.  Basically thought I'd run a couple of searches (including the very one you mentioned above) and go from there.  I'd like to get most of them though.  One of the problems I think Twisted has is that there are just too dogone many open tickets.  It gets tricky to figure out what really needs to be done unless you've been doing it for years (as the core devs have) because of all the stuff getting in the way.  Which is why I've been trying to find outstanding tickets that are almost finished and "shepherd" them to completion.  But that's a discussion for another day/thread.
 
I see the biggest risk at the "docs freeze" step, that the doc conversion guy (or team, as the case may be by that point) will start work, then get distracted and walk away for 6 months, leaving a long period of time where nobody is supposed to write or edit documentation.

The whole point of our branch-based review process is to avoid this sort of situation.  We can't always avoid it (for example, the immense outstanding Conch branches that made everyone afraid to edit those warning-filled tests for years) but I think it's best to follow the same plan as for any branch, and have no formal "freeze" duration, just a point where the conversion branch gets merged to trunk.  It's OK if a few stale doc branches get left out in the cold during the conversion; if they're still stale once the conversion is ready, they must not be terribly actively worked on anyway.


To be honest, I'm not even sure such a freeze would be necessary.  I just thought I would throw it out there since the actual changeover would involve a rather large "structural" change to that part of the Twisted source tree.  And I didn't expect it to last more than a few days.  Maybe even hours.  Perhaps I'm just used to having this kind of step.  If the consensus is that it doesn't need to happen, I'm fine with that.

(FYI, I'm currently working in an environment where version control is ACTIVELY DISCOURAGED, so it makes me a little paranoid that some kind of disaster could happen.)
 
Even in the worst case, where a branch is left stale well after the docs have been deleted from trunk, presumably the author of the change could run lore2sphinx against the conversion, copy the sphinxified doc to their trunk working copy, and make a new branch.  Assuming that the output of the conversion tool is deterministic, the diff should be small and readable.

Hmmm.   This kind of procedure could sort of work.  Pondering....

The only potential problem I see with this idea is that I think we'll need to do some manual cleanups on the converted docs.  But I guess the author could do as you suggested, then do their own (presumably small) manual cleanup of their working copy after the conversion of their modified branch.  Should be workable, if not _quite_ as simple as you've laid it out (though almost as simple).
 
I'm not saying that we shouldn't identify those changes... It would be good to identify the branches with outstanding doc changes so that we would at least *know* how many changes will be broken, and perhaps motivate their authors to fix them ahead of time.
 
I think we're on the same page here.
 
I'm mainly interested in the "phase 0" outlined in the roadmap; I think that the stuff for phase 1 and 2 sounds good, but I don't think we need any special planning for it, since it fits into our normal development workflow pretty neatly.  People will be filing bugs for documentation typos pretty much forever ;-).  I'd like it if you could break down the "phase 0" a bit more clearly with regard to what happens when, since the review *before* the Big Switch gets thrown to put these changes in trunk is the most important part to get timely feedback from the community.

Looking at this after having been away from it for a day or so, I can see that it's a bit chaotic :)  I guess I had been looking at it too long. 

I'll try to update this soonish.  Anything specific that is unclear?

Kevin Horn