-----BEGIN PGP SIGNED MESSAGE----- Hi hackers and others, first of all I'd like to apologize if this is off-topic on this list. If so it would be nice if someone could tell me, where this belongs to. I don't know whether you already know, but reStructuredText is mentioned on http://www.ibm.com/developerworks/library/x-matters24/ I saw it there, read the specification and some more - and I'm absolutely delighted :-) !! First, I'd like to tell you a bit about my background. I love the Perl idea of embedding documentation in the source code using POD. To me it became a habit documenting everything while or often even before coding something. I guess that was, what Larry had in mind when inventing POD :-) . Another nice aspect of POD is, that it's ASCII based and thus it is useful to use CVS on it and/or the miracles of the Emacs' ediff package. And of course you do not need any special editor to work with this format. Instead every editor supporting plain ASCII works. And nonetheless you can produce lots of output formats. By a simple `make' call if you like. All in all POD allowed for a lot of nice features with a high degree if automatization, versioning and so on. So I started to write all my documents in POD. However, POD is a rather limited markup format. If you want to write something different from a manual page you'll immediately run into a number of limitations. So I looked for an alternative. I found SDF [http://search.cpan.org/author/IANC/sdf-2.001/, the original site doesn't exist any more but it is embedded in the package]. SDF is a rather great tool very much in the spirit of reStructuredText. One feature I particularly like is, that it is possible to embed (Perl) code in your text document to do some very specific job. Anyway it is easy to combine SDF with Perl to create really great stuff. The most elaborated thing I did with SDF were some filters which take a high level interface description for an API and produce include files for C, C++, Sicstus Prolog, and Java. Well, since I discovered SDF I'm using SDF for nearly everything. In particular the web sites I'm maintaining are all created by SDF and `make'. However, SDF has a number of drawbacks: * The code is based on Perl 4 (before references were invented in Perl) and it's - ummm - somewhat ugly. I thought about rewriting it from scratch but I've got no time for such a task. * There is a great number of nice ideas but there is not much of an overall concept. * No community developed - at least no community visible to me. This may be partly because the maintainer Ian Clatworthy (thanks Ian for all of your efforts) seemingly gave up the project shortly after I discovered it (in 1999). (I never contacted him, so I could not have been the reason ;-) .) * It did not reach a wide distribution and thus virtually nobody is used to using it. So though personally I'm rather happy with SDF I don't like the thought of being trapped in a evolutionary dead end. And now I found reStructuredText. I read the specification and though today reStructuredText misses some things of what SDF has, I see that there is a potential for reStructuredText for being a very powerful tool - much more than SDF ever was. Particularly because there seems to be an active maintainer (thanks David Goodger and all for the *great* web site, I particularly like that the discussion of alternatives is listed) and seemingly a very interested community. Personally I think it would be a great gift to the world to establish a text format which is easy to write but yet powerful. In short: I fully support all the goals listed for reStructuredText. I think over time I'll reformat all my sources to use reStructuredText. Ahmm - as you may have noted I'm coming from the Perl community and I don't know Python yet and at the moment I'm not planning to change that. But when talking about a format this is of little importance of course. Well, now you know why I'm here :-) . Now for the question part of this mail. Indeed I have a number of questions about reStructuredText and it's implementation. I'm sorry if any of these questions is already answered on one some web page. I did not read all of the pages yet. * Is there a Perl implementation already? Does anybody plan such a beast? Admittedly I thought of doing it - but my time is *very* limited :-( . However, that's a itch I'd like to scratch. * Is there a Emacs mode already? Particularly it there `font-lock' support for reStructuredText documents available? Personally I got used so much to font-locking I find it an unbearable pain to read text containing formal syntax without appropriate coloring. If no such thing is available I could give it a try. * Are there transformation tools from OpenOffice format to reStructuredText? This question may need some explanation. I have the problem, that sometimes I receive M$ `.doc's and `.ppt's or `.rtf's I'd like to transform to SDF to embed them in a web site. Before OpenOffice and it's XML based document format this was a difficult task needing a lot of unautomatable work. I wrote a tool for RTF but Powerpoint was beyond automatization. For SDF lately I created a suite of XSLT style sheets which transform the `styles.xml' and `content.xml' contained in `.sxw' and `.sxc' to SDF. Though this involves some post processing by hand it is *much* easier than anything I did before. In effect by (ab)using the input filters of OpenOffice I now have a whole lot of formats I can transform to SDF with reasonable effort. Of course I'd like to see this for reStructuredText as well. If no-one did this until know, I could rewrite my XSLT scripts a bit to support reStructuredText. * Is there a transformation for Docutils Document Tree documents in XML to a reStructuredText source form? Or following PEP 258: Is there a writer for reStructuredText? If so the Docutils Document Tree could be used as an XML based representation and all the nice things of XML are available. For instance one could write an XSLT transformation from an OpenOffice format to Docutils Document Tree format and transformations in all thinkable directions are possible. * Are there plans to support embedded code in reStructuredText documents? The embedded code needs access to the embedding document of course. May be this is similar to JavaScript embedded in HTML pages. Of course I'd prefer support for Perl ;-) . * Are there writers for plain text and/or manual pages? Finally I'd like to draw your attention to http://template-toolkit.org/ It's a very cool tool which allows for a wide range of automation. It virtually adds a powerful macro language to every ASCII based format. One thing I particularly liked with SDF is the feature to embed SDF or other stuff in anything which supports comments. This way you were able to embed for instance POD documentation in a C++ file. One of the nice features is, that it rearranges the embedded code allowing you to write your documentation at the place of the definition and nonetheless get every piece of documentation at the right place in the final document. However, as SDF is usually not available this feature is of limited use if you're going to share your source with others :-( . In the free software project I started lately [http://savannah.nongnu.org/projects/mamo/] I created a plug-in for the Template Toolkit which exactly does this sort of gathering of snippets of documentation from an arbitrary source file [http://savannah.nongnu.org/cgi-bin/viewcvs/mamo/mamo/lib/perl5/Mamo/Template...]. Thinking reStructuredText and Docutils it might qualify as the input part of a reader. Mit Freien Grüßen Stefan -----BEGIN PGP SIGNATURE----- Version: 2.6.3in Charset: noconv Comment: Processed by Mailcrypt 3.5.7, an Emacs/PGP interface iQCVAwUBPlqjdAnTZgC3zSk5AQGjVgQArL3NYg0OThGf7Xi++LJPUgKnS0HyRfT6 bek6sjLiYOSfDLQPFAfWjeVhRKaMt4P+nKhIqVPuT/Jfe3nj/LgxTOMdWspjE7gU CT6VA2qKNpuAAvoIc9BoW9nL4bQE3fgnETENtdZROlg4kiQLwGKzTr1VWs+uLvqt nSzPi3EddpI= =alwL -----END PGP SIGNATURE-----
Stefan Merten wrote:
first of all I'd like to apologize if this is off-topic on this list.
Not really off-topic here, but there are two Docutils-specific lists also: <http://lists.sourceforge.net/lists/listinfo/docutils-users> for usage questions, and <http://lists.sourceforge.net/lists/listinfo/docutils-develop> for developers. Recent archives are also available at <http://gmane.org>. This list is more general and it has a greater readership.
I don't know whether you already know, but reStructuredText is mentioned on
Yes, it's a good article. A few details could be corrected, but nothing major.
I saw it there, read the specification and some more - and I'm absolutely delighted :-) !!
Glad you like it.
* Is there a Perl implementation already? Does anybody plan such a beast?
Mark Nodine <nodine@somerset.sps.mot.com> is working on a Perl implementation of the reStructuredText parser. He hasn't released any code yet, though.
* Is there a Emacs mode already?
Not yet. There are a few functions in tools/editors/emacs/restructuredtext.el, available from CVS or the snapshot (<http://docutils.sf.net/docutils-snapshot.tgz>).
Particularly it there `font-lock' support for reStructuredText documents available?
No. It's an interesting idea, although I'm not sure how it would work.
If no such thing is available I could give it a try.
Please do!
* Are there transformation tools from OpenOffice format to reStructuredText?
There is work being done on OpenOffice.org XML *writers*, but nothing to my knowledge the other way.
For SDF lately I created a suite of XSLT style sheets which transform the `styles.xml' and `content.xml' contained in `.sxw' and `.sxc' to SDF. Though this involves some post processing by hand it is *much* easier than anything I did before.
Sounds like a good approach. I've heard of people setting up semi-automated systems to assist with conversion.
Of course I'd like to see this for reStructuredText as well. If no-one did this until know, I could rewrite my XSLT scripts a bit to support reStructuredText.
Such code would be welcome.
* Is there a transformation for Docutils Document Tree documents in XML to a reStructuredText source form? Or following PEP 258: Is there a writer for reStructuredText?
There has been some discussion about a generic to-reStructuredText conversion system, perhaps a Docutils "Writer", but no implementation yet.
* Are there plans to support embedded code in reStructuredText documents?
I believe Mark Nodine's Perl implementation has it. There are notes about at <http://docutils.sf.net/spec/notes.html#misc-exec>. Such a feature is inherently dangerous though, and there have to be safeguards in place to prevent reStructuredText/Docutils being used as a vector for viruses & trojans.
* Are there writers for plain text and/or manual pages?
Not that I know of. I recall some discussion, but nothing concrete.
Finally I'd like to draw your attention to
Looks useful, but templating systems are plentiful.
One thing I particularly liked with SDF is the feature to embed SDF or other stuff in anything which supports comments. This way you were able to embed for instance POD documentation in a C++ file. One of the nice features is, that it rearranges the embedded code allowing you to write your documentation at the place of the definition and nonetheless get every piece of documentation at the right place in the final document.
Sounds like "literate programming". Do you have a link to a description of how SDF accomplishes this rearranging?
In the free software project I started lately [http://savannah.nongnu.org/projects/mamo/] I created a plug-in for the Template Toolkit which exactly does this sort of gathering of snippets of documentation from an arbitrary source file ... Thinking reStructuredText and Docutils it might qualify as the input part of a reader.
I don't understand the implications. Can you describe this for us? -- David Goodger http://starship.python.net/~goodger Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv
-----BEGIN PGP SIGNED MESSAGE----- Hi David and all! Sorry for the delay but lately I have been too busy with other things to code :-( . Last month (28 days ago) David Goodger wrote:
Stefan Merten wrote:
* Is there a Perl implementation already? Does anybody plan such a beast?
Mark Nodine <nodine@somerset.sps.mot.com> is working on a Perl implementation of the reStructuredText parser. He hasn't released any code yet, though.
Ok, so it's probably better to wait for some code from Mark while not holding breath ;-) .
* Is there a Emacs mode already?
Not yet. There are a few functions in tools/editors/emacs/restructuredtext.el, available from CVS or the snapshot (<http://docutils.sf.net/docutils-snapshot.tgz>).
Yes. This could and should be integrated with my work then.
Particularly it there `font-lock' support for reStructuredText documents available?
No. It's an interesting idea, although I'm not sure how it would work.
I coded a font-lock mode which recognizes most reST constructs. I checked it with http://docutils.sourceforge.net/spec/rst/reStructuredText.txt and it worked pretty good. At the moment blocks are not handled. For the most part this is no problem, but comments, section headers, and literal blocks should really be font-locked correctly. Also definition lists are not font-locked yet. Apart from that it is not clear whether it makes sense to font-lock tables. Everything else should work fine. Everything missing is marked in the code by ``FIXME``. I already played around a bit with font-locking for comment blocks, but this is a bit difficult.
If no such thing is available I could give it a try.
Please do!
:-) You can download it from http://www.merten-home.de/FreeSoftware/rest-mode/index.html I tried it with Emacs 20 and Emacs 21. XEmacs 21 did not like everything as it seems :-( . But for sure it's a start. If you want to include it in an official CVS tree that's fine. However, please contact me so I can continue to maintain it or we can look for other arrangements. Please note that in the file I'm suggesting file name extensions ``.rest`` and ``.reST`` for reStructuredText documents. I don't know whether there is a widely accepted extension already. However, I think there should be one. Also take care about the mode settings at the end of for instance http://docutils.sourceforge.net/spec/rst/reStructuredText.txt This conflicts with automated settings depending on the file name.
* Are there transformation tools from OpenOffice format to reStructuredText?
There is work being done on OpenOffice.org XML *writers*, but nothing to my knowledge the other way.
For SDF lately I created a suite of XSLT style sheets which transform the `styles.xml' and `content.xml' contained in `.sxw' and `.sxc' to SDF. Though this involves some post processing by hand it is *much* easier than anything I did before.
Sounds like a good approach. I've heard of people setting up semi-automated systems to assist with conversion.
Of course I'd like to see this for reStructuredText as well. If no-one did this until know, I could rewrite my XSLT scripts a bit to support reStructuredText.
Such code would be welcome.
:-) Now font-locking turned the light on ;-) I'll work on that.
* Is there a transformation for Docutils Document Tree documents in XML to a reStructuredText source form? Or following PEP 258: Is there a writer for reStructuredText?
There has been some discussion about a generic to-reStructuredText conversion system, perhaps a Docutils "Writer", but no implementation yet.
I'd find it useful because then one could use the XML format as a general intermediate format.
* Are there plans to support embedded code in reStructuredText documents?
I believe Mark Nodine's Perl implementation has it.
Perl people seem to like that ;-) .
There are notes about at <http://docutils.sf.net/spec/notes.html#misc-exec>. Such a feature is inherently dangerous though, and there have to be safeguards in place to prevent reStructuredText/Docutils being used as a vector for viruses & trojans.
Yes, security will be an issue then. May be it could help to have any code execution switched off by default and code is only executed when this is switched on explicitly.
* Are there writers for plain text and/or manual pages?
Not that I know of. I recall some discussion, but nothing concrete.
I'd find that great - particularly for manual pages. Hmm... While thinking about it: It would suffice to have a writer for POD. This way you'll indirectly have a writer for manual pages for free.
Finally I'd like to draw your attention to
Looks useful, but templating systems are plentiful.
One thing I particularly liked with SDF is the feature to embed SDF or other stuff in anything which supports comments. This way you were able to embed for instance POD documentation in a C++ file. One of the nice features is, that it rearranges the embedded code allowing you to write your documentation at the place of the definition and nonetheless get every piece of documentation at the right place in the final document.
Sounds like "literate programming". Do you have a link to a description of how SDF accomplishes this rearranging?
This was solved fairly easy. The extractor (``sdfget``) used a table like this:: Key Format NAME "=head1 $key\n\n$data\n\n" SYNOPSIS "=head1 $key\n\n$data\n\n" DESCRIPTION "=head1 $key\n\n$data\n\n" OPTIONS "=head1 $key\n\n$data\n\n" FEATURES "=head1 $key\n\n$data\n\n" EXAMPLE "=head1 $key\n\n$data\n\n" EXAMPLES "=head1 $key\n\n$data\n\n" RETURN VALUES "=head1 $key\n\n$data\n\n" EXIT STATUS "=head1 $key\n\n$data\n\n" DIAGNOSTICS "=head1 $key\n\n$data\n\n" ENVIRONMENT "=head1 $key\n\n=over 4\n\n$data\n\n=back\n\n" FILES "=head1 $key\n\n=over 4\n\n$data\n\n=back\n\n" PREREQUISITES "=head1 $key\n\n$data\n\n" SEE ALSO "=head1 $key\n\n$data\n\n" AUTHOR "=head1 $key\n\n$data\n\n" LICENSE "=head1 $key\n\n$data\n\n" AVAILABILITY "=head1 $key\n\n$data\n\n" NOTES "=head1 $key\n\n$data\n\n" BUGS "=head1 $key\n\n$data\n\n" The table is given in an extra report file given as an argument to ``sdfget``. The ``Key`` column gives some keys to be recognized in your source code while the ``Format`` column gave the format to be used as a Perl string. ``$key`` was the ``Key`` from the first column while ``$data`` was the data contained in *all* sections tagged with the respective key. As you may have recognized the above table transforms to POD snippets forming a manual page. Then in the source you can write things like that:: #>>NAME:: # # rest-mode - An Emacs mode for reStructuredText documents #>>AUTHOR:: Stefan Merten <smerten@oekonux.de> #>>SYNOPSIS:: # # B<someprogram> I<option>... [Some Perl code here] #>>FILES:: # # =item F<somefile> # # F<somefile> is useful for... [More Perl code here] #>>ENVIRONMENT:: # # =item I<TMPDIR> # # This is the standard directory where temporary files are created. It is # overwritten by the option B<--tmpdir>. [More Perl code here] #>>FILES:: # # =item F<anotherfile> # # F<anotherfile> can be used to... As you may have recognized the triggering markup is::
key::
and it can be ended by a non-comment line (though there is explicit end markup, too). I.e. you can distribute documentation snippets over the whole source code - i.e. where things are used in the code. All snippets are concatenated to a single ``$data``. The sequence of sections is given by the key sequence given in the table. This results (in this case) in a clean POD file which can then be processed further.
In the free software project I started lately [http://savannah.nongnu.org/projects/mamo/] I created a plug-in for the Template Toolkit which exactly does this sort of gathering of snippets of documentation from an arbitrary source file
This works similar to what I showed above - however without a table but with a template file which determines the resulting structure.
Thinking reStructuredText and Docutils it might qualify as the input part of a reader.
I don't understand the implications. Can you describe this for us?
There seems to be a general trend to integrate documentation in the source code. This applies to any language and I think this is a *very* good thing. However, there is the problem, that the structure of a program rarely is the best structure for its documentation. But if you have some sort of gathering mechanism like the one described above you can scatter documentation over the source(s) and collect it later. Then all you need is a program for that. Though this might be a useful feature in general of course it is for people who want to use reStructuredText for their program documentation. The approach using ``sdfget`` was one option. However, SDF is dead. The approach using the Template Toolkit and the ``Gather`` plugin mentioned above would be a second option. However, this would need the Template Toolkit in addition to the Docutils code to work with reStructuredText. A third option would be to have such a functionality in the reStructuredText / Docutils context. That is what I'm trying to suggest. Mit Freien Grüßen Stefan -----BEGIN PGP SIGNATURE----- Version: 2.6.3in Charset: noconv Comment: Processed by Mailcrypt 3.5.7, an Emacs/PGP interface iQCVAwUBPoNZqAnTZgC3zSk5AQF1YgQAo7FHkd30RKxS4u1qa2NQXiguO/n9smuj H8G7JH7MNnqgKKY702aqeFg5OeNP0gLa0VPOASutAXxBbVDda/rUmPUF5WrlI+ER YRhI+E2ZR8Y1QoUiE/9bUcl9+u4pddMuj+Dl/7YZzgw0uyhpkm66U43YkZ/K+Oow jpgA3k8BL7M= =kgrP -----END PGP SIGNATURE-----
thanks for this great works ! I use it imediatly ! -- William Dodé - http://flibuste.net
[Stefan Merten]
* Is there a Emacs mode already?
[David Goodger]
Not yet. There are a few functions in tools/editors/emacs/restructuredtext.el, available from CVS or the snapshot (<http://docutils.sf.net/docutils-snapshot.tgz>).
[Stefan Merten]
Yes. This could and should be integrated with my work then.
Indeed. I think basing a custom mode on indented-text-mode would be a good place to begin. Although I hack elisp functions, I've never tackled a mode.
I coded a font-lock mode which recognizes most reST constructs.
It looks quite good! I hope you continue to develop it. Could it be made a derivative of indented-text-mode, so that it inherits that mode's behavior? Or is it already, except for keymaps? (How does one tell?) Or could this become a minor mode of indented-text-mode?
If you want to include it in an official CVS tree that's fine.
Thank you. I think I'll include it as tools/editors/emacs/rst-mode.el ("rst" not "rest", see below).
Please note that in the file I'm suggesting file name extensions ``.rest`` and ``.reST`` for reStructuredText documents. I don't know whether there is a widely accepted extension already. However, I think there should be one.
See the discussion at <http://docutils.sf.net/FAQ.html>, questions 2.3 & 2.4. I think .rst would be preferable to .rest, for length and to distinguish it from the other REST. Personally, I'll never use a .rst or .rest extension on files, only .txt, so the auto-mode-alist will not be useful to me. I may just set my default major mode to rst-mode in my .emacs file; it's now set to indented-text-mode.
* Are there writers for plain text and/or manual pages? ... Hmm... While thinking about it: It would suffice to have a writer for POD. This way you'll indirectly have a writer for manual pages for free.
The same is true of DocBook, since it can generate man pages. But either one means that an external tool is needed.
Finally I'd like to draw your attention to
http://template-toolkit.org/ [... and SDF ...] Sounds like "literate programming". Do you have a link to a description of how SDF accomplishes this rearranging?
This was solved fairly easy. The extractor (``sdfget``) used a table like this:: ...
Thanks for the explanation. Something similar to SDF or the template toolkit might be feasible with reStructuredText directives in a template file. It's worth exploring, and other literate programming systems may have ideas we can use. I'd like to see any resulting system be as simple and intuitive as possible, and be able to use all information already marked up explicitly or identified implicitly in the source (such as bibliographic fields such as "author"). -- David Goodger http://starship.python.net/~goodger Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv
-----BEGIN PGP SIGNED MESSAGE----- Hi David and all! 5 days ago David Goodger wrote:
[Stefan Merten]
I coded a font-lock mode which recognizes most reST constructs.
It looks quite good!
:-) I'm glad it's helping people :-) .
I hope you continue to develop it.
I already did. It now supports comments and literal blocks. However, this involves multi-line font-locking which is slightly beyond simple font-lock. Particularly typing gets **slooooow** when point is after such a block. Because of that the mode switches to `lazy-lock-mode' which one might configure independently. There is a customizable variable `rst-mode-lazy' which triggers this behavior so this can be switched off if it is more disturbing than useful.
Could it be made a derivative of indented-text-mode, so that it inherits that mode's behavior? Or is it already, except for keymaps? (How does one tell?) Or could this become a minor mode of indented-text-mode?
<disclaimer> I'm actually nor an Emacs lisp hacker. I learned some Emacs lisp a long time ago and added to that over time but I did not follow the modern developments such as customization. </disclaimer> In modern Emacsen I once saw a technique to derive a new mode from an old one. However, this seems not be supported everywhere. I'll check out how ``rst-mode`` can be integrated with ``indented-text-mode`` (which at least in 20.7 is just an alias for ``text-mode`` anyway).
If you want to include it in an official CVS tree that's fine.
Thank you. I think I'll include it as tools/editors/emacs/rst-mode.el
Meanwhile you added me to the developers. Thanks :-) . I just added:: docutils/tools/editors/emacs/rst-mode.el to the CVS repository. You may download it from:: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/docutils/docutils/tools/editors/emacs/rst-mode.el?rev=HEAD&content-type=text/plain
("rst" not "rest", see below).
I changed all the names to "rst" instead of "rest". I just needed a name to start with. Mit Freien Grüßen Stefan -----BEGIN PGP SIGNATURE----- Version: 2.6.3in Charset: noconv Comment: Processed by Mailcrypt 3.5.7, an Emacs/PGP interface iQCVAwUBPos+SgnTZgC3zSk5AQHy3QQAqgyAcANHpAdpcAdegJC4wpKQbWf9N7+N HEHjuKPntw0trDv0PWx6sVrtxcEs/y8Kaf/+Z7/a2/AeSAp5bvk25hKxwcQI2jIM 2CQaPQqH19dkYwsVv59sldnBTMSmiDdpDnBEAsbSFqOwybHW9foAIUQckvJ97J1O 7npyJh7EoF8= =fH1D -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hi all! I forgot this. 4 minutes ago Stefan Merten wrote:
I just added::
docutils/tools/editors/emacs/rst-mode.el
to the CVS repository. You may download it from::
I'm keen to know of any problems with that (besides the ones noted in the source already). Feel free to write me private mail. However, please don't expect to get a reply immediately. Mit Freien Grüßen Stefan -----BEGIN PGP SIGNATURE----- Version: 2.6.3in Charset: noconv Comment: Processed by Mailcrypt 3.5.7, an Emacs/PGP interface iQCVAwUBPos/1wnTZgC3zSk5AQGB9gQAkPTnJG4aIerRwgsF5DVvBFggHJNz2NvV Lh26JSYUZdQ4QR6f+L5xYijMzp1And+f3KSDr0QTijWUuzBBxoX58JCqTjYjHxOy 144/HNIekHAzbF3CQVJLsgc0kHATPixhicEMqSOnBkbyeRUv2EsKc7OQANgUI7D+ fT3p1FsNrNA= =ObKA -----END PGP SIGNATURE-----
participants (3)
-
David Goodger -
Stefan Merten -
William Dode