[pypy-dev] new (documentation) infrastructure

[holger krekel]

hello pypy,

if you just want to see some results of the infrastructure-hacks:

    http://codespeak.net/pypy/?doc

jum and me hacked the previous days on various bits of the (web)-
infrastructure for the pypy project.  One outcome is the spawn of a new
"vpath" project which encapsulates local filesystem pathes as well as
versioned subversion pathes.  It makes it very easy to e.g. save/load
objects to filesystem pathes (or even from the svn-repo) in a uniform
"expected" way. 

Here is a real-world Example for gathering all (previously generated)
"subversion info" and the ReST-generated html from a filesystem subtree.

    from vpath.local import Path
    from vpath.filters import endswidth, nodotdir

    docpath = Path('/projects/pypy/doc')
    for path in docpath.visit(endswith('.txt'), rfunc=nodotdir):
        infopath = path.newsuffix('.svninfo')
        info = infopath.load()
        info.htmlpath = path.newsuffix('.html')
        infolist.append(info)

The idea to put this "vpath" module into its own project was made
possible by a new svn-concept we discovered: you can stick a property
called "svn:externals" to a directory.  This property contains name-url
pairs. The 'name' is put into the namespace of the directory and the
bound object is taken from "importing" what the 'url' points to.  
Thus if you want to use the "vpath" module in your project you just do

  svn propset "svn:externals" \
              "vpath http://codespeak.net/svn/vpath/trunk/dist" .

then subsequently on "svn up" you'll get the trunk-revision of
'vpath' - externally fetched (it needn't be in the same repo).  If you
modify the vpath-module (because you have commit rights) the others
working on it get a diff.  This makes it extremely easy to couple
projects in a version-controlled way. 

Using this and some other fresh infrastructure we now have a 
"doc" item in the menu-bar of the pypy-site
        
    http://codespeak.net/pypy/?doc

which should make it easy to browse documentation and spot which
documents are outdated etc.  The web-page reflects the "trunk" 
version of our "doc"-directory.  If you checkin new ReST-docs 
then on the server-side a script will generate all information
which is subsequently used from the cgi to display it dynamically.

Now i have a deal to offer.  If someone writes a "plugin" (or whatever)
for ReST that accepts parameters and is allowed to return some html
fragment which is inserted in-place (where the plugin is invoked) then
i'll do a syntax-highlighted cross-referenced "source-view" of the pypy
source, including presenting "document-links" to a source-file 
(which are configured using some svn-property like "pypy:doc").

Both projects are probably a bit of work but it would certainly help 
to increase the "cross-referencedness" of documentation and source 
code (including inlining snippets from a source-file to have 
up-to-date examples). 

Please bear with me if i messed up the web-site at some places.  The
messiest part definitely was to get CSS-related rendering "right". 

so much for now, cheers,

    holger