[Distutils] on integrated docs in Warehouse and PyPI

[Wes Turner]

On Sunday, June 5, 2016, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sun, Jun 5, 2016 at 6:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>
>> On 4 Jun 2016 6:54 am, "Donald Stufft" <donald at stufft.io> wrote:
>> >
>> >
>> >> On Jun 4, 2016, at 9:33 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> I think everyone would agree that having some nice doc hosting service
>> available as an option would be, well, nice. Everyone likes options. But
>> the current doc hosting is unpopular and feature poor, falls outside of the
>> PyPI core mission, and is redundant with other more popular services, at a
>> time when the PyPI developers are struggling to maintain core services.
>> >
>> >
>> >
>> > To add to what Nathaniel said here, there are a few problems with the
>> current situation:
>> >
>> > Documentation hosting largely worked “OK” when it was just writing
>> files out to disk, however we’ve since removed all use of the local disk
>> (so that we can scale past 1 machine) and we’re now storing things in S3.
>> This makes documentation hosting particularly expensive in terms of API
>> calls because we need to do expensive list key operations to discover which
>> files exist (versus package files where we have a database full of files).
>>
>> Amazon do offer higher level alternatives like
>> https://aws.amazon.com/efs/ for use cases like PyPI's docs hosting that
>> assume they have access to a normal filesystem.
>>
>> Given the credential management benefits of integrated docs,
>>
> From the RTD blog post linked by Nathaniel:
> ""
> Our proposed grant, for $48,000, is to build a separate instance that
> integrates with the Python Package Index’s upcoming website, Warehouse
> <https://warehouse.python.org/>. This integration will provide automatic
> API reference documentation upon package release, with authentication tied
> to PyPI and simple configuration inside the distribution.
> ""
>
>> it does seem worthwhile to me for the PSF to invest in a lowest common
>> denominator static file hosting capability,
>>
> Seems like a very poor way to spend money and developer time imho. The
> original post by Jason brings up a few shortcomings of RTD, but I'm amazed
> that that leads multiple people here to conclude that starting a new doc
> hosting effort is the right answer to that. The much better alternative is:
> read the RTD contributing guide [1] and their plans for PyPI integration
> [2], then start helping out with adding those features to RTD.
>

+1

RTD builds static HTML from Sphinx ReStructuredText and Commonmark Markdown


IDK what the best way to do eg epydoc API docs (static HTML hosting) is w/
RTD.
- cp into ./docs/_static/ before the RTD build?

for GitHub Pages, I like ghp-import (which pushes a directory and a
.nojekyll file to a fresh commit in a gh-pages branch). BitBucket Pages is
a bit different.

https://pypi.python.org/pypi/ghp-import


I wrote a tool called 'pgs' (as a bottle app) which will serve static HTML
from a git-branch (with pa/th -> pa/th.html redirection, too) but haven't
yet spent the time to add proper git bindings, so the per-file process
overhead is n (after browser (and potentially WSGI) caching)


git branch static HTML support is not in scope for Warehouse because
Warehouse doesnt do SCM.


.. note::

   | project.readthedocs.org URLs are now
   | project.readthedocs.io

   (Because cookie origins)

   http://blog.readthedocs.com/securing-subdomains/


One great feature of RTD, IMHO, is that you can host historical versions of
the docs with stable versioned URLs like /en/v1.0.0/ (and a /en/latest/
redirect).


> There is very little chance that a new effort as discussed here can come
> close to RTD, which is a quite active project with by now over 200
> contributors. Starting a new project should be done for the right reasons:
> existing projects don't have and don't want to implement features you need,
> you have a better technical design, you want to reimplement to learn from
> it, etc. There are no such reasons here as far as I can tell.
>
> If there's money left for packaging related work, I'm sure we can think of
> better ways to spend it. First thoughts:
> - Accelerate PyPI integration plans for RTD
>

- add a link to the Docs: in README.rst (long_description)?


> - Accelerate work on Warehouse
>
- Pay someone to review and merge distutils patches in the Python bug
> tracker
>
>
> Final thought: there's nothing wrong with distributed infrastructure for
> projects. A typical project today may have code hosting on GitHub or
> Bitbucket, use multiple CI providers in parallel, use a separate code
> coverage service, upload releases to PyPI, conda-forge and GitHub Releases,
> and host docs on RTD. Integrating doc hosting with PyPI doesn't change that
> picture really.
>

+1


I like to include anchor-text-free links to all of these project links in
the README.rst (and thus long_description) as RST inline blocks:

[CI build badges]

name
============

[description]

| Wikipedia: `<https://()>`_
| Homepage: https://
| Docs:
| Src: git
| Src: hg
| Build:
| PyPI:
| Warehouse:
| Conda:
| Conda-forge:

( #PEP426 JSONLD / Metadata 2.0 could, ideally, include these links as
structured data; this is a very simple solution which works w/ GitHub,
BitBucket, PyPI, Warehouse, RTD, and anything else that'll render
README.rst.

An RST DL definition list or just a list would be more semantically helpful
than inline blocks, but less visual space efficient ... YMMV )

awesome-sphinxdoc is an outstanding resource for Sphinx (and thus RTD)
things like themes:
https://github.com/yoloseem/awesome-sphinxdoc/blob/master/README.rst#themes


> Ralf
>
> [1]
> https://github.com/rtfd/readthedocs.org/blob/master/docs/contribute.rst
> [2] https://github.com/rtfd/readthedocs.org/issues/1957
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160605/1d897bef/attachment-0001.html>