On Sun, Jun 5, 2016 at 6:33 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On 4 Jun 2016 6:54 am, "Donald Stufft" <donald@stufft.io> wrote:
>
>
>> On Jun 4, 2016, at 9:33 AM, Nathaniel Smith <njs@pobox.com> wrote:
>>
>> I think everyone would agree that having some nice doc hosting service available as an option would be, well, nice. Everyone likes options. But the current doc hosting is unpopular and feature poor, falls outside of the PyPI core mission, and is redundant with other more popular services, at a time when the PyPI developers are struggling to maintain core services.
>
>
>
> To add to what Nathaniel said here, there are a few problems with the current situation:
>
> Documentation hosting largely worked “OK” when it was just writing files out to disk, however we’ve since removed all use of the local disk (so that we can scale past 1 machine) and we’re now storing things in S3. This makes documentation hosting particularly expensive in terms of API calls because we need to do expensive list key operations to discover which files exist (versus package files where we have a database full of files).
Amazon do offer higher level alternatives like https://aws.amazon.com/efs/ for use cases like PyPI's docs hosting that assume they have access to a normal filesystem.
Given the credential management benefits of integrated docs,

From the RTD blog post linked by Nathaniel:
""
Our proposed grant, for $48,000, is to build a separate instance that integrates with the Python Package Index’s upcoming website, Warehouse. This integration will provide automatic API reference documentation upon package release, with authentication tied to PyPI and simple configuration inside the distribution.
""

it does seem worthwhile to me for the PSF to invest in a lowest common denominator static file hosting capability,

Seems like a very poor way to spend money and developer time imho. The original post by Jason brings up a few shortcomings of RTD, but I'm amazed that that leads multiple people here to conclude that starting a new doc hosting effort is the right answer to that. The much better alternative is: read the RTD contributing guide [1] and their plans for PyPI integration [2], then start helping out with adding those features to RTD.

RTD builds static HTML from Sphinx ReStructuredText and Commonmark Markdown

IDK what the best way to do eg epydoc API docs (static HTML hosting) is w/ RTD.

- cp into ./docs/_static/ before the RTD build?

for GitHub Pages, I like ghp-import (which pushes a directory and a .nojekyll file to a fresh commit in a gh-pages branch). BitBucket Pages is a bit different.

I wrote a tool called 'pgs' (as a bottle app) which will serve static HTML from a git-branch (with pa/th -> pa/th.html redirection, too) but haven't yet spent the time to add proper git bindings, so the per-file process overhead is n (after browser (and potentially WSGI) caching)

git branch static HTML support is not in scope for Warehouse because Warehouse doesnt do SCM.

(Because cookie origins)

One great feature of RTD, IMHO, is that you can host historical versions of the docs with stable versioned URLs like /en/v1.0.0/ (and a /en/latest/ redirect).

There is very little chance that a new effort as discussed here can come close to RTD, which is a quite active project with by now over 200 contributors. Starting a new project should be done for the right reasons: existing projects don't have and don't want to implement features you need, you have a better technical design, you want to reimplement to learn from it, etc. There are no such reasons here as far as I can tell.

If there's money left for packaging related work, I'm sure we can think of better ways to spend it. First thoughts:

- Accelerate PyPI integration plans for RTD

- add a link to the Docs: in README.rst (long_description)?

- Accelerate work on Warehouse

- Pay someone to review and merge distutils patches in the Python bug tracker

Final thought: there's nothing wrong with distributed infrastructure for projects. A typical project today may have code hosting on GitHub or Bitbucket, use multiple CI providers in parallel, use a separate code coverage service, upload releases to PyPI, conda-forge and GitHub Releases, and host docs on RTD. Integrating doc hosting with PyPI doesn't change that picture really.

I like to include anchor-text-free links to all of these project links in the README.rst (and thus long_description) as RST inline blocks:

| Wikipedia: `<https://()>`_

| Homepage: https://

( #PEP426 JSONLD / Metadata 2.0 could, ideally, include these links as structured data; this is a very simple solution which works w/ GitHub, BitBucket, PyPI, Warehouse, RTD, and anything else that'll render README.rst.

An RST DL definition list or just a list would be more semantically helpful than inline blocks, but less visual space efficient ... YMMV )

awesome-sphinxdoc is an outstanding resource for Sphinx (and thus RTD) things like themes: