Voting on possible subdomains for the devguide
With part of the goal of moving to GitHub being to minimize how much infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4).
Thanks to the work of various people we have https://cpython-devguide.readthedocs.io/ so the toolchain for building the devguide on RTFD is done. That means the next step is deciding on a subdomain on the python.org domain to host the docs from. If you have an opinion, go to https://github.com/python/core-workflow/issues/4 and leave a reaction on the appropriate message.
With Fastly we could probably send www.python.org/peps/ http://www.python.org/peps/ to RTD or docs.python.org/devguide/ http://docs.python.org/devguide/ to the devuide. I think RTD supports setting a canonical URL that includes a path too. FWIW. Of course the look and feel might be different so different domains might be a good idea anyways.
On Apr 11, 2017, at 12:50 PM, Brett Cannon brett@python.org wrote:
With part of the goal of moving to GitHub being to minimize how much infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4 https://github.com/python/peps/issues/4).
Thanks to the work of various people we have https://cpython-devguide.readthedocs.io/ https://cpython-devguide.readthedocs.io/ so the toolchain for building the devguide on RTFD is done. That means the next step is deciding on a subdomain on the python.org http://python.org/ domain to host the docs from. If you have an opinion, go to https://github.com/python/core-workflow/issues/4 https://github.com/python/core-workflow/issues/4 and leave a reaction on the appropriate message. _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
— Donald Stufft
On Tue, 11 Apr 2017 at 10:01 Donald Stufft donald@stufft.io wrote:
With Fastly we could probably send www.python.org/peps/ to RTD or docs.python.org/devguide/ to the devuide. I think RTD supports setting a canonical URL that includes a path too. FWIW. Of course the look and feel might be different so different domains might be a good idea anyways.
That was my thinking. The devguide and peps are distinct enough in how their purposes that sticking them on their own domain that's easy to remember I think makes sense.
On Apr 11, 2017, at 12:50 PM, Brett Cannon brett@python.org wrote:
With part of the goal of moving to GitHub being to minimize how much infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4).
Thanks to the work of various people we have https://cpython-devguide.readthedocs.io/ so the toolchain for building the devguide on RTFD is done. That means the next step is deciding on a subdomain on the python.org domain to host the docs from. If you have an opinion, go to https://github.com/python/core-workflow/issues/4 and leave a reaction on the appropriate message.
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
—
Donald Stufft
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
[1] https://magicstack.github.io/asyncpg/current/
Elvis
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
On Tue, Apr 11, 2017 at 05:58:38PM +0000, Brett Cannon brett@python.org wrote:
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
GH Pages don't need much to setup. You have to choose between Org Pages (at the root of the subdomain) and Repo Pages (in subdirectories). I even believe GH Pages can do both on the same subdomain but I've never tried.
Oleg.
On Tuesday, April 11, 2017 2:09:33 PM EDT Oleg Broytman wrote:
On Tue, Apr 11, 2017 at 05:58:38PM +0000, Brett Cannon brett@python.org wrote:
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
GH Pages don't need much to setup. You have to choose between Org Pages (at the root of the subdomain) and Repo Pages (in subdirectories). I even believe GH Pages can do both on the same subdomain but I've never tried.
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch. If there's interest in pursuing this option, I can help with setting things up.
[1] https://help.github.com/articles/custom-domain-redirects-for-github-pages-si...
Elvis
On Tue, Apr 11, 2017 at 02:28:20PM -0400, Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 2:09:33 PM EDT Oleg Broytman wrote:
On Tue, Apr 11, 2017 at 05:58:38PM +0000, Brett Cannon brett@python.org wrote:
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
GH Pages don't need much to setup. You have to choose between Org Pages (at the root of the subdomain) and Repo Pages (in subdirectories). I even believe GH Pages can do both on the same subdomain but I've never tried.
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch. If there's interest in pursuing this option, I can help with setting things up.
[1] https://help.github.com/articles/custom-domain-redirects-for-github-pages-si...
For Org Pages with a custom domain you also need a CNAME file at the root of the repo.
To publish Sphinx-generated sites you need an empty .nojekyll file at the root. Without it GitHub blocks special Sphinx directories (those ones that starts with an underscore).
Elvis
Oleg.
On Tue, 11 Apr 2017 at 11:28 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 2:09:33 PM EDT Oleg Broytman wrote:
On Tue, Apr 11, 2017 at 05:58:38PM +0000, Brett Cannon brett@python.org
wrote:
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com
wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
GH Pages don't need much to setup. You have to choose between Org Pages (at the root of the subdomain) and Repo Pages (in subdirectories). I even believe GH Pages can do both on the same subdomain but I've never tried.
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch.
So we would have to either set up another repo or commit the rendered files to the repo?
If there's interest in pursuing this option, I can help with setting things up.
I assume we would lose the widget Read the Docs provides which is handy for getting to the GitHub page for editing the content? I guess we lose anything else that https://docs.readthedocs.io/en/latest/features.html lists.
I'm just trying to think of what we gain/lose if we switch to GH Pages over RTFD as I don't want to end up with three different documentation hosting solutions sort of like we have now where we build the devguide, PEPs, and the docs themselves in special ways that all happen to be hosted on PSF infrastructure (e.g. getting Pygments installed for the PEPs has stalled out and it would be good that any solution we choose be flexible enough to handle this kind of situation: https://github.com/python/peps/pull/229/files/70c6c1d010c1d50a375430c4ba6ae3... ).
On Tuesday, April 11, 2017 3:11:10 PM EDT Brett Cannon wrote:
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch.
So we would have to either set up another repo or commit the rendered files to the repo?
Either a separate repo, or commit to the gh-pages branch.
If there's
interest in pursuing this option, I can help with setting things up.
I assume we would lose the widget Read the Docs provides which is handy for getting to the GitHub page for editing the content? I guess we lose anything else that https://docs.readthedocs.io/en/latest/features.html lists.
https://gist.github.com/mgedmin/6052926
I'm just trying to think of what we gain/lose if we switch to GH Pages over RTFD as I don't want to end up with three different documentation hosting solutions sort of like we have now where we build the devguide, PEPs, and the docs themselves in special ways that all happen to be hosted on PSF infrastructure (e.g. getting Pygments installed for the PEPs has stalled out and it would be good that any solution we choose be flexible enough to handle this kind of situation: https://github.com/python/peps/pull/229/files/70c6c1d010c1d50a375430c4 ba6ae3e4f439306c#r107471836 ).
The idea is that the doc build is done as part of CI and the product is uploaded as static content (GH pages is an option, but it might be anything capable of serving static content).
One of the reasons why we went with GH Pages for asyncpg is that at the time RTFD did not support Python 3.5 syntax and we wanted to use autodoc. Could this be an issue when building Python documentation? Can the master branch Docs/ be built by Sphinx running on Python 3.4?
Elvis
On Tue, 11 Apr 2017 at 12:29 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 3:11:10 PM EDT Brett Cannon wrote:
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch.
So we would have to either set up another repo or commit the rendered files to the repo?
Either a separate repo, or commit to the gh-pages branch.
If there's
interest in pursuing this option, I can help with setting things up.
I assume we would lose the widget Read the Docs provides which is handy for getting to the GitHub page for editing the content? I guess we lose anything else that https://docs.readthedocs.io/en/latest/features.html lists.
https://gist.github.com/mgedmin/6052926
I'm just trying to think of what we gain/lose if we switch to GH Pages over RTFD as I don't want to end up with three different documentation hosting solutions sort of like we have now where we build the devguide, PEPs, and the docs themselves in special ways that all happen to be hosted on PSF infrastructure (e.g. getting Pygments installed for the PEPs has stalled out and it would be good that any solution we choose be flexible enough to handle this kind of situation: https://github.com/python/peps/pull/229/files/70c6c1d010c1d50a375430c4 ba6ae3e4f439306c#r107471836 ).
The idea is that the doc build is done as part of CI and the product is uploaded as static content (GH pages is an option, but it might be anything capable of serving static content).
One place where this might actually show itself to be useful is with the PEPs. That build toolchain generates HTML files directly and doesn't even use Sphinx (at least not yet). I have opened https://github.com/python/peps/issues/241 to discuss the idea.
One of the reasons why we went with GH Pages for asyncpg is that at the time RTFD did not support Python 3.5 syntax and we wanted to use autodoc. Could this be an issue when building Python documentation? Can the master branch Docs/ be built by Sphinx running on Python 3.4?
There's no version requirement for the docs as we don't use the autodoc extension. And I think Read the Docs uses a newer image now as I was buliding the docs for gidgethub using autodoc and that uses 3.5 syntax.
And how would versions be handles for GH Pages? E.g. how would docs.python.org handle viewing the e.g. 2.7 data? I guess you're just really careful to upload to subdirectories in the gh-pages branch?
On Tuesday, April 11, 2017 4:11:44 PM EDT Brett Cannon wrote:
One place where this might actually show itself to be useful is with the PEPs. That build toolchain generates HTML files directly and doesn't even use Sphinx (at least not yet). I have opened https://github.com/python/peps/issues/241 to discuss the idea.
Sounds like a good place to try the process.
One of the reasons why we went with GH Pages for asyncpg is that at the time RTFD did not support Python 3.5 syntax and we wanted to use autodoc. Could this be an issue when building Python documentation? Can the master branch Docs/ be built by Sphinx running on Python 3.4?
There's no version requirement for the docs as we don't use the autodoc extension. And I think Read the Docs uses a newer image now as I was buliding the docs for gidgethub using autodoc and that uses 3.5 syntax.
And how would versions be handles for GH Pages? E.g. how would docs.python.org handle viewing the e.g. 2.7 data? I guess you're just really careful to upload to subdirectories in the gh-pages branch?
Yes, we can continue using the current switcher and subfolders.
Elvis
On Wed, Apr 12, 2017 at 5:11 AM, Brett Cannon brett@python.org wrote:
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch.
So we would have to either set up another repo or commit the rendered files to the repo?
One neat trick is to have a deployment script that commits the rendered files only to the gh-pages branch. I've done that for other purposes with some of my students, and it's easy enough to use (although the script itself contains some black magic aka "git plumbing commands").
ChrisA
My understanding is that by hosting the docs at RTD, we'll be getting better search feature powered by Elasticsearch, instead of the built-in sphinx search. If so, I would prefer this route.
Mariatta Wijaya
On Tue, Apr 11, 2017 at 12:11 PM, Brett Cannon brett@python.org wrote:
On Tue, 11 Apr 2017 at 11:28 Elvis Pranskevichus elprans@gmail.com wrote:
On Tuesday, April 11, 2017 2:09:33 PM EDT Oleg Broytman wrote:
On Tue, Apr 11, 2017 at 05:58:38PM +0000, Brett Cannon <
brett@python.org> wrote:
On Tue, 11 Apr 2017 at 10:52 Elvis Pranskevichus elprans@gmail.com
wrote:
On Tuesday, April 11, 2017 1:41:45 PM EDT Guido van Rossum wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
Is Gihub Pages considered an option? We host the asyncpg docs [1] there. Builds are done by Travis. The advantage is that we don't have to add another service dependency.
I guess so. No one has ever suggested it before or bothered to set it up to show how it would work. :)
GH Pages don't need much to setup. You have to choose between Org Pages (at the root of the subdomain) and Repo Pages (in subdirectories). I even believe GH Pages can do both on the same subdomain but I've never tried.
GH Pages support custom domain redirects [1]. "setup" in this case is putting together scripts to automate sphinx runs and the git push of the compiled output to the appropriate repo/branch.
So we would have to either set up another repo or commit the rendered files to the repo?
If there's interest in pursuing this option, I can help with setting things up.
I assume we would lose the widget Read the Docs provides which is handy for getting to the GitHub page for editing the content? I guess we lose anything else that https://docs.readthedocs.io/en/latest/features.html lists.
I'm just trying to think of what we gain/lose if we switch to GH Pages over RTFD as I don't want to end up with three different documentation hosting solutions sort of like we have now where we build the devguide, PEPs, and the docs themselves in special ways that all happen to be hosted on PSF infrastructure (e.g. getting Pygments installed for the PEPs has stalled out and it would be good that any solution we choose be flexible enough to handle this kind of situation: https://github.com/ python/peps/pull/229/files/70c6c1d010c1d50a375430c4ba6ae3 e4f439306c#r107471836).
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On Tue, 11 Apr 2017 at 10:42 Guido van Rossum guido@python.org wrote:
Last I heard from then, RTD was struggling financially. Hosting isn't really free, you know -- nor is administering a site like that :-). The PSF should offer to cover their costs for hosting the Python docs, which I presume are pretty dang popular compared to many other things currently hosted at RTD -- the peps and devguide won't cause any load in comparison.
I've brought it up with Eric Holscher before and he's said he doesn't want any open source project to feel like they have to pay. That being said I would ask the PSF to be a sponsor of some form if we did move all the docs over (for the devguide I'm not worried about the increased traffic). I've also suggested to Eric to apply for a PSF grant to move the docs over to Read the Docs so he can get paid to help make sure any such migration would be done well.
-Brett
-- --Guido van Rossum (python.org/~guido)
On Apr 11, 2017, at 12:50, Brett Cannon brett@python.org wrote:
With part of the goal of moving to GitHub being to minimize how much infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4).
From a current infrastructure POV, there are several different issues here. IIUC, currently we have at least three server instances involved in python.org docs.
1. I believe, the PEP docs are built and served from the main python.org server where the main Django-based python.org website is based. AFAIK, no one is proposing to replace that server. I'm not sure why the PEP docs are served there and not on the "docs" server (server 3 below); probably just an artifact of the gradual migration from the old python.org website infrastructure (e.g. dinsdale) several years ago.
2, The second server is used to serve the download files for releases, like source tar balls, binary installers (Win/Mac), and the pre-built documentation formats (PDF, HTML, epub, etc) for each release (for example, https://docs.python.org/release/3.6.1/download.html has download links like https://docs.python.org/ftp/python/doc/3.6.1/python-3.6.1-docs-pdf-letter.zi...). These files are built and managed by the release managers for each release and do not get updated.
3. The third server is used to serve the most recent HTML version of docs for *all* Python releases going back to 1.4. Docs prior to 2.6 (?) were not produced with Sphinx, so are effectively static HTML except possibly for JavaScript. The HTML versions of docs for releases still receiving maintenance fixes are auto-updated each day using Sphinx (for example, https://docs.python.org/release/3.6.1/index.html).
So, to actually reduce the number of servers in the PSF infrastructure, solutions for all of these docs need to be found. Since the main python.org server is not going away, I'm not sure what is gained by spending a lot of time on trying to eliminate the other two, which I suspect are very low-maintenance and could probably be combined. In other words, I guess I don't see how we gain much, if anything, in trying to move things to RTD.
-- Ned Deily nad@python.org -- []
On Tue, 11 Apr 2017 at 14:42 Ned Deily nad@python.org wrote:
On Apr 11, 2017, at 12:50, Brett Cannon brett@python.org wrote:
With part of the goal of moving to GitHub being to minimize how much
infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4).
From a current infrastructure POV, there are several different issues here. IIUC, currently we have at least three server instances involved in python.org docs.
- I believe, the PEP docs are built and served from the main python.org
server where the main Django-based python.org website is based. AFAIK, no one is proposing to replace that server. I'm not sure why the PEP docs are served there and not on the "docs" server (server 3 below); probably just an artifact of the gradual migration from the old python.org website infrastructure (e.g. dinsdale) several years ago.
python.org doesn't fall under python-dev's purview so I'm not proposing to do anything here except get the PEPs taken out.
2, The second server is used to serve the download files for releases, like source tar balls, binary installers (Win/Mac), and the pre-built documentation formats (PDF, HTML, epub, etc) for each release (for example, https://docs.python.org/release/3.6.1/download.html has download links like https://docs.python.org/ftp/python/doc/3.6.1/python-3.6.1-docs-pdf-letter.zi...). These files are built and managed by the release managers for each release and do not get updated.
- The third server is used to serve the most recent HTML version of docs
for *all* Python releases going back to 1.4. Docs prior to 2.6 (?) were not produced with Sphinx, so are effectively static HTML except possibly for JavaScript. The HTML versions of docs for releases still receiving maintenance fixes are auto-updated each day using Sphinx (for example, https://docs.python.org/release/3.6.1/index.html).
So, to actually reduce the number of servers in the PSF infrastructure, solutions for all of these docs need to be found. Since the main python.org server is not going away, I'm not sure what is gained by spending a lot of time on trying to eliminate the other two, which I suspect are very low-maintenance and could probably be combined. In other words, I guess I don't see how we gain much, if anything, in trying to move things to RTD.
From my perspective there are a few reasons for thinking about moving. One
is that low-maintenance isn't no-maintenance. Anything that helps take some load off the infrastructure team is a good thing in my opinion (especially when the PEPs and devguides are "unique snowflakes" in all of this as they are not built like the rest of docs.python.org). Two, getting changes to these machines isn't always easy or fast. As I pointed out to Elvis, we don't have Pygments installed so we can get source highlighting in PEPs and the PR to fix it has been sitting there for quite a while. And because the infrastructure is custom not many people even know where to make changes to change things like what dependencies are installed are on the machines (I mean how many people even knew there are three machines before this email?). Three, updates to any of these docs only happens a couple of times a day instead of instantly. Obviously not always a big deal, but for the PEPs it can be annoying when you want to email out the link to the rendered version and you can't simply because the cron job has not run yet.
So even if we can't get rid of docs.python.org and we don't move that over to RTD, at least getting python.org/dev/peps and docs.python.org/devguide to no longer be odd-ball infrastructure points is still a win in my book.
Brett Cannon wrote:
On Tue, 11 Apr 2017 at 14:42 Ned Deily <nad@python.org mailto:nad@python.org> wrote:
On Apr 11, 2017, at 12:50, Brett Cannon <brett@python.org <mailto:brett@python.org>> wrote: > With part of the goal of moving to GitHub being to minimize how much infrastructure we have to run, one of the long-term goals I have is to use Read the Docs to host Python's documentation. But to get there we have to move any "special" docs over first. That means relocating the devguide (it also means relocating the PEPs, but that's another issue and is blocked first and foremost by https://github.com/python/peps/issues/4). From a current infrastructure POV, there are several different issues here. IIUC, currently we have at least three server instances involved in python.org <http://python.org> docs. 1. I believe, the PEP docs are built and served from the main python.org <http://python.org> server where the main Django-based python.org <http://python.org> website is based. AFAIK, no one is proposing to replace that server. I'm not sure why the PEP docs are served there and not on the "docs" server (server 3 below); probably just an artifact of the gradual migration from the old python.org <http://python.org> website infrastructure (e.g. dinsdale) several years ago.python.org http://python.org doesn't fall under python-dev's purview so I'm not proposing to do anything here except get the PEPs taken out.
Big +1 on moving the PEPs.
2, The second server is used to serve the download files for releases, like source tar balls, binary installers (Win/Mac), and the pre-built documentation formats (PDF, HTML, epub, etc) for each release (for example, https://docs.python.org/release/3.6.1/download.html has download links like https://docs.python.org/ftp/python/doc/3.6.1/python-3.6.1-docs-pdf-letter.zip). These files are built and managed by the release managers for each release and do not get updated. 3. The third server is used to serve the most recent HTML version of docs for *all* Python releases going back to 1.4. Docs prior to 2.6 (?) were not produced with Sphinx, so are effectively static HTML except possibly for JavaScript. The HTML versions of docs for releases still receiving maintenance fixes are auto-updated each day using Sphinx (for example, https://docs.python.org/release/3.6.1/index.html). So, to actually reduce the number of servers in the PSF infrastructure, solutions for all of these docs need to be found. Since the main python.org <http://python.org> server is not going away, I'm not sure what is gained by spending a lot of time on trying to eliminate the other two, which I suspect are very low-maintenance and could probably be combined. In other words, I guess I don't see how we gain much, if anything, in trying to move things to RTD.From my perspective there are a few reasons for thinking about moving. One is that low-maintenance isn't no-maintenance. Anything that helps take some load off the infrastructure team is a good thing in my opinion (especially when the PEPs and devguides are "unique snowflakes" in all of this as they are not built like the rest of docs.python.org http://docs.python.org).
I agree. ReadTheDocs has some effective hooks and badges for operations/build status. It's straightforward to administer. It also has the added benefit that it is where many Python users go to search for documentation.
Two, getting changes to these machines isn't always easy or fast. As I pointed out to Elvis, we don't have Pygments installed so we can get source highlighting in PEPs and the PR to fix it has been sitting there for quite a while. And because the infrastructure is custom not many people even know where to make changes to change things like what dependencies are installed are on the machines (I mean how many people even knew there are three machines before this email?).
I also think this is a win since it is much easier to create a consistent build environment using readthedoc.yml and environment.yml files. I'm also in favor of RTD over Jekyll or hosting on GitHub since Eric has been a key evangelist for Python for a long time and the code for RTD is a Python/Django base. (As an aside, I agree with Guido that we should provide some level of financial support towards RTD sustainability).
Three, updates to any of these docs only happens a couple of times a day instead of instantly. Obviously not always a big deal, but for the PEPs it can be annoying when you want to email out the link to the rendered version and you can't simply because the cron job has not run yet.
So even if we can't get rid of docs.python.org http://docs.python.org and we don't move that over to RTD, at least getting python.org/dev/peps http://python.org/dev/peps and docs.python.org/devguide http://docs.python.org/devguide to no longer be odd-ball infrastructure points is still a win in my book.
While RTD may or may not be right for docs.python.org, I think it's worth looking at least mirroring the CPython docs on RTD. An added benefit for hosting on RTD is that there is lots of documentation around for using Sphinx/RTD which provides an opportunity for attracting more contributors to documentation and community management.
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On Apr 11, 2017, at 18:29, Carol Willing willingc@willingconsulting.com wrote:
While RTD may or may not be right for docs.python.org, I think it's worth looking at least mirroring the CPython docs on RTD. An added benefit for hosting on RTD is that there is lots of documentation around for using Sphinx/RTD which provides an opportunity for attracting more contributors to documentation and community management.
I guess I don't see the value of maintaining a RTD mirror for CPython docs as long as we are also serving the documents from docs.python.org, something which I think we need to continue to do. There is a fair amount of work that goes on behind the scenes with each release for maintaining the docs, not all of which is included in the docs source itself, things like updating symlinks and redirects and what not. Adding the mirror seems like adding extra work for no gain. And how would Sphinx documentation not also apply to the current CPython docs as served on docs.python.org?
-- Ned Deily nad@python.org -- []
Ned Deily wrote:
On Apr 11, 2017, at 18:29, Carol Willingwillingc@willingconsulting.com wrote:
While RTD may or may not be right for docs.python.org, I think it's worth looking at least mirroring the CPython docs on RTD. An added benefit for hosting on RTD is that there is lots of documentation around for using Sphinx/RTD which provides an opportunity for attracting more contributors to documentation and community management.
I guess I don't see the value of maintaining a RTD mirror for CPython docs as long as we are also serving the documents from docs.python.org, something which I think we need to continue to do.
I'm not advocating moving to the current docs right now, but I do believe there is value in having some form of the current master 3 docs on RTD. The value comes from these benefits:
- a place to begin simplifying the documentation workflow for both maintainability (bus factor) and attracting a wider group of contributors and reviewers - making the documentation development and review process simpler would encourage people who are good writers and educators to contribute more to the "how to use" sections of the docs
While the reference sections of the docs are excellent, the tutorials and "how to" sections could be more community focused.
I'm throwing it out as a suggestion for improvement, not a distraction or extra work. I'm happy to keep the status quo too.
There is a fair amount of work that goes on behind the scenes with each release for maintaining the docs, not all of which is included in the docs source itself, things like updating symlinks and redirects and what not. Adding the mirror seems like adding extra work for no gain. And how would Sphinx documentation not also apply to the current CPython docs as served on docs.python.org?
-- Ned Deily nad@python.org -- []
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On Apr 11, 2017, at 18:12, Brett Cannon brett@python.org wrote:
From my perspective there are a few reasons for thinking about moving. One is that low-maintenance isn't no-maintenance. Anything that helps take some load off the infrastructure team is a good thing in my opinion (especially when the PEPs and devguides are "unique snowflakes" in all of this as they are not built like the rest of docs.python.org). Two, getting changes to these machines isn't always easy or fast. As I pointed out to Elvis, we don't have Pygments installed so we can get source highlighting in PEPs and the PR to fix it has been sitting there for quite a while.
There are a number of people who could do that. Perhaps pinging the issue would help?
And because the infrastructure is custom not many people even know where to make changes to change things like what dependencies are installed are on the machines (I mean how many people even knew there are three machines before this email?). Three, updates to any of these docs only happens a couple of times a day instead of instantly. Obviously not always a big deal, but for the PEPs it can be annoying when you want to email out the link to the rendered version and you can't simply because the cron job has not run yet.
Most of the scripting is on github:
https://github.com/python/docsbuild-scripts https://github.com/python/pythondotorg
And any of the infra team can change the cron jobs to run more frequently :)
So even if we can't get rid of docs.python.org and we don't move that over to RTD, at least getting python.org/dev/peps and docs.python.org/devguide to no longer be odd-ball infrastructure points is still a win in my book.
OK, I don't have a big problem with moving the devguide and/or the PEPs, though I do think Guido has a point about being mindful of adding burden to the RTD folks (although, yes, that's probably not a big deal for the devguide and probably not for the PEPs either). And I don't think moving them is going to make any significant difference in the workload of the Python infra team as long as we do not outsource the main python.org web site components. My big concern would be with trying to move the rest of the docs. That is much more complicated and has implications for the release process and backward compatibility wrt URLs etc etc. (Release managers handle a lot of the complexities of this behind the scenes.) I would urge that we not make that (e.g. moving the rest of the docs) a goal for now.
-- Ned Deily nad@python.org -- []
On Apr 11, 2017, at 8:18 PM, Ned Deily nad@python.org wrote:
OK, I don't have a big problem with moving the devguide and/or the PEPs, though I do think Guido has a point about being mindful of adding burden to the RTD folks (although, yes, that's probably not a big deal for the devguide and probably not for the PEPs either). And I don't think moving them is going to make any significant difference in the workload of the Python infra team as long as we do not outsource the main python.org http://python.org/ web site components. My big concern would be with trying to move the rest of the docs. That is much more complicated and has implications for the release process and backward compatibility wrt URLs etc etc. (Release managers handle a lot of the complexities of this behind the scenes.) I would urge that we not make that (e.g. moving the rest of the docs) a goal for now.
*Puts Infra Hat on*
Outsourcing is always good where possible. We’re technically over our donation amount for Rackspace so we’re paying for some servers and bandwidth now. That being said, these particular machines are not high maintenance but it would still be nice to get them off of our infra if possible/reasonable.
— Donald Stufft
On Apr 11, 2017, at 20:25, Donald Stufft donald@stufft.io wrote:
*Puts Infra Hat on*
Outsourcing is always good where possible. We’re technically over our donation amount for Rackspace so we’re paying for some servers and bandwidth now. That being said, these particular machines are not high maintenance but it would still be nice to get them off of our infra if possible/reasonable.
Sure, I agree. But without substantially revising how we provide documentation and release downloadables - things which I suggest we not contemplate now as being beyond the scope of this project - we wouldn't be able to get rid of any machines for now. Moving the devguide and peps will not allow us to free any machines up - at least as best I understand it! So that's really not a factor at this point. Unless I'm missing something.
-- Ned Deily nad@python.org -- []
On Apr 11, 2017, at 18:12, Brett Cannon brett@python.org wrote:
python.org doesn't fall under python-dev's purview so I'm not proposing to do anything here except get the PEPs taken out.
Just for the record, the main python.org Django CMS is a critical component in making CPython releases. Release managers do lots of things there and we have some say in maintaining and enhancing it. So, to that extent, it's important to recognize that it is a core component of CPython releases and, as such, is of interest to python-dev.
-- Ned Deily nad@python.org -- []
participants (9)
-
Brett Cannon -
Carol Willing -
Chris Angelico -
Donald Stufft -
Elvis Pranskevichus -
Guido van Rossum -
Mariatta Wijaya -
Ned Deily -
Oleg Broytman