[Distutils] PyPI and Uploading Documentation

[Ian Cordasco]

On Fri, May 15, 2015 at 8:48 AM, Donald Stufft <donald at stufft.io> wrote:
> Hey!
>
> First, for anyone who isn't aware we recently migrated PyPI and TestPyPI so
> that instead of storing files and documentation locally (really in a glusterfs
> cluster) it will store them inside of S3. This will reduce maintenance overhead
> of running PyPI by two servers since we'll no longer need to run our own
> glusterfs cluster as well as improve the reliaiblity and scalability of the
> PyPI service as a whole since we've had nothing but problems from glusterfs in
> this regard.
>
> One of the things that this brought to light was that the documentation
> upload ability in PyPI is something that is not often used* however it
> represents something which is one of our slowest routes. It's not a well
> supported feature and I feel that it's going outside of the core competancy for
> PyPI itself and instead PyPI should be focused on the files themselves. In
> addition since the time this was added to PyPI a number of free services or
> cheap services have came about that allow people to sanely upload raw document
> without a reliance on any particular documentation system and we've also had
> the rise of ReadTheDocs for when someone is using Sphinx as their documentation
> system.
>
> I think that it's time to retire this aspect of PyPI which has never been well
> supported and instead focus on just the things that are core to PyPI. I don't
> have a fully concrete proposal for doing this, but I wanted to reach out here
> and figure out if anyone had any ideas. The rough idea I have currently is to
> simply disable new documentation uploads and add two new small features. One
> will allow users to delete their existing documentation from PyPI and the other
> would allow them to register a redirect which would take them from the current
> location to wherever they move their documentation too. In order to prevent
> breaking documentation for projects which are defunct or not actively
> maintained we would maintain the archived documentation (sans what anyone has
> deleted) indefinetely.
>
> Ideally I hope people start to use ReadTheDocs instead of PyPI itself. I think
> that ReadTheDocs is a great service with heavy ties to the Python community.
> They will do a better job at hosting documentation than PyPI ever could since
> that is their core goal. In addition there is a dialog between ReadTheDocs and
> PyPI where there is an opportunity to add integration between the two sites as
> well as features to ReadTheDocs that it currently lacks that people feel are a
> requirement before we move PyPI's documentation to read-only.
>
> Thoughts?
>
> * Out of ~60k projects only ~2.8k have ever uploaded documentation. It's not
>   easy to tell if all of them are still using it as their primary source of
>   documentation though or if it's old documentation that they just can't
>   delete.
>
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>

I'm +1 on reducing the responsibilities of PyPI so it can act as an
index/repository in a much more efficient manner. I'm also +1 on
recommending people use ReadTheDocs. It supports more than just Sphinx
so it's a rather flexible option. It's also open source, which means
that anyone can contribute to it.

I'm curious to hear more about integrations between PyPI and
ReadTheDocs but I fully understand if they're not concrete enough to
be worthy of discussion.

--
Ian