[Distutils] PyPI and Uploading Documentation

Donald Stufft donald at stufft.io
Fri May 15 15:48:21 CEST 2015


First, for anyone who isn't aware we recently migrated PyPI and TestPyPI so
that instead of storing files and documentation locally (really in a glusterfs
cluster) it will store them inside of S3. This will reduce maintenance overhead
of running PyPI by two servers since we'll no longer need to run our own
glusterfs cluster as well as improve the reliaiblity and scalability of the
PyPI service as a whole since we've had nothing but problems from glusterfs in
this regard.

One of the things that this brought to light was that the documentation
upload ability in PyPI is something that is not often used* however it
represents something which is one of our slowest routes. It's not a well
supported feature and I feel that it's going outside of the core competancy for
PyPI itself and instead PyPI should be focused on the files themselves. In
addition since the time this was added to PyPI a number of free services or
cheap services have came about that allow people to sanely upload raw document
without a reliance on any particular documentation system and we've also had
the rise of ReadTheDocs for when someone is using Sphinx as their documentation

I think that it's time to retire this aspect of PyPI which has never been well
supported and instead focus on just the things that are core to PyPI. I don't
have a fully concrete proposal for doing this, but I wanted to reach out here
and figure out if anyone had any ideas. The rough idea I have currently is to
simply disable new documentation uploads and add two new small features. One
will allow users to delete their existing documentation from PyPI and the other
would allow them to register a redirect which would take them from the current
location to wherever they move their documentation too. In order to prevent
breaking documentation for projects which are defunct or not actively
maintained we would maintain the archived documentation (sans what anyone has
deleted) indefinetely.

Ideally I hope people start to use ReadTheDocs instead of PyPI itself. I think
that ReadTheDocs is a great service with heavy ties to the Python community.
They will do a better job at hosting documentation than PyPI ever could since
that is their core goal. In addition there is a dialog between ReadTheDocs and
PyPI where there is an opportunity to add integration between the two sites as
well as features to ReadTheDocs that it currently lacks that people feel are a
requirement before we move PyPI's documentation to read-only.


* Out of ~60k projects only ~2.8k have ever uploaded documentation. It's not
  easy to tell if all of them are still using it as their primary source of
  documentation though or if it's old documentation that they just can't

Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150515/7bc65bdb/attachment.sig>

More information about the Distutils-SIG mailing list