[Distutils] PyPI and Uploading Documentation

[Daniel Holth]

I'm using pypi's documentation hosting for pysdl2-cffi because I
thought it would be too difficult to run the documentation generator
(which parses documentation comments out of the wrapped C code) on the
readthedocs server. Perhaps there is a different way to do it that I'm
not familiar with.

On Fri, May 15, 2015 at 9:55 AM, Ian Cordasco
<graffatcolmingov at gmail.com> wrote:
> On Fri, May 15, 2015 at 8:48 AM, Donald Stufft <donald at stufft.io> wrote:
>> Hey!
>>
>> First, for anyone who isn't aware we recently migrated PyPI and TestPyPI so
>> that instead of storing files and documentation locally (really in a glusterfs
>> cluster) it will store them inside of S3. This will reduce maintenance overhead
>> of running PyPI by two servers since we'll no longer need to run our own
>> glusterfs cluster as well as improve the reliaiblity and scalability of the
>> PyPI service as a whole since we've had nothing but problems from glusterfs in
>> this regard.
>>
>> One of the things that this brought to light was that the documentation
>> upload ability in PyPI is something that is not often used* however it
>> represents something which is one of our slowest routes. It's not a well
>> supported feature and I feel that it's going outside of the core competancy for
>> PyPI itself and instead PyPI should be focused on the files themselves. In
>> addition since the time this was added to PyPI a number of free services or
>> cheap services have came about that allow people to sanely upload raw document
>> without a reliance on any particular documentation system and we've also had
>> the rise of ReadTheDocs for when someone is using Sphinx as their documentation
>> system.
>>
>> I think that it's time to retire this aspect of PyPI which has never been well
>> supported and instead focus on just the things that are core to PyPI. I don't
>> have a fully concrete proposal for doing this, but I wanted to reach out here
>> and figure out if anyone had any ideas. The rough idea I have currently is to
>> simply disable new documentation uploads and add two new small features. One
>> will allow users to delete their existing documentation from PyPI and the other
>> would allow them to register a redirect which would take them from the current
>> location to wherever they move their documentation too. In order to prevent
>> breaking documentation for projects which are defunct or not actively
>> maintained we would maintain the archived documentation (sans what anyone has
>> deleted) indefinetely.
>>
>> Ideally I hope people start to use ReadTheDocs instead of PyPI itself. I think
>> that ReadTheDocs is a great service with heavy ties to the Python community.
>> They will do a better job at hosting documentation than PyPI ever could since
>> that is their core goal. In addition there is a dialog between ReadTheDocs and
>> PyPI where there is an opportunity to add integration between the two sites as
>> well as features to ReadTheDocs that it currently lacks that people feel are a
>> requirement before we move PyPI's documentation to read-only.
>>
>> Thoughts?
>>
>> * Out of ~60k projects only ~2.8k have ever uploaded documentation. It's not
>>   easy to tell if all of them are still using it as their primary source of
>>   documentation though or if it's old documentation that they just can't
>>   delete.
>>
>>
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> I'm +1 on reducing the responsibilities of PyPI so it can act as an
> index/repository in a much more efficient manner. I'm also +1 on
> recommending people use ReadTheDocs. It supports more than just Sphinx
> so it's a rather flexible option. It's also open source, which means
> that anyone can contribute to it.
>
> I'm curious to hear more about integrations between PyPI and
> ReadTheDocs but I fully understand if they're not concrete enough to
> be worthy of discussion.
>
> --
> Ian
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig