[Distutils] on integrated docs in Warehouse and PyPI

Donald Stufft donald at stufft.io
Sun Jun 5 18:21:50 EDT 2016


> On Jun 5, 2016, at 6:04 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> 
> On 5 Jun 2016 2:18 am, "Ralf Gommers" <ralf.gommers at gmail.com <mailto:ralf.gommers at gmail.com>> wrote:
> >
> >
> >
> > On Sun, Jun 5, 2016 at 6:33 AM, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>> wrote:
> >>
> >>
> >> On 4 Jun 2016 6:54 am, "Donald Stufft" <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> >> >
> >> >
> >> >> On Jun 4, 2016, at 9:33 AM, Nathaniel Smith <njs at pobox.com <mailto:njs at pobox.com>> wrote:
> >> >>
> >> >> I think everyone would agree that having some nice doc hosting service available as an option would be, well, nice. Everyone likes options. But the current doc hosting is unpopular and feature poor, falls outside of the PyPI core mission, and is redundant with other more popular services, at a time when the PyPI developers are struggling to maintain core services.
> >> >
> >> >
> >> >
> >> > To add to what Nathaniel said here, there are a few problems with the current situation:
> >> >
> >> > Documentation hosting largely worked “OK” when it was just writing files out to disk, however we’ve since removed all use of the local disk (so that we can scale past 1 machine) and we’re now storing things in S3. This makes documentation hosting particularly expensive in terms of API calls because we need to do expensive list key operations to discover which files exist (versus package files where we have a database full of files).
> >>
> >> Amazon do offer higher level alternatives like https://aws.amazon.com/efs/ <https://aws.amazon.com/efs/> for use cases like PyPI's docs hosting that assume they have access to a normal filesystem.
> >>
> >> Given the credential management benefits of integrated docs,
> >
> > From the RTD blog post linked by Nathaniel: 
> > ""
> > Our proposed grant, for $48,000, is to build a separate instance that integrates with the Python Package Index’s upcoming website, Warehouse. This integration will provide automatic API reference documentation upon package release, with authentication tied to PyPI and simple configuration inside the distribution.
> > "" 
> >>
> >> it does seem worthwhile to me for the PSF to invest in a lowest common denominator static file hosting capability,
> >
> > Seems like a very poor way to spend money and developer time imho. The original post by Jason brings up a few shortcomings of RTD, but I'm amazed that that leads multiple people here to conclude that starting a new doc hosting effort is the right answer to that.
> 
> It isn't about funding a new idea, it's about keeping an existing solution working rather than breaking it abruptly and forcing other time strapped community volunteers to change how they do things immediately, or else leave their users without documentation.
> 

I mean “abruptly”. It was originally posted on distutils-sig just over a year ago with a relevant Warehouse issue at the same time [1]. In addition, we’re not deleting the capability to do it in legacy, just not adding it to Warehouse, so as we phase people over to Warehouse (like earlier I posted asking for folks to start opting in to using Warehouse) people will lose the ability to do this— but they’ll be able to go back and use legacy PyPI to regain it for as long as legacy PyPI is still running. We’re also going to switch the twine default (and the Python default) for uploads prior to legacy getting shut down. So essentially, we’re going to get a phased rollout with the ability to revert to the legacy behavior— at least until the legacy behavior is gone.

The plan as indicated a year ago also says we’re not going to delete the existing documentation (since that requires zero ongoing effort to keep up, it’s just chucking things in a S3 bucket and forgetting about it) so their users won’t be without documentation, they’ll just be without *new* documentation until they come up with an alternative plan (after having it been phased in over time).

> Since there's been zero previous discussion with distutils-sig or the PSF Board regarding improving the integration of ReadTheDocs with PyPI, I had absolutely no idea they had sought a grant from Mozilla to invest time in improving that aspect of things.
> 
> 

There was some discussion on the Warehouse issues [1] and Eric had been talking to me on IRC about it for awhile too. Probably it would have been good to mention it on distutils-sig too, but just to be clear that RTD hadn’t done this in stealth mode :)
> Regards,
> Nick
> 


[1] https://github.com/pypa/warehouse/issues/509 <https://github.com/pypa/warehouse/issues/509>

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160605/745dc3d6/attachment-0001.html>


More information about the Distutils-SIG mailing list