[Python-Dev] Automating the maintenance release pipeline (was Re: Request for CPython 3.5.3 release)

[Chris Krycho]

The bots Mozilla runs around both Rust and Servo should make a lot of this much lower overhead if they can be repurposed (as I believe several other communities have already done).

Homu, the build manager tool, runs CI (including buildbots, Travis, etc.), is integrated with GitHub PRs so maintainers can trigger it with a comment there, and can also roll up a bunch of changes into one (handy to pull together e.g. a bunch of small documentation changes like typo fixes): https://github.com/barosl/homu That seems to keep the pain level of having an always-building-and-passing-tests nightly version much lower.

Aside: I don't want to flood these discussions with "yay Rust!" stuff, so this will probably be my last such response unless something else really jumps out. ;-) Thanks for the work you're all doing here.

Regards,
Chris Krycho

> On Jul 3, 2016, at 7:34 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> On 4 July 2016 at 00:39, Guido van Rossum <guido at python.org> wrote:
>> Another thought recently occurred to me. Do releases really have to be
>> such big productions? A recent ACM article by Tom Limoncelli[1]
>> reminded me that we're doing releases the old-fashioned way --
>> infrequently, and with lots of manual labor. Maybe we could
>> (eventually) try to strive for a lighter-weight, more automated
>> release process? It would be less work, and it would reduce stress for
>> authors of stdlib modules and packages -- there's always the next
>> release. I would think this wouldn't obviate the need for carefully
>> planned and timed "big deal" feature releases, but it could make the
>> bug fix releases *less* of a deal, for everyone.
> 
> Yes, getting the maintenance releases to the point of being largely
> automated would be beneficial. However, I don't think the problem is
> lack of desire for that outcome, it's that maintaining the release
> toolchain pretty much becomes a job at that point, as you really want
> to be producing nightly builds (since the creation of those nightlies
> in effect becomes the regression test suite for the release
> toolchain), and you also need to more strictly guard against even
> temporary regressions in the maintenance branches.
> 
> There are some variants we could pursue around that model (e.g.
> automating Python-only updates without automating updates that require
> rebuilding the core interpreter binaries for Windows and Mac OS X),
> but none of it is the kind of thing likely to make anyone say "I want
> to work on improving this in my free time". Even for commercial
> redistributors, it isn't easy for us to make the business case for
> assigning someone to work on it, since we're generally working from
> the source trees rather than the upstream binary releases.
> 
> I do think it's worth putting this into our bucket of "ongoing
> activities we could potentially propose to the PSF for funding",
> though. I know Ewa (Jodlowska, the PSF's Director of Operations) is
> interested in better supporting the Python development community
> directly (hence https://donate.pypi.io/ ) in addition to the more
> indirect community building efforts like PyCon US and the grants
> program, so I've been trying to build up a mental list of CPython
> development pain points where funded activities could potentially
> improve the contributor experience for volunteers. So far I have:
> 
> - issue triage (including better acknowledging folks that help out
> with triage efforts)
> - patch review (currently "wait and see" pending the impact of the
> GitHub migration)
> - nightly pre-release builds (for ease of contribution without first
> becoming a de facto C developer and to help make life easier for
> release managers)
> 
> That last one is a new addition to my list based on this thread, and I
> think it's particularly interesting in that it would involve a much
> smaller set of target users than the first two (with the primary
> stakeholders being the release managers and the folks preparing the
> binary installers), but also a far more concrete set of deliverables
> (i.e. nightly binary builds being available for active development and
> maintenance branches for at least Windows and Mac OS X, and
> potentially for the manylinux1 baseline API defined in PEP 513)
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris%40chriskrycho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160703/85409e51/attachment.html>