"Alex Walters" writes:

I'd like to know the rationale behind source only releases of cpython.

Software freedom entails the freedom to modify and build the software. For that, one needs the source form of the software. Portable software should be feasible to build from source, on a platform where no builds (of that particular release) were done before. For that, one needs the source form of the software.

I have an opinion on their utility and perhaps an idea about changing them, but I'd like to know why they are done

The above rationales seem sufficient to me. Are you looking for additional ones?

(as opposed to source+binary releases or no release at all)

I don't see a good justification for adding “source+binary” releases to the existing ones. We already have a source release (once), anda separate binary (one per platform). Why bother *also* making a source+binary release — presumably an additional one per platform? As for “no release at all”, it seems that those who want that can download it very quickly now :-)

before I head over to python-ideas. Is this documented somewhere where my google-fu can't find it?

I am not clear on why this would need specific documentation for Python; these are not issues that are different from any other software where the recipients have software freedom in the work. I hope these answers are useful. -- \ “My business is to teach my aspirations to conform themselves | `\ to fact, not to try and make facts harmonise with my | _o__) aspirations.“ —Thomas Henry Huxley, 1860-09-23 | Ben Finney

On May 16, 2018, at 1:06 AM, Ben Finney wrote:

...

I'd like to know the rationale behind source only releases of cpython.

Software freedom entails the freedom to modify and build the software. For that, one needs the source form of the software.

Portable software should be feasible to build from source, on a platform where no builds (of that particular release) were done before. For that, one needs the source form of the software.

I’m guessing the question isn’t why is it useful to have a source release of CPython, but why does CPython transition from having both source releases and binary releases to only source releases. My assumption is the rationale is to reduce the maintenance burden as time goes on for older release channels.

On Tue, May 15, 2018 at 10:55:07PM -0700, Chris Jerdonek wrote:

What does “no release at all” mean? If it’s not released, how would people use it?

I've been using Python 1.7 for years now. It is the perfect Python, with exactly all the features I want, and none that I don't want, and so much faster than Python 2.7 or 3.7 it is ridiculous. Unfortunately once I've woken up and tried to port my code to an actual computer, it doesn't work. *wink* In principle, we could continue adding fixes to a version in the source repository, but never cut a release with a new version. But I don't think we do that: once a version hits "no release", we stop adding fixes to the repo for that version: - full source and binary releases - source only releases - accumulate fixes in the VCS but don't cut a new release - stop making releases at all (the version is now unmaintained) The third (second from the bottom) doesn't (as far as I am aware) occur. -- Steve

16.05.18 07:35, Alex Walters пише:

In the spirit of learning why there is a fence across the road before I tear it down out of ignorance [1], I'd like to know the rationale behind source only releases of cpython. I have an opinion on their utility and perhaps an idea about changing them, but I'd like to know why they are done (as opposed to source+binary releases or no release at all) before I head over to python-ideas. Is this documented somewhere where my google-fu can't find it?

Taking a snapshot of sources at the random point of time is dangerous. You can get broken sources. Making a source only release means that sources are in consistent state, most buildbots are green, and core developers made necessary changes and stopped merging risky changes for some period before the release. The difference with source+binary releases is that latter adds additional burden to release managers: building binaries and installers on different platforms and publishing results on the site.

On 16 May 2018 at 09:34, Serhiy Storchaka wrote:

16.05.18 07:35, Alex Walters пише:

...

[1]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

And I wish that every author who suggested the idea for Python was familiar with the Chesterton's fence principle.

Agreed - thanks Alex for taking the time to research the issue! Paul

On May 16, 2018, at 00:35, Alex Walters wrote:

In the spirit of learning why there is a fence across the road before I tear it down out of ignorance [1], I'd like to know the rationale behind source only releases of cpython. I have an opinion on their utility and perhaps an idea about changing them, but I'd like to know why they are done (as opposed to source+binary releases or no release at all) before I head over to python-ideas. Is this documented somewhere where my google-fu can't find it?

The Python Developer's Guide has a discussion of the lifecycle of cPython releases here: https://devguide.python.org/#status-of-python-branches The ~short answer is that we produce source+binary (Windows and macOS binary installers) artifacts for release branches in "bugfix" (AKA "maintenance") mode (currently 3.6 and 2.7) as well as during the later stages of the in-development phase for future feature releases ("prerelease" mode) (currently 3.7); we produce only source releases for release branches in "security" mode. After the initial release of a new feature branch (for example, the upcoming 3.7.0 release), we will continue to support the previous release branch in bugfix mode for some overlapping period of time. So, for example, the current plan is to support both 3.7.x and 3.6.x (along with 2.7.x) in bugfix mode, releasing both source and binary artifacts for about six months after the 3.7.0 release. At that point, 3.6.x will transition to security-fix-only mode, where we will only produce releases on an as-needed basis and only in source form. Currently, 3.5 and 3.4 are also in security-fix-only mode. Eventually, usually five years after its initial release, a release branch will reach end-of-life: the branch will be frozen and no further issues for that release branch will be accepted nor will fixes be produced by Python Dev. 2.7 is a special case, with a greatly extended bugfix phase; it will proceed directly to end-of-life status as of 2020-01-01. There is more information later elsewhere in the devguide: https://devguide.python.org/devcycle/ and in the release PEPs linked in the Status of Python Branches section. Hope that helps! -- Ned Deily nad@python.org -- []

-----Original Message----- From: Ned Deily Sent: Wednesday, May 16, 2018 7:07 AM To: Alex Walters Cc: Python-Dev Subject: Re: [Python-Dev] What is the rationale behind source only releases?

On May 16, 2018, at 00:35, Alex Walters wrote:

...

In the spirit of learning why there is a fence across the road before I tear it down out of ignorance [1], I'd like to know the rationale behind

...

only releases of cpython. I have an opinion on their utility and

...

idea about changing them, but I'd like to know why they are done (as opposed to source+binary releases or no release at all) before I head over to python-ideas. Is this documented somewhere where my google-fu can't find it?

The Python Developer's Guide has a discussion of the lifecycle of cPython releases here:

https://devguide.python.org/#status-of-python-branches

The ~short answer is that we produce source+binary (Windows and macOS binary installers) artifacts for release branches in "bugfix" (AKA "maintenance") mode (currently 3.6 and 2.7) as well as during the later stages of the in-development phase for future feature releases ("prerelease" mode) (currently 3.7); we produce only source releases for release branches in "security" mode.

After the initial release of a new feature branch (for example, the upcoming 3.7.0 release), we will continue to support the previous release branch in bugfix mode for some overlapping period of time. So, for example, the current plan is to support both 3.7.x and 3.6.x (along with 2.7.x) in bugfix mode, releasing both source and binary artifacts for about six months after the 3.7.0 release. At that point, 3.6.x will transition to security-fix-only mode, where we will only produce releases on an as-needed basis and only in source form. Currently, 3.5 and 3.4 are also in security-fix-only mode. Eventually, usually five years after its initial release, a release branch will reach end-of-life: the branch will be frozen and no further issues for

release branch will be accepted nor will fixes be produced by Python Dev. 2.7 is a special case, with a greatly extended bugfix phase; it will

Thank you, that's exactly what I needed to read. source perhaps an that proceed

directly to end-of-life status as of 2020-01-01.

There is more information later elsewhere in the devguide:

https://devguide.python.org/devcycle/

and in the release PEPs linked in the Status of Python Branches section.

Hope that helps!

-- Ned Deily nad@python.org -- []

-----Original Message----- From: Paul Moore Sent: Wednesday, May 16, 2018 4:07 AM To: Alex Walters Cc: Python Dev Subject: Re: [Python-Dev] What is the rationale behind source only releases?

On 16 May 2018 at 05:35, Alex Walters wrote:

...

In the spirit of learning why there is a fence across the road before I tear it down out of ignorance [1], I'd like to know the rationale behind source only releases of cpython. I have an opinion on their utility and perhaps an idea about changing them, but I'd like to know why they are done (as opposed to source+binary releases or no release at all) before I head over to python-ideas. Is this documented somewhere where my google-fu can't find it?

[1]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

Assuming you're referring to the practice of no longer distributing binaries for patch releases of older versions of Python, the reason is basically as follows:

1. Producing binaries (to the quality we normally deliver - I'm not talking about auto-built binaries produced from a CI system) is a chunk of extra work for the release managers.

This is actually the heart of the reason I asked the question. CI tools are fairly good now. If the CI tools could be used in such a way to make the building of binary artifacts less of a burden on the release managers, would there be interest in doing that, and in the process, releasing binary artifact installers for all security update releases. My rationale for asking if its possible is... well.. security releases are important, and it's hard to ask Windows users to install Visual Studio and build python to use the most secure version of python that will run your python program. Yes there are better ideal solutions (porting your code to the latest and greatest feature release version), but that’s not a zero burden option either. If CI tools just aren't up to the task, then so be it, and this isn't something I would darken -ideas' door with.

2. The releases in question are essentially end of life, and we're only accepting security fixes. 3. Not even releasing sources means that people still using those releases will no longer have access to security fixes, so we'd be reducing the length of time we offer that level of support.

So extra binaries = more work for the release managers, no source release = less support for our users.

There's no reason we couldn't have a discussion on changing the policy, but any such discussion would probably need active support from the release managers if it were to stand any chance of going anywhere (as they are the people directly impacted by any such change).

Paul

On 17 May 2018 at 04:46, Alex Walters wrote:

...

1. Producing binaries (to the quality we normally deliver - I'm not talking about auto-built binaries produced from a CI system) is a chunk of extra work for the release managers.

This is actually the heart of the reason I asked the question. CI tools are fairly good now. If the CI tools could be used in such a way to make the building of binary artifacts less of a burden on the release managers, would there be interest in doing that, and in the process, releasing binary artifact installers for all security update releases.

My rationale for asking if its possible is... well.. security releases are important, and it's hard to ask Windows users to install Visual Studio and build python to use the most secure version of python that will run your python program. Yes there are better ideal solutions (porting your code to the latest and greatest feature release version), but that’s not a zero burden option either.

If CI tools just aren't up to the task, then so be it, and this isn't something I would darken -ideas' door with.

I honestly don't know if we're at a point where an auto-built security release would be sufficient and/or useful. That's mostly a question for the release manager(s). One sticking point might be that I believe the Windows installers (at least) are signed, and only the release managers have the signing key. It's probably *not* OK to leave the security releases unsigned ;-) So there would be a key management issue to address there. Paul.

On Thu, 17 May 2018 at 04:25 Paul Moore wrote:

...

...

1. Producing binaries (to the quality we normally deliver - I'm not talking about auto-built binaries produced from a CI system) is a chunk of extra work for the release managers.

This is actually the heart of the reason I asked the question. CI tools are fairly good now. If the CI tools could be used in such a way to make

On 17 May 2018 at 04:46, Alex Walters wrote: the building of binary artifacts less of a burden on the release managers, would there be interest in doing that, and in the process, releasing binary artifact installers for all security update releases.

...

My rationale for asking if its possible is... well.. security releases

are important, and it's hard to ask Windows users to install Visual Studio and build python to use the most secure version of python that will run your python program. Yes there are better ideal solutions (porting your code to the latest and greatest feature release version), but that’s not a zero burden option either.

...

If CI tools just aren't up to the task, then so be it, and this isn't

something I would darken -ideas' door with.

I honestly don't know if we're at a point where an auto-built security release would be sufficient and/or useful. That's mostly a question for the release manager(s). One sticking point might be that I believe the Windows installers (at least) are signed, and only the release managers have the signing key. It's probably *not* OK to leave the security releases unsigned ;-) So there would be a key management issue to address there.

If I understand things correctly, our planned migration to VSTS will include eventually automating the signing of the Windows releases so that part wont be an issue (which are currently signed manually be Steve).

On 17 May 2018 at 14:42, Brett Cannon wrote:

If I understand things correctly, our planned migration to VSTS will include eventually automating the signing of the Windows releases so that part wont be an issue (which are currently signed manually be Steve).

Somewhat off-topic for this discussion, but is there any background on the "planned migration to VSTS" that I can go and read up on? I've seen the comments on the committers mailing list and it looks cool, but I got the impression it was an addition, rather than a migration. (If it is just an additional CI service we'll be using, then no worries - more is always better!) Paul

On 17May2018 1004, Brett Cannon wrote:

On Thu, 17 May 2018 at 09:57 Paul Moore mailto:p.f.moore@gmail.com> wrote:

On 17 May 2018 at 14:42, Brett Cannon mailto:brett@python.org> wrote: > > If I understand things correctly, our planned migration to VSTS will include > eventually automating the signing of the Windows releases so that part wont > be an issue (which are currently signed manually be Steve).

Somewhat off-topic for this discussion, but is there any background on the "planned migration to VSTS" that I can go and read up on? I've seen the comments on the committers mailing list and it looks cool, but I got the impression it was an addition, rather than a migration. (If it is just an additional CI service we'll be using, then no worries - more is always better!)

To be a bit more specific, it's "planned assuming the testing of VSTS works out as we expect it to". :) IOW it isn't definite quite yet, but I am not expecting any blockers or people objecting, so in my head I'm optimistic that it's going to happen. (Any more discussion can be brought up on core-workflow.)

I just posted another email, but it looks like it's working out :) The migration hasn't really been planned as such, which is why so few people have heard about it. I've just spent the PyCon US sprints proving that it's a viable option to migrate to, and it can certainly help relieve the burden on AppVeyor and Travis. As Brett says, it'll be up to core-workflow as to whether we switch completely and when. On doing release builds through it, that is somewhat orthogonal. Right now, the Windows build still requires using my secure VM, which doesn't really let just anyone do the release, but we could easily get to a point where specifically authorised people can produce a complete build. Similarly, the macOS build probably shouldn't be done on the provided (up-to-date) CI machine, as I believe that would impact our compatibility. So perhaps having a well-powered and flexible build service available will help, but no promises. Cheers, Steve