[Linux-SIG] PEP 476 and backward incompatibilities for distros

[Nick Coghlan]

On 9 October 2015 at 08:27, Barry Warsaw <barry at python.org> wrote:
> On Oct 09, 2015, at 04:24 AM, Nick Coghlan wrote:
>>Providing a better migration path for cases where commercial support
>>obligations apply is what the proposed PEP 493 recommendations are about:
>>https://www.python.org/dev/peps/pep-0493/
>
> The other thing to keep in mind is that distros generally have a team
> dedicated to security issues.  Was this change in response to a CVE?

While we made the change first, it eventually became CVE-2014-9365:
https://bugzilla.redhat.com/show_bug.cgi?id=1173041

Alex made the request: http://seclists.org/oss-sec/2014/q4/1022

The original escalation that led to my writing PEP 466 to upgrade
2.7's network security features and Alex writing PEP 476 came from
Rackspace due to security problems encountered in OpenStack
development.

> Certainly those teams are better able to judge the implications for the
> insecure compatible default for users and applications on their platform.  I
> don't envy the work they do to balance those backward compatibility issues
> against increased security.

Yeah, Red Hat's SRT have been some of the main reviewers for the
proposals in PEP 493 as we've tried to figure out how best to handle
PEP 476 in RHEL

> A change here by upstream is an end-around of those team's responsibilities.
> Which might still mean it's the right decision for *Python* (and we can argue
> that on python-dev) but it behooves us to make that very obvious for
> downstream consumers who come out on the other side of that equation.

Alex wrote PEP 476 in August 2014, Guido approved it in October [1],
Python 2.7.9 was released and the CVE was issued in December, and then
Python 3.4.3 was released in February 2015.

A new section was also added to the 3.4 What's New document:
https://docs.python.org/dev/whatsnew/3.4.html#changed-in-3-4-3

[1] https://mail.python.org/pipermail/python-dev/2014-October/136676.html

>>This is on redistributors to handle - if we want more or different
>>documentation in cases like these, then we need to be paying people to
>>spend more time on representing our interests (and those of our users) in
>>CPython upstream development, rather than focusing our efforts solely on
>>integration work for our respective operating systems.
>
> No, because as I said it doesn't only affect distros with paid employees.
> Maybe it affects distros that are purely volunteer run.  Maybe it affects
> people installing Python from source for their own internal usage.  Etc.

It's my view that folks running open source community projects
directly have no legitimate expectation of interface stability unless
they're running the free variant of a commercially supported product
with strong stability guarantees (e.g. Ubuntu LTS, CentOS/RHEL,
openSUSE/SLES).

I've become much, much harsher about this in recent years as I've come
to understand the degree to which most organisations take open source
developers for granted (my perspective had previously been skewed by
Boeing's approach, where Supplier Management required that we first
evaluate the project's sustaining engineering model before allowing us
to incorporate a new open source project into our designs. I assumed
every commercial operation consumed open source that way, but I was
wrong - most orgs instead seem to go straight from the "No open source
allowed" model to the "Magic Internet Pixies" approach, where they'll
happily build business critical software atop dependencies where they
have no idea who is maintaining them or why they're working on them)

> Upstream is who knows that a compatibility break is coming in a point
> release.  We owe it to our users to first, be very careful about that, and
> second to make sure that information is very well advertised.

We did all that, though. If folks missed the PEP *and* the CVE *and*
the What's New update *and* the NEWS entry *and* the voluminous
mailing list threads...

One place we did miss was the download pages for 3.4.3, but does
anyone in the distros actually read those rather than the NEWS file?

>>The perpetuation of bad default behaviours in the name of backwards
>>compatibility is a large part of what turned our industry into the insecure
>>user hostile mess that it is today, though.
>
> So, we're going to have to break compatibility at some point to fix it.  The
> question is where and when?  Buried in a point release?  Wait until the next
> major release?  In documentation, a tech note, or announcement?

While we unfortunately didn't capture it in the PEP, the main benefit
of updating 3.4.3 rather than waiting until 3.5.0 was released was to
avoid creating a situation where folks suffered a security regression
in upgrading from 2.7.9+ to 3.4.3+. Making the change in both versions
kept that regression window to less than 3 months for folks consuming
upstream directly, while leaving downstreams to figure out how to
incorporate the change into their Python 2 & 3 stacks in a less
disruptive way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia