[Python-Dev] 2.7 is here until 2020, please don't call it a waste.

[Nick Coghlan]

On 30 May 2015 10:46, "Alexander Walters" <tritium-list at sdamon.com> wrote:
>
> Python is a giant cache-miss generator.  A little performance boost on the opt-code dispatch isn't going to change that much.  If we really do care about improving python to do less environmental damage, then that is a discussion we should be having on it's own merits.  It was really out of place, even in this tangenty thread.

I think the way core development gets funded is entirely on topic for
the main core development mailing list, we just historically haven't
discussed it openly, even though some of us have been advocating for
various improvements to arrangements behind the scenes. I personally
consider becoming more transparent about how we go about that process
to be a good thing.

Intel are looking to get involved in CPython core development
*specifically* to work on performance improvements, so it's important
to offer folks in the community good reasons for why we're OK with
seeing at least some of that work applied to Python 2, rather than
restricting their contributions to Python 3.

The key is that the reason for not backporting performance
enhancements *differs* from the reasons for not backporting new
features. Rolling out new features has a broad ripple effect on the
Python ecosystem as books, training material, etc, all need to be
updated, and projects need to decide how to communicate their version
dependencies appropriately if they decide to depend on one of the
backported features. We pushed that kind of Python 2 change out to
PyPI years ago, and aside from truly exceptional cases like the
network security enhancements in PEPs 466 & 476 and the decision to
bundle pip to make it easier to access PyPI, it isn't open for
reconsideration as a general principle.

Performance improvements, by contrast, historically haven't been
backported solely due to the stability and maintainability
implications for CPython itself - they don't have a change management
ripple effect the way new language and standard library level features
do. That lack of negative ripple effects that cause work for other
people is why the proposal to contribute paid development time makes
such a big difference to the acceptability of Python 2.7 performance
patches, as it should be a pure gain for current Python 2.7 users, and
the paid development contributions should address the maintainability
concerns on the core development side (particularly since Intel are
*paying* for their coaching in core contribution practices and
politics, rather than expecting to receive that coaching from
community volunteers for free).

Backporting the computed goto patch is an easy place for them to
start, since the change is already well tested in the Python 3 branch,
but we don't expect this to be the end of the line for CPython 2 (or
3) performance enhancements.

However, we also shouldn't downplay the significance of this as a
notable policy change for the Python 2.7 maintenance branch, which
means it is useful to offer folks as many reasons as we can to help
them come to terms with the idea that Python 2 performance still
matters, and that it is only the limitations on our development and
support capacity that prevented us from further improving it
previously.

The commercially pragmatic reason is because Python 2 is where the
largest current installed base is today, so applying some of the
increased development capacity arising from sponsored contributions to
Python 2.7 performance improvements is a good way to demonstrate to
Python 2 developers that we still care about them *as Python 2 users*,
rather than only being interested in them as potential future Python 3
users. This is the rationale that's likely to get our paid
contributors (both current and future) on board with the idea, but it
isn't necessarily going to be compelling to folks that are here as
volunteers.

The first "What's in it for the volunteers?" reason is the one I
raised: giving the nod to an increased corporate developer presence in
Python 2 maintenance should eventually let volunteers stop worrying
about even Python 2.7 bug fix changes with a clear conscience,
confident that as volunteer efforts drop away redistributors and other
folks with an institutional interest will pick up the slack with paid
development time. "Do the fun stuff for free, figure out a way to get
paid for the boring-but-necessary stuff (or leave those tasks to
someone else that's getting paid to handle them)" is a good
sustainable approach to open source development, while trying to do it
*all* for free is a fast path to burnout.

Being ready, willing and able to handle the kind of situation created
by the Python 2->3 community transition is a large part of what it
means to offer commercial support for community driven open source
projects, as it buys customers' time for either migration technologies
to mature to a point where the cost of migration drops dramatically,
for the newer version of a platform to move far enough ahead of the
legacy version for there to be a clear and compelling business case
for forward porting existing software, or (as is the case we're aiming
to engineer for Python), both.

The environmental argument is another one that may be appealing to
folks that have no commercial interest in improving Python 2
performance. Regardless of which programming language we use to write
our own software, we all still share the same planet, so reducing the
amount of power we collectively use is something we can all benefit
from. Even though none of us have the necessary data to even guess at
the absolute magnitude of that reduction, we can at least be confident
it's a non-trivial portion of the amount of power Python 2
applications are currently consuming.

Regards,
Nick.