On Wed, May 25, 2022 at 4:56 PM Thomas Caswell <tcaswell@gmail.com> wrote:
To reiterate what Ralf said, the possibility of Python going to a faster cadence was one of the things on our mind when drafting NEP 29 (see https://numpy.org/neps/nep-0029-deprecation_policy.html#n-minor-versions-of-... for why did not go with a fixed number of versions) because the reality of the effort and timelines to upgrade deployments does not change with Python release frequency. The two things we are trying to balance here are the needs of the projects to use the cool new stuff in Python and the needs of our (institutional) users who are using our libraries for operations where updating the version of Python deployed is not trivial.
There was a lot of discussion around how big the window should be with 42mo being about in the middle of what people advocated for. I am aware of institutions that are on an every-other Python upgrade pattern (py37 -> py39 -> (expected) py311) so always having at least 3 Pythons in the window is important. Based on what we have seen so far, I still think 42mo is a good choice, but do not think we have seen it in operations long enough to draw any conclusions (the downside of year-scale planning is you need to wait years to see if it worked out like you expected!) and hence do not think we should make any changes to the time window for another few years. That said, I am currently sympathetic to making the window longer and against making it shorter.
It may be useful to have a discussion about what "support" means though given the combinatorial rise of (Python version) x (Python implementation) x (Operating system) x (architecture) that we are building wheels for. I think it would still be within the spirit of NEP 29 to run CI on CPython for all of the supported versions (maybe across the 3 big OS's) and then only publish wheels for the latest (2 version of Python) x (full matrix of the rest).
We cannot do that (yet, at least). Failing to publish wheels for a supported Python version on a major OS is far worse than dropping support completely. This will remain true until the time that Pip starts defaulting to wheels-only and never picks the sdist if there's an older release for that platform + Python combo. Stealing some language/concepts from Microsoft (if I recall it correctly),
we should sort out which entries in that support matrix are Level 1 (CI + wheels), Level 2 (CI), Level 3 (we test something that looks like this), and Level 4 (your on your own, but we will take your patches to un-break things as you send them). We would obviously need more thought out definitions of the levels as well.
Agreed, this would be useful to map out. Starting with what we currently do, and not mix in any changes in our CI/wheel coverage, in order to decouple the support model from other, more contentious proposals. Cheers, Ralf
Tom
On Wed, May 25, 2022 at 5:23 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Tue, May 24, 2022 at 3:24 PM Ewout ter Hoeven < E.M.terHoeven@student.tudelft.nl> wrote:
Personally I would be in favor of updating NEP 29 to a support timespan in which at most 3 (minor) Python versions are supported. The development of Python is still at a high pace and NumPy is a high performance library which thrives when be able to adopt the latest Python features and having clean codebase without having "if sys.version_info[:2] < (3, n):" guards everywhere.
Especially with the developments of the faster Faster CPython project and the continued improvements in type hinting support, I think shortening the support timespan a bit is reasonable, beneficial and proportional.
More important, NEP 29 is adopted by many other projects, providing a signal to the ecosystem that's okay to focus on the latest Python versions. The overlap between users using a Python version more than 2.5 years old and wanting the latest features and performance is probably pretty small. Older NumPy and other project's releases will won't disappear and be usable when the Python requirements are lifted.
I don't think this is true. These graphs show that Python 3.7 is the version with the most downloaded wheels for, and our 1.21.x downloads are still higher than our 1.22.x downloads - very likely because we dropped 3.7 support in 1.22.x: https://pypistats.org/packages/numpy
I have seen problems popping up already in a few places with latest numpy not supported what is still the most commonly used Python version (don't have links, sorry - but they were real packaging-related issues). So I don't think it makes sense to shorten the time window. I also don't think there's a need to drop one version that's urgent - it's some effort and CI time, but the balance is decent right now.
Cheers, Ralf
In my opinion a 30 month support window would provide a nice balance. Each Python version is supported for over 2 minor releases, with updates on the maintenance branches extending that with another few months. You can comfortably stay a full minor Python version behind year-round and still use the latest NumPy. For even older Python versions the old NumPy releases will stay available, and it allow NumPy to move on to new Python features a full year earlier then with a 42 month support window. Which not only improves feature and performance adaptation, but also lowers maintenance, testing, backporting and CI effort.
Ewout ter Hoeven (EwoutH)