NumPy 1.20.x branch in two weeks
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
Hi All, Time to start planning for the 1.20.x branch. These are my thoughts at the moment: - Keep support for Python 3.6. Python 3.7 came out in June 2018, which seems too recent to be our oldest supported version. - Drop Python 3.6 for 1.21.x, that will make the oldest supported version about three years old. - Drop manylinux1 for 1.21.x. It would be nice to drop earlier, but manylinux2010 is pretty recent. There were 33 wheels in the 1.19.3 release, I think we can live with that for 1.20.x. I'm more worried about our tools aging out. After Python has settled into its yearly release cycle, I think we will end up supporting the latest 4 versions. Thoughts? Chuck
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
I know it seems silly, but would an amendment to NEP29 be reasonable? Many downstream packages look to numpy to understand what versions should be supported and NEP29 gave some good guidance. That said, if it is worth ignoring, or revisiting, some clarity on how to apply NEP29 given recent development would be appreciated. Best, Mark On Sat, Oct 31, 2020 at 8:24 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 4:28 PM Mark Harfouche <mark.harfouche@gmail.com> wrote: that we drop anything more than 42 months old, it is just recommended. The change in the Python release cycle has created some difficulty. With the yearly cycle, 4 python yearly releases will cover 3-4 years, which seems reasonable and we can probably drop to 3 releases towards the end, but with 3.7 coming 18 months after 3.6, four releases is on the long side, and three releases on the short side, so keeping 3.6 is the conservative choice. Once the yearly cycle sets in I think we will be fine. Chuck
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
I believe that it really helps to "lead by example". I don't mean to reference threads that you have all participated in, but the discussion in: https://mail.python.org/pipermail/scipy-dev/2020-August/024336.html Makes it clear to me at least, that downstream will follow the example that numpy sets. At the time of writing, it was anticipated that Python 3.7, 3.8, and maybe 3.9 would exist in Nov 1st. The support table https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table suggests that any release July 23 should only support 3.7. Barring COVID delays, it seems natural that in Nov 2020, support for Python 3.6 be dropped or that the NEP be revised. These decisions are hard, and take up alot of mental capacity, if the support window needs revisiting, that is fine, it just really helps to be able to point to a document (which is what NEP29 seemed to do).
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 6:48 PM Mark Harfouche <mark.harfouche@gmail.com> wrote:
The problem is that if we drop 3.6 the oldest version of Python will only be 30 months old, not 36. Dropping 3.6 for 1.20.x will make it 36 months, which is the recommended minimum coverage. I made sure that the language did not preclude longer support periods in any case. It would be helpful here if more people would comment, I would be happy to go with the shorter period if a majority of downstream projects want to go that way. It's not that I love 3.6, but there is no compelling reason to drop it, as there was for 3.5, at least that I am aware of. Chuck
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
NetworkX is currently planning to support 3.6 for our coming 2.6 release (dec 2020) and 3.0 release (early 2021). We had originally thought about following NEP 29. But I assumed it had been abandoned, since neither NumPy nor SciPy dropped Python 3.6 on Jun 23, 2020. NetworkX is likely to continue supporting whatever versions of Python both NumPy and SciPy support regardless of what NEP 29 says. I wouldn't be surprised if other projects do the same thing. Jarrod
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
I also misunderstood the purpose of the NEP. I assumed it was intended to encourage projects to drop old versions of Python. Other people have viewed the NEP similarly: https://github.com/networkx/networkx/issues/4027 If the intention of the NEP is to specify that projects not drop old version of Python too early, I don't think it is obvious from the NEP. It would be helpful if you added a simple motivation statement near the top of the document. Something like: ## Motivation and Scope The purpose of the NEP is to ensure projects in the scientific Python ecosystem don't drop support for old version of Python and NumPy too soon. On Sun, Nov 1, 2020 at 6:44 PM Jarrod Millman <millman@berkeley.edu> wrote:
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020, at 18:54, Jarrod Millman wrote:
Of all the packages, it makes sense for NumPy to behave most conservatively with depreciations. The NEP suggests allowable support periods, but as far as I recall does not enforce minimal support. Stephan Hoyer had a good recommendation on how we can clarify the NEP to be easier to intuit. Stephan, shall we make an ammendment to the NEP with your idea? Best regards, Stéfan
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 7:47 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
For reference, here was my proposed revision: https://github.com/numpy/numpy/pull/14086#issuecomment-649287648 Specifically, rather than saying "the latest release of NumPy supports all versions of Python released in the 42 months before NumPy's release", it says "NumPy will only require versions of Python that were released more than 24 months ago". In practice, this works out to the same thing (at least given Python's old 18 month release cycle). This changes the definition of the support window (in a way that I think is clearer and that works better for infrequent releases), but there is still the question of how large that window should be for NumPy. My personal opinion is that somewhere in the range of 24-36 months would be appropriate.
Best regards, Stéfan
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Mon, Nov 2, 2020 at 7:47 AM Stephan Hoyer <shoyer@gmail.com> wrote:
It was. It is. I think the NEP is very clear on that. Honestly we should just follow the NEP and drop 3.6 now for both NumPy and SciPy, I just am tired of arguing for it - which the NEP should have prevented being necessary, and I don't want to do again right now, so this will probably be my last email on this thread. Other
It doesn't *enforce* it, but the recommendation is very clear. It would be good to follow it.
I'm not sure it's clearer, the current NEP has a nice graphic and literally says "a project with a major or minor version release in November 2020 should support Python 3.7 and newer."). However happy to adopt it if it makes others happy - in the end it comes down to the same thing: it's recommended to drop Python 3.6 now. My personal opinion is that somewhere in the range of 24-36 months would be
appropriate.
+1 Cheers, Ralf
![](https://secure.gravatar.com/avatar/008b55030cffb9a4c4f7d8422e10343e.jpg?s=120&d=mm&r=g)
I like Ralf's email, and most of all I agree that the existing wording is clearer. My view on the NEP is that it does not mandate dropping support, but encourage it. In my projects I would drop it if I had use for Python 3.7+ features. It so happens that we want to use PEP-593 so we were grateful for NEP-29 giving us "permission" to drop 3.6. I would suggest that 3.6 be dropped immediately if there are any open PRs that would benefit from it, or code cleanups that it would enable. The point of the NEP is to short-circuit discussion about whether it's "worth" dropping 3.6. If it's valuable at all, do it. Thanks all, Juan. On Mon, 2 Nov 2020, at 2:01 AM, Ralf Gommers wrote:
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Mon, 2020-11-02 at 06:49 -0600, Juan Nunez-Iglesias wrote:
Probably the only thing that requires 3.7 in NumPy at this time is the module level `__getattr__`, which is used only for deprecations (and to make the financial removal slightly more gentle). I am not sure if PyPy already has stable support for 3.7 yet? Although PyPy is maybe not a big priority. We don't have to support 3.6 and I don't care if we do. Until this discussion my assumption was we would probably drop it. But, current master is tested against 3.6, so the main work seems release related. If Chuck thinks that is no hassle I don't mind if NumPy is a bit more conservative than NEP 29. Or is there a danger of setting a precedent where projects are wrongly expected to keep support just because NumPy still has it, so that NumPy not being conservative actually helps everyone? - Sebastian
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
Juan made a pretty good argument for keeping 3.6 support in the next scikit-image release, let me try to paraphrase: - Since nobody has made the PR to explicitly drop python 3.6 from the scikit-image build matrix, we will continue to support it, but if somebody were to make the PR, I (Juan) would support it. As for supporting PyPy: it already exists in the build matrix AFAICT. Breaking PyPy would be a deliberate action, as opposed to an accidental byproduct of dropping CPython 3.6. On Mon, Nov 2, 2020, 13:50 Sebastian Berg <sebastian@sipsolutions.net> wrote:
![](https://secure.gravatar.com/avatar/8744048060e5931c500d3c9d1ecb997e.jpg?s=120&d=mm&r=g)
I am in favor of dropping py36 for np1.20, I think it would be good to lead by example. Similar to pandas, the next Matplotlib release (3.4 targeted for Dec/Jan) will not support py36. Tom On Tue, Nov 3, 2020 at 9:18 AM Mark Harfouche <mark.harfouche@gmail.com> wrote:
-- Thomas Caswell tcaswell@gmail.com
![](https://secure.gravatar.com/avatar/f78541328b8123c0d16f58bae858c3c3.jpg?s=120&d=mm&r=g)
Hi, For what it's worth, python 3.6 is also dropped for astropy 4.2 (RC1 to be released in the next few days). We haven't yet formally adopted NEP29, but are very close to it peding some word smithing, and no one from the dev team was fighting for keeping support for 3.6. or numpy 1.16. Cheers, Brigitta On Tue, 3 Nov 2020 at 10:53, Thomas Caswell <tcaswell@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
Hi all, just to note: We discussed this yesterday briefly and decided to drop official support for 3.6 in the 1.20 release. We never had ambition to support 1.20 and there seems advantage in dropping it, if mainly for clarity and consistency with many other projects. If you disagree with this decision, please just bring it up so we can reconsider. Cheers, Sebastian PS: We may keep testing on 3.6 for the moment, at least for PyPy for technical reasons. On Tue, 2020-11-03 at 11:58 -0800, Brigitta Sipocz wrote:
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
Hi all, It looks like Python 3.6 support has not been dropped. For example, - https://github.com/numpy/numpy/blob/master/setup.py#L30 - https://github.com/numpy/numpy/blob/master/setup.py#L43 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L29 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L43 Other files (e.g., tox.ini) also make it appear that 3.6 has not been dropped. Did you decide to not drop Python 3.6? (Sorry if I missed the discussion.) Best regards, Jarrod PS. I am trying to decide whether NetworkX should drop 3.6 and prefer to follow NumPy's lead rather than NEP 29. On Thu, Nov 5, 2020 at 7:28 AM Sebastian Berg <sebastian@sipsolutions.net> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Tue, Dec 1, 2020 at 4:54 PM Jarrod Millman <millman@berkeley.edu> wrote:
Yes, we did decide to drop 3.6 support, but apparently we need a checklist when dropping support :) It is gone from testing and the wheel builds. <snip> Chuck
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
I know it seems silly, but would an amendment to NEP29 be reasonable? Many downstream packages look to numpy to understand what versions should be supported and NEP29 gave some good guidance. That said, if it is worth ignoring, or revisiting, some clarity on how to apply NEP29 given recent development would be appreciated. Best, Mark On Sat, Oct 31, 2020 at 8:24 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 4:28 PM Mark Harfouche <mark.harfouche@gmail.com> wrote: that we drop anything more than 42 months old, it is just recommended. The change in the Python release cycle has created some difficulty. With the yearly cycle, 4 python yearly releases will cover 3-4 years, which seems reasonable and we can probably drop to 3 releases towards the end, but with 3.7 coming 18 months after 3.6, four releases is on the long side, and three releases on the short side, so keeping 3.6 is the conservative choice. Once the yearly cycle sets in I think we will be fine. Chuck
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
I believe that it really helps to "lead by example". I don't mean to reference threads that you have all participated in, but the discussion in: https://mail.python.org/pipermail/scipy-dev/2020-August/024336.html Makes it clear to me at least, that downstream will follow the example that numpy sets. At the time of writing, it was anticipated that Python 3.7, 3.8, and maybe 3.9 would exist in Nov 1st. The support table https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table suggests that any release July 23 should only support 3.7. Barring COVID delays, it seems natural that in Nov 2020, support for Python 3.6 be dropped or that the NEP be revised. These decisions are hard, and take up alot of mental capacity, if the support window needs revisiting, that is fine, it just really helps to be able to point to a document (which is what NEP29 seemed to do).
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 6:48 PM Mark Harfouche <mark.harfouche@gmail.com> wrote:
The problem is that if we drop 3.6 the oldest version of Python will only be 30 months old, not 36. Dropping 3.6 for 1.20.x will make it 36 months, which is the recommended minimum coverage. I made sure that the language did not preclude longer support periods in any case. It would be helpful here if more people would comment, I would be happy to go with the shorter period if a majority of downstream projects want to go that way. It's not that I love 3.6, but there is no compelling reason to drop it, as there was for 3.5, at least that I am aware of. Chuck
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
NetworkX is currently planning to support 3.6 for our coming 2.6 release (dec 2020) and 3.0 release (early 2021). We had originally thought about following NEP 29. But I assumed it had been abandoned, since neither NumPy nor SciPy dropped Python 3.6 on Jun 23, 2020. NetworkX is likely to continue supporting whatever versions of Python both NumPy and SciPy support regardless of what NEP 29 says. I wouldn't be surprised if other projects do the same thing. Jarrod
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
I also misunderstood the purpose of the NEP. I assumed it was intended to encourage projects to drop old versions of Python. Other people have viewed the NEP similarly: https://github.com/networkx/networkx/issues/4027 If the intention of the NEP is to specify that projects not drop old version of Python too early, I don't think it is obvious from the NEP. It would be helpful if you added a simple motivation statement near the top of the document. Something like: ## Motivation and Scope The purpose of the NEP is to ensure projects in the scientific Python ecosystem don't drop support for old version of Python and NumPy too soon. On Sun, Nov 1, 2020 at 6:44 PM Jarrod Millman <millman@berkeley.edu> wrote:
![](https://secure.gravatar.com/avatar/d9ac9213ada4a807322f99081296784b.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020, at 18:54, Jarrod Millman wrote:
Of all the packages, it makes sense for NumPy to behave most conservatively with depreciations. The NEP suggests allowable support periods, but as far as I recall does not enforce minimal support. Stephan Hoyer had a good recommendation on how we can clarify the NEP to be easier to intuit. Stephan, shall we make an ammendment to the NEP with your idea? Best regards, Stéfan
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Sun, Nov 1, 2020 at 7:47 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
For reference, here was my proposed revision: https://github.com/numpy/numpy/pull/14086#issuecomment-649287648 Specifically, rather than saying "the latest release of NumPy supports all versions of Python released in the 42 months before NumPy's release", it says "NumPy will only require versions of Python that were released more than 24 months ago". In practice, this works out to the same thing (at least given Python's old 18 month release cycle). This changes the definition of the support window (in a way that I think is clearer and that works better for infrequent releases), but there is still the question of how large that window should be for NumPy. My personal opinion is that somewhere in the range of 24-36 months would be appropriate.
Best regards, Stéfan
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Mon, Nov 2, 2020 at 7:47 AM Stephan Hoyer <shoyer@gmail.com> wrote:
It was. It is. I think the NEP is very clear on that. Honestly we should just follow the NEP and drop 3.6 now for both NumPy and SciPy, I just am tired of arguing for it - which the NEP should have prevented being necessary, and I don't want to do again right now, so this will probably be my last email on this thread. Other
It doesn't *enforce* it, but the recommendation is very clear. It would be good to follow it.
I'm not sure it's clearer, the current NEP has a nice graphic and literally says "a project with a major or minor version release in November 2020 should support Python 3.7 and newer."). However happy to adopt it if it makes others happy - in the end it comes down to the same thing: it's recommended to drop Python 3.6 now. My personal opinion is that somewhere in the range of 24-36 months would be
appropriate.
+1 Cheers, Ralf
![](https://secure.gravatar.com/avatar/008b55030cffb9a4c4f7d8422e10343e.jpg?s=120&d=mm&r=g)
I like Ralf's email, and most of all I agree that the existing wording is clearer. My view on the NEP is that it does not mandate dropping support, but encourage it. In my projects I would drop it if I had use for Python 3.7+ features. It so happens that we want to use PEP-593 so we were grateful for NEP-29 giving us "permission" to drop 3.6. I would suggest that 3.6 be dropped immediately if there are any open PRs that would benefit from it, or code cleanups that it would enable. The point of the NEP is to short-circuit discussion about whether it's "worth" dropping 3.6. If it's valuable at all, do it. Thanks all, Juan. On Mon, 2 Nov 2020, at 2:01 AM, Ralf Gommers wrote:
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Mon, 2020-11-02 at 06:49 -0600, Juan Nunez-Iglesias wrote:
Probably the only thing that requires 3.7 in NumPy at this time is the module level `__getattr__`, which is used only for deprecations (and to make the financial removal slightly more gentle). I am not sure if PyPy already has stable support for 3.7 yet? Although PyPy is maybe not a big priority. We don't have to support 3.6 and I don't care if we do. Until this discussion my assumption was we would probably drop it. But, current master is tested against 3.6, so the main work seems release related. If Chuck thinks that is no hassle I don't mind if NumPy is a bit more conservative than NEP 29. Or is there a danger of setting a precedent where projects are wrongly expected to keep support just because NumPy still has it, so that NumPy not being conservative actually helps everyone? - Sebastian
![](https://secure.gravatar.com/avatar/8a6447f6df923c9aaac74a3b47a1e81c.jpg?s=120&d=mm&r=g)
Juan made a pretty good argument for keeping 3.6 support in the next scikit-image release, let me try to paraphrase: - Since nobody has made the PR to explicitly drop python 3.6 from the scikit-image build matrix, we will continue to support it, but if somebody were to make the PR, I (Juan) would support it. As for supporting PyPy: it already exists in the build matrix AFAICT. Breaking PyPy would be a deliberate action, as opposed to an accidental byproduct of dropping CPython 3.6. On Mon, Nov 2, 2020, 13:50 Sebastian Berg <sebastian@sipsolutions.net> wrote:
![](https://secure.gravatar.com/avatar/8744048060e5931c500d3c9d1ecb997e.jpg?s=120&d=mm&r=g)
I am in favor of dropping py36 for np1.20, I think it would be good to lead by example. Similar to pandas, the next Matplotlib release (3.4 targeted for Dec/Jan) will not support py36. Tom On Tue, Nov 3, 2020 at 9:18 AM Mark Harfouche <mark.harfouche@gmail.com> wrote:
-- Thomas Caswell tcaswell@gmail.com
![](https://secure.gravatar.com/avatar/f78541328b8123c0d16f58bae858c3c3.jpg?s=120&d=mm&r=g)
Hi, For what it's worth, python 3.6 is also dropped for astropy 4.2 (RC1 to be released in the next few days). We haven't yet formally adopted NEP29, but are very close to it peding some word smithing, and no one from the dev team was fighting for keeping support for 3.6. or numpy 1.16. Cheers, Brigitta On Tue, 3 Nov 2020 at 10:53, Thomas Caswell <tcaswell@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
Hi all, just to note: We discussed this yesterday briefly and decided to drop official support for 3.6 in the 1.20 release. We never had ambition to support 1.20 and there seems advantage in dropping it, if mainly for clarity and consistency with many other projects. If you disagree with this decision, please just bring it up so we can reconsider. Cheers, Sebastian PS: We may keep testing on 3.6 for the moment, at least for PyPy for technical reasons. On Tue, 2020-11-03 at 11:58 -0800, Brigitta Sipocz wrote:
![](https://secure.gravatar.com/avatar/547665790a071bb74c66ff10cc3a378a.jpg?s=120&d=mm&r=g)
Hi all, It looks like Python 3.6 support has not been dropped. For example, - https://github.com/numpy/numpy/blob/master/setup.py#L30 - https://github.com/numpy/numpy/blob/master/setup.py#L43 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L29 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L43 Other files (e.g., tox.ini) also make it appear that 3.6 has not been dropped. Did you decide to not drop Python 3.6? (Sorry if I missed the discussion.) Best regards, Jarrod PS. I am trying to decide whether NetworkX should drop 3.6 and prefer to follow NumPy's lead rather than NEP 29. On Thu, Nov 5, 2020 at 7:28 AM Sebastian Berg <sebastian@sipsolutions.net> wrote:
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Tue, Dec 1, 2020 at 4:54 PM Jarrod Millman <millman@berkeley.edu> wrote:
Yes, we did decide to drop 3.6 support, but apparently we need a checklist when dropping support :) It is gone from testing and the wheel builds. <snip> Chuck
participants (12)
-
Brigitta Sipocz
-
Charles R Harris
-
Jarrod Millman
-
Jeff Reback
-
Juan Nunez-Iglesias
-
Kevin Sheppard
-
Mark Harfouche
-
Ralf Gommers
-
Sebastian Berg
-
Stefan van der Walt
-
Stephan Hoyer
-
Thomas Caswell