Hi all, Apologies if this mail appear out of thread I just subscribed to respond.

Yeah, agreed. I don't feel like this is incompatible with the spirit of python3statement.org, though looking at the text I can see how it's not clear. My guess is they'd be happy to adjust the text, especially if it lets them add numpy :-). CC'ing Thomas and Matthias.

Happy to see NumPy at least having this conversation ! I agree with Thomas, we're pretty loose on what dropping support means; one of the main reason for the Python-3-Statement is communication to users and other project; and covey that there is a strong intent that you have until 2020 to get ready (if not before). The voice of NumPy have a huge weight in the balance. I quickly went through the thread and have a few responses:

NumPy (and to a lesser extent SciPy) is in a tough position being at the bottom of many scientific Python programming stacks. Whenever you drop Python 2 support is going to upset someone.

And that is why you should decide of doing it at some point, and telling it to the world and the sooner you decide and advertise it (regardless of effective "deadline" the better. The Scientific Python is in a catch 22 position; Most of the ecosystem will not drop 2.7 because "numpy is still compatible python 2.7", and numpy does not drop it because "many packages rely on numpy support for 2.7".

We'd have to make sure we could persuade pypi to give the older version for Windows, by default - I don't know if that is possible.

I don't think there is, though if you tag a release with `requires_python>3.3`, then pip 9+ users on python 2.7 will not even realise there are new release compatible only with 3.3+. Technically you can make numpy a meta-package that requires numpy-27 on windows only... but it has its own drawbacks.

And yet another is that when we do finally drop 2.7, I think we'd want to get the full benefits of doing so. That's new 3.x features (@ in particular), cleaning up lots of support code, etc.

Regarding http://www.python3statement.org/: I'd say that as long as there are people who want to spend their energy on the LTS release (contributors *and* enough maintainer power to review/merge/release), we should not actively prevent them from doing that.

These two are _in practice_ against each other; if you do major cleaning then most backports will have a hard time being auto applied (just a warning). If you have a team that want to do a LTS I would suggest "cleaning" only when you are actually touching some code and the python-2 support code is in the way. not cleaning "for the sake of cleaning" at least until the 2 code base are far enough. We have a bot on Jupyter/Matplotlib that help to backport PRs to older branches. I'm happy to open it to the numpy org if it helps.

It is too ambitious to pledge to drop support for Python 2.7 no later than 2020, coinciding with the Python development team’s timeline for dropping support for Python 2.7?

The hardest part is communication. And not just "We're dropping in 2020" but also "We still care about you, 2.7 users", ans especially tell 2.7 users and old pip users how to correctly pip their dependency on numpy (to still get LTS if LTS there is) One more thing, there is a lot of of discussion about a "Volonteer LTS", you may also want to consider a partnership with a company for an officially recommended commercial offer. Thanks, -- Matthias On Thu, Nov 9, 2017 at 10:21 AM, Nathaniel Smith wrote:

See Thomas's reply quoted below (it was rejected by the mailing list since he's not subscribed):

On Nov 9, 2017 01:24, "Thomas Kluyver" wrote:

On Thu, Nov 9, 2017, at 08:52 AM, Nathaniel Smith wrote:

On Nov 8, 2017 23:59, "Ralf Gommers" wrote:

Regarding http://www.python3statement.org/: I'd say that as long as there are people who want to spend their energy on the LTS release (contributors *and* enough maintainer power to review/merge/release), we should not actively prevent them from doing that.

Yeah, agreed. I don't feel like this is incompatible with the spirit of python3statement.org, though looking at the text I can see how it's not clear. My guess is they'd be happy to adjust the text, especially if it lets them add numpy :-). CC'ing Thomas and Matthias.

Thanks Nathaniel. We have (IMO) left a degree of deliberate ambiguity around precisely what 'drop support' means, because it's not going to be the same for all projects. The nature of open source also means that there can be ambiguity over what 'support' entails and who is considered part of the project.

I would say that the idea of the statement is compatible with an LTS release series receiving critical bugfixes beyond 2020, while the main energy of the project is focused on Py3-only feature releases.

[If numpy-discussion doesn't allow non-member posts, feel free to pass this on or quote it in on-list messages]

Thomas

On Nov 9, 2017, at 13:35, Charles R Harris wrote:

One thing worth considering might be making the release that drops 2.7 NumPy 2.0 just so that there is a clear break point. Then if someone wants to continue the 1.x line of releases supporting 2.7 they can do so. ISTR that git now has some features that might aid in that. If the `requires` bit works well with pip then we can also share the same pip page (maybe).

I personally think this is definitely advisable. FWIW Bokeh will definitely be bumping major numbers when dropping Python 2 support, or classic notebook support, etc. Bryan

Fortunately we can wait until we're a bit closer before we have to make any final decision on the version numbering :-) Right now though it would be good to start communicating to users/downstreams about whatever our plans our though, so they can make plans. Here's a first attempt at some text we can put in the documentation and point people to -- any thoughts, on either the plan or the wording? ---- DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK ---- The Python core team plans to stop supporting Python 2 in 2020. The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2 support as well. Now that we're entering the final years of community-supported Python 2, the NumPy project wants to clarify our plans, with the goal of to helping our downstream ecosystem make plans and accomplish the transition with as little disruption as possible. Our current plan is as follows: Until **December 31, 2018**, all NumPy releases will fully support both Python 2 and Python 3. Starting on **January 1, 2019**, any new feature releases will support only Python 3. The last Python-2-supporting release will be designated as a long-term support (LTS) release, meaning that we will continue to merge bug-fixes and make bug-fix releases for a longer period than usual. Specifically, it will be supported by the community until **December 31, 2019**. On **January 1, 2020** we will raise a toast to Python 2, and community support for the last Python-2-supporting release will come to an end. However, it will continue to be available on PyPI indefinitely, and if any commercial vendors wish to extend the LTS support past this point then we are open to letting them use the LTS branch in the official NumPy repository to coordinate that. If you are a NumPy user who requires ongoing Python 2 support in 2020 or later, then please contact your vendor. If you are a vendor who wishes to continue to support NumPy on Python 2 in 2020+, please get in touch; ideally we'd like you to get involved in maintaining the LTS before it actually hits end-of-life, so we can make a clean handoff. To minimize disruption, running 'pip install numpy' on Python 2 will continue to give the last working release in perpetuity; but after January 1, 2019 it may not contain the latest features, and after January 1, 2020 it may not contain the latest bug fixes. For more information on the scientific Python ecosystem's transition to Python-3-only, see: http://www.python3statement.org/ For more information on porting your code to run on Python 3, see: https://docs.python.org/3/howto/pyporting.html ---- Thoughts? -n On Thu, Nov 9, 2017 at 12:53 PM, Marten van Kerkwijk wrote:

In astropy we had a similar discussion about version numbers, and decided to make 2.0 the LTS that still supports python 2.7 and 3.0 the first that does not. If we're discussing jumping a major number, we could do the same for numpy. (Admittedly, it made a bit more sense with the numbering scheme astropy had adopted anyway.) -- Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

-- Nathaniel J. Smith -- https://vorpus.org

On Fri, Nov 10, 2017 at 2:52 PM, Nathaniel Smith wrote:

Fortunately we can wait until we're a bit closer before we have to make any final decision on the version numbering :-)

Right now though it would be good to start communicating to users/downstreams about whatever our plans our though, so they can make plans. Here's a first attempt at some text we can put in the documentation and point people to -- any thoughts, on either the plan or the wording?

---- DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK ----

The Python core team plans to stop supporting Python 2 in 2020. The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2 support as well. Now that we're entering the final years of community-supported Python 2, the NumPy project wants to clarify our plans, with the goal of to helping our downstream ecosystem make plans and accomplish the transition with as little disruption as possible.

Our current plan is as follows:

Until **December 31, 2018**, all NumPy releases will fully support both Python 2 and Python 3.

Starting on **January 1, 2019**, any new feature releases will support only Python 3.

The last Python-2-supporting release will be designated as a long-term support (LTS) release, meaning that we will continue to merge bug-fixes and make bug-fix releases for a longer period than usual. Specifically, it will be supported by the community until **December 31, 2019**.

On **January 1, 2020** we will raise a toast to Python 2, and community support for the last Python-2-supporting release will come to an end. However, it will continue to be available on PyPI indefinitely, and if any commercial vendors wish to extend the LTS support past this point then we are open to letting them use the LTS branch in the official NumPy repository to coordinate that.

If you are a NumPy user who requires ongoing Python 2 support in 2020 or later, then please contact your vendor. If you are a vendor who wishes to continue to support NumPy on Python 2 in 2020+, please get in touch; ideally we'd like you to get involved in maintaining the LTS before it actually hits end-of-life, so we can make a clean handoff.

To minimize disruption, running 'pip install numpy' on Python 2 will continue to give the last working release in perpetuity; but after January 1, 2019 it may not contain the latest features, and after January 1, 2020 it may not contain the latest bug fixes.

For more information on the scientific Python ecosystem's transition to Python-3-only, see: http://www.python3statement.org/

For more information on porting your code to run on Python 3, see: https://docs.python.org/3/howto/pyporting.html

----

Thoughts?

Thanks for writing that up. Text works for me! Ralf

On Nov 9, 2017 20:52, "Nathaniel Smith" wrote: Fortunately we can wait until we're a bit closer before we have to make any final decision on the version numbering :-) Right now though it would be good to start communicating to users/downstreams about whatever our plans our though, so they can make plans. Here's a first attempt at some text we can put in the documentation and point people to -- any thoughts, on either the plan or the wording? ---- DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK ---- The Python core team plans to stop supporting Python 2 in 2020. The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2 support as well. Now that we're entering the final years of community-supported Python 2, the NumPy project wants to clarify our plans, with the goal of to helping our downstream ecosystem make plans and accomplish the transition with as little disruption as possible. Our current plan is as follows: Until **December 31, 2018**, all NumPy releases will fully support both Python 2 and Python 3. Starting on **January 1, 2019**, any new feature releases will support only Python 3. The last Python-2-supporting release will be designated as a long-term support (LTS) release, meaning that we will continue to merge bug-fixes and make bug-fix releases for a longer period than usual. Specifically, it will be supported by the community until **December 31, 2019**. On **January 1, 2020** we will raise a toast to Python 2, and community support for the last Python-2-supporting release will come to an end. However, it will continue to be available on PyPI indefinitely, and if any commercial vendors wish to extend the LTS support past this point then we are open to letting them use the LTS branch in the official NumPy repository to coordinate that. If you are a NumPy user who requires ongoing Python 2 support in 2020 or later, then please contact your vendor. If you are a vendor who wishes to continue to support NumPy on Python 2 in 2020+, please get in touch; ideally we'd like you to get involved in maintaining the LTS before it actually hits end-of-life, so we can make a clean handoff. To minimize disruption, running 'pip install numpy' on Python 2 will continue to give the last working release in perpetuity; but after January 1, 2019 it may not contain the latest features, and after January 1, 2020 it may not contain the latest bug fixes. For more information on the scientific Python ecosystem's transition to Python-3-only, see: http://www.python3statement.org/ For more information on porting your code to run on Python 3, see: https://docs.python.org/3/howto/pyporting.html ---- Thoughts? -n On Thu, Nov 9, 2017 at 12:53 PM, Marten van Kerkwijk wrote:

In astropy we had a similar discussion about version numbers, and decided to make 2.0 the LTS that still supports python 2.7 and 3.0 the first that does not. If we're discussing jumping a major number, we could do the same for numpy. (Admittedly, it made a bit more sense with the numbering scheme astropy had adopted anyway.) -- Marten _______________________________________________

Might it make sense to do this in a synchronized manner with scipy? So both numpy and scipy drop support for python 2 on the first release after December 31 2018, and numpy's first python3-only release comes before (or simultaneously with) scipy's. Then scipy can set is minimum supported numpy version to be the first python3-only version. That allows scipy to have a clean, obvious point where scipy supports only the latest numpy. This will diverge later, but it seems to be a relatively safe place to bring them back into sync.

Apparently this is actually uncontroversial, the discussion's died down (see also the comments on Chuck's PR [1]), and anyone who wanted to object has had more than a week to do so, so... I guess we can say this is what's happening and start publicizing it to our users! A direct link to the rendered NEP in the repo is: https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-propo... (I guess that at some point it will also show up on docs.scipy.org.) -n [1] https://github.com/numpy/numpy/pull/10006 On Thu, Nov 9, 2017 at 5:52 PM, Nathaniel Smith wrote:

Fortunately we can wait until we're a bit closer before we have to make any final decision on the version numbering :-)

Right now though it would be good to start communicating to users/downstreams about whatever our plans our though, so they can make plans. Here's a first attempt at some text we can put in the documentation and point people to -- any thoughts, on either the plan or the wording?

---- DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK ----

The Python core team plans to stop supporting Python 2 in 2020. The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2 support as well. Now that we're entering the final years of community-supported Python 2, the NumPy project wants to clarify our plans, with the goal of to helping our downstream ecosystem make plans and accomplish the transition with as little disruption as possible.

Our current plan is as follows:

Until **December 31, 2018**, all NumPy releases will fully support both Python 2 and Python 3.

Starting on **January 1, 2019**, any new feature releases will support only Python 3.

The last Python-2-supporting release will be designated as a long-term support (LTS) release, meaning that we will continue to merge bug-fixes and make bug-fix releases for a longer period than usual. Specifically, it will be supported by the community until **December 31, 2019**.

On **January 1, 2020** we will raise a toast to Python 2, and community support for the last Python-2-supporting release will come to an end. However, it will continue to be available on PyPI indefinitely, and if any commercial vendors wish to extend the LTS support past this point then we are open to letting them use the LTS branch in the official NumPy repository to coordinate that.

If you are a NumPy user who requires ongoing Python 2 support in 2020 or later, then please contact your vendor. If you are a vendor who wishes to continue to support NumPy on Python 2 in 2020+, please get in touch; ideally we'd like you to get involved in maintaining the LTS before it actually hits end-of-life, so we can make a clean handoff.

To minimize disruption, running 'pip install numpy' on Python 2 will continue to give the last working release in perpetuity; but after January 1, 2019 it may not contain the latest features, and after January 1, 2020 it may not contain the latest bug fixes.

For more information on the scientific Python ecosystem's transition to Python-3-only, see: http://www.python3statement.org/

For more information on porting your code to run on Python 3, see: https://docs.python.org/3/howto/pyporting.html

----

Thoughts?

-n

On Thu, Nov 9, 2017 at 12:53 PM, Marten van Kerkwijk wrote:

...

In astropy we had a similar discussion about version numbers, and decided to make 2.0 the LTS that still supports python 2.7 and 3.0 the first that does not. If we're discussing jumping a major number, we could do the same for numpy. (Admittedly, it made a bit more sense with the numbering scheme astropy had adopted anyway.) -- Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

-- Nathaniel J. Smith -- https://vorpus.org

-- Nathaniel J. Smith -- https://vorpus.org

I've actually engaged with him on Twitter too but just to repeat one part here : scarce academic resources to maintain code is not an argument. Out of all places, it is academia that should have come up with or should have contributed greatly to open-source instead of paper-writing frenzy among each other. As many people already have written many blog posts/tweets etc. academia does not value software as scientific products but demand software continuously. As an ex-academician I can safely ignore that argument.Scientific code is expected to be maintained properly. I understand the sentiment but blocking progress because of legacy code is a burden on the posterity and a luxury for the past. On Fri, Nov 17, 2017 at 1:35 PM, Peter Cock wrote:

Since Konrad Hinsen no longer follows the NumPy discussion list for lack of time, he has not posted here - but he has commented about this on Twitter and written up a good blog post:

http://blog.khinsen.net/posts/2017/11/16/a-plea-for- stability-in-the-scipy-ecosystem/

In a field where scientific code is expected to last and be developed on a timescale of decades, the change of pace with Python 2 and 3 is harder to handle.

Regards,

Peter

On Wed, Nov 15, 2017 at 2:19 AM, Nathaniel Smith wrote:

...

Apparently this is actually uncontroversial, the discussion's died down (see also the comments on Chuck's PR [1]), and anyone who wanted to object has had more than a week to do so, so... I guess we can say this is what's happening and start publicizing it to our users!

A direct link to the rendered NEP in the repo is: https://github.com/numpy/numpy/blob/master/doc/neps/ dropping-python2.7-proposal.rst

(I guess that at some point it will also show up on docs.scipy.org.)

-n

[1] https://github.com/numpy/numpy/pull/10006

On Thu, Nov 9, 2017 at 5:52 PM, Nathaniel Smith wrote:

...

Fortunately we can wait until we're a bit closer before we have to make any final decision on the version numbering :-)

Right now though it would be good to start communicating to users/downstreams about whatever our plans our though, so they can make plans. Here's a first attempt at some text we can put in the documentation and point people to -- any thoughts, on either the plan or the wording?

---- DRAFT TEXT - NOT FINAL - DO NOT POST THIS TO HACKERNEWS OK? OK ----

The Python core team plans to stop supporting Python 2 in 2020. The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2 support as well. Now that we're entering the final years of community-supported Python 2, the NumPy project wants to clarify our plans, with the goal of to helping our downstream ecosystem make plans and accomplish the transition with as little disruption as possible.

Our current plan is as follows:

Until **December 31, 2018**, all NumPy releases will fully support both Python 2 and Python 3.

Starting on **January 1, 2019**, any new feature releases will support only Python 3.

The last Python-2-supporting release will be designated as a long-term support (LTS) release, meaning that we will continue to merge bug-fixes and make bug-fix releases for a longer period than usual. Specifically, it will be supported by the community until **December 31, 2019**.

On **January 1, 2020** we will raise a toast to Python 2, and community support for the last Python-2-supporting release will come to an end. However, it will continue to be available on PyPI indefinitely, and if any commercial vendors wish to extend the LTS support past this point then we are open to letting them use the LTS branch in the official NumPy repository to coordinate that.

If you are a NumPy user who requires ongoing Python 2 support in 2020 or later, then please contact your vendor. If you are a vendor who wishes to continue to support NumPy on Python 2 in 2020+, please get in touch; ideally we'd like you to get involved in maintaining the LTS before it actually hits end-of-life, so we can make a clean handoff.

To minimize disruption, running 'pip install numpy' on Python 2 will continue to give the last working release in perpetuity; but after January 1, 2019 it may not contain the latest features, and after January 1, 2020 it may not contain the latest bug fixes.

For more information on the scientific Python ecosystem's transition to Python-3-only, see: http://www.python3statement.org/

For more information on porting your code to run on Python 3, see: https://docs.python.org/3/howto/pyporting.html

----

Thoughts?

-n

On Thu, Nov 9, 2017 at 12:53 PM, Marten van Kerkwijk wrote:

...

In astropy we had a similar discussion about version numbers, and decided to make 2.0 the LTS that still supports python 2.7 and 3.0 the first that does not. If we're discussing jumping a major number, we could do the same for numpy. (Admittedly, it made a bit more sense with the numbering scheme astropy had adopted anyway.) -- Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

-- Nathaniel J. Smith -- https://vorpus.org

-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

---

NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

On Fri, Nov 17, 2017 at 4:35 AM, Peter Cock wrote:

Since Konrad Hinsen no longer follows the NumPy discussion list for lack of time, he has not posted here - but he has commented about this on Twitter and written up a good blog post:

http://blog.khinsen.net/posts/2017/11/16/a-plea-for- stability-in-the-scipy-ecosystem/

In a field where scientific code is expected to last and be developed on a timescale of decades, the change of pace with Python 2 and 3 is harder to handle.

sure -- but I do not get what the problem is here! from his post: """ The disappearance of Python 2 will leave much scientific software orphaned, and many published results irreproducible. """ This is an issue we should all be concerned about, and, in fact, the scipy community has been particularly active in the reproducibility realm. BUT: that statement makes NO SENSE. dropping Python2 support in numpy (or any other package) means that newer versions of numpy will not run on py2 -- but if you want to reproduce results, you need to run the code WITH THE VERSION THAT WAS USED IN THE FIRST PLACE. So if someone publishes something based on code written in python2.7 and numpy 1.13, then it is not helpful for reproducibility at all for numpy 1.18 (or 2.*, or whatever we call it) to run on python2. So there is no issue here. Potential issues will arise post 2020, when maybe python2.7 (and numpy 1.13) will no longer run on an up to date OS. But the OS vendors do a pretty good job of backward compatibility -- so we've got quite a few years to go on that. And it will also be important that older versions of packages are available -- but as long as we don't delete the archives, that should be the case for a good long while. So not sure what the problem is here. note relevant for reproducibility,but I have always been puzzled that folks often desperately want to run the very latest numpy on an old Python (2.6, 1.5, ....) if you can update your numy, update your darn Python too! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov