proposal to drop Python 3.6 and NumPy 1.14 / 1.15

Hi all, Python 3.9 is coming out before the next SciPy release, and NEP 29 recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2. Any objections? Cheers, Ralf [1] https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table

+1 On Sat, 15 Aug 2020 at 16:49, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Hi all,
Python 3.9 is coming out before the next SciPy release, and NEP 29 recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2.
Any objections?
Cheers, Ralf
[1] https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

This post is by way of trying to articulate some concerns from the end user side of things. It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html) For Python itself, I find on their releases, the 3.6.12 schedule was 3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9 but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one. Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
If there were a way to provide _only_ actual bug fixes for some versions longer, a la the LTS releases of some Linux distributions, that might be of benefit to users and would reduce the support burden on developers who would not need to worry about fitting new features into the older framework. All of that is not to say that you should not continue with current deprecation plans. Just to be sure that these concerns were voiced at least once. I hope that this perspective is of some use in your considerations. Thanks for all the software, regardless! On Sat, Aug 15, 2020 at 6:48 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Hi all,
Python 3.9 is coming out before the next SciPy release, and NEP 29 recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2.
Any objections?
Cheers, Ralf
[1] https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

On Sun, Aug 16, 2020 at 2:23 PM Bennet Fauber <bennet@umich.edu> wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
If there were a way to provide _only_ actual bug fixes for some versions longer, a la the LTS releases of some Linux distributions, that might be of benefit to users and would reduce the support burden on developers who would not need to worry about fitting new features into the older framework.
All of that is not to say that you should not continue with current deprecation plans. Just to be sure that these concerns were voiced at least once. I hope that this perspective is of some use in your considerations.
Thanks for your thoughts Bennet. All of this was brought up in the NEP 29 discussion which recommended timelines for Python and NumPy version support. I don't think any of it is specific to SciPy. It's a trade-off: users may want perfect stability and updates for whatever version they're on for as long as possible, but that comes at the cost of making maintainers doing tedious work, like more release effort and figuring out why CI fails on some particular combination of old Python and NumPy versions. That tedious work then reduces the amount of time for tackling more relevant bugs and adding new features, and it likely also reduces total effort spent by maintainers over time (most people do this voluntarily, so motivation comes partly from enjoying to contribute). Cheers, Ralf
Thanks for all the software, regardless!
On Sat, Aug 15, 2020 at 6:48 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Hi all,
Python 3.9 is coming out before the next SciPy release, and NEP 29
recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2.
Any objections?
Cheers, Ralf
[1]
https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Thanks, Ralf. Yes, I get that there are good reasons for wanting it both ways and that is not possible. That's why I asked whether something like an LTS had ever been considered, which might make it possible for both sides to be happier. In the end, if development outpaces users's ability to update, they will keep using old versions and SciPy/NumPy will move on without those people until they can update. Whatever you choose, thanks for all the hard work you folks put into this. Computing is a better place for it. On Sun, Aug 16, 2020 at 11:03 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Sun, Aug 16, 2020 at 2:23 PM Bennet Fauber <bennet@umich.edu> wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
If there were a way to provide _only_ actual bug fixes for some versions longer, a la the LTS releases of some Linux distributions, that might be of benefit to users and would reduce the support burden on developers who would not need to worry about fitting new features into the older framework.
All of that is not to say that you should not continue with current deprecation plans. Just to be sure that these concerns were voiced at least once. I hope that this perspective is of some use in your considerations.
Thanks for your thoughts Bennet. All of this was brought up in the NEP 29 discussion which recommended timelines for Python and NumPy version support. I don't think any of it is specific to SciPy. It's a trade-off: users may want perfect stability and updates for whatever version they're on for as long as possible, but that comes at the cost of making maintainers doing tedious work, like more release effort and figuring out why CI fails on some particular combination of old Python and NumPy versions. That tedious work then reduces the amount of time for tackling more relevant bugs and adding new features, and it likely also reduces total effort spent by maintainers over time (most people do this voluntarily, so motivation comes partly from enjoying to contribute).
Cheers, Ralf
Thanks for all the software, regardless!
On Sat, Aug 15, 2020 at 6:48 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Hi all,
Python 3.9 is coming out before the next SciPy release, and NEP 29 recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2.
Any objections?
Cheers, Ralf
[1] https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind: - developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees - freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science - offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder. Matti [1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence

No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a suggestion to consider holding it back. I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others). - for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6? - for the RM and other packagers: how much of a hassle is it to keep an older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now? Cheers, Evgeni On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com> wrote:
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be
used when building
with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

I have too many personas in my head arguing about this. And many of them are in conflict. 1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump. On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a suggestion to consider holding it back. I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others). - for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6? - for the RM and other packagers: how much of a hassle is it to keep an older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now?
Cheers, Evgeni
On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com> wrote:
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be
used when building
with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Boy, yes, Ilhan, I agree. Too many different roles in my head! Just to be clear, I also am not concerned about deprecating older NumPy, only the base Python version, and specifically 3.6. It looks like the latest fully released NumPy supports Python 3.6. In my Linux world, we still have a preponderance of RedHat/CentOS 7 installed, and it does not easily provide Python 3.7: python36 is available from EPEL. Our most prevalent Ubuntu is 16.04. We will probably jump to 20.04 next calendar year. I had not realized that Python itself was going to a 12-month release cycle. That seems to me destructive and predicated on the notion that software is disposable. I think there is a very good reason why many Linux distributions have separate distributions for longer term stability and more up-to-date features. Red Hat and Fedora, Ubuntu and LTS, Debian Stable and Unstable. Is that viable with SciPy, NumPy, where only the bug fixes get backported, which might reduce the cost of longer maintenance (except for the wheels, as noted)? Thanks for the discussion on this, no matter where it ends up. On Sun, Aug 16, 2020 at 7:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a suggestion to consider holding it back. I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others). - for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6? - for the RM and other packagers: how much of a hassle is it to keep an older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now?
Cheers, Evgeni
On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com> wrote:
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

FWIW, the policy for all minor point releases is already that we only backport bug fixes and not enhancements, so there's not much more we can trim on that side of things. We've done LTS releases before with i.e., the Python 2.x support series. I can pretty much guarantee that we will take on a non-zero maintenance burden if we support an additional Python version. Devs spend a fair bit of time just "keeping CI green" so that new contributions/fixes can flow in without confusion/delays, and that is compounded by the backport CI and separate wheels repo CI. If you convince Ralf et al. that we should go against the NEP I don't really mind, but it is going to cost us time that could be spent on other things. On Mon, 17 Aug 2020 at 06:59, Bennet Fauber <bennet@umich.edu> wrote:
Boy, yes, Ilhan, I agree. Too many different roles in my head!
Just to be clear, I also am not concerned about deprecating older NumPy, only the base Python version, and specifically 3.6. It looks like the latest fully released NumPy supports Python 3.6. In my Linux world, we still have a preponderance of RedHat/CentOS 7 installed, and it does not easily provide Python 3.7: python36 is available from EPEL. Our most prevalent Ubuntu is 16.04. We will probably jump to 20.04 next calendar year.
I had not realized that Python itself was going to a 12-month release cycle. That seems to me destructive and predicated on the notion that software is disposable.
I think there is a very good reason why many Linux distributions have separate distributions for longer term stability and more up-to-date features. Red Hat and Fedora, Ubuntu and LTS, Debian Stable and Unstable. Is that viable with SciPy, NumPy, where only the bug fixes get backported, which might reduce the cost of longer maintenance (except for the wheels, as noted)?
Thanks for the discussion on this, no matter where it ends up.
On Sun, Aug 16, 2020 at 7:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them
are in conflict.
1- I think I had enough arguments online to be known as buying none of
2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now
6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use
that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) they want 12 months. (What Ralf said) pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <
No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a
suggestion to consider holding it back.
I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of
- for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6? - for the RM and other packagers: how much of a hassle is it to keep an
Cheers, Evgeni
On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com>
wrote:
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the
end
user side of things.
It looks like NumPy 19.1 supports Python 3.6.
This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain
section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user
a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as
and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using
evgeny.burovskiy@gmail.com> wrote: things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others). older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now? the perspective, proficient things
like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Thanks for the report 'from the trenches', Tyler. It's been thouroughly discussed, which is all I could ask. I appreciate the time everyone has taken. This gives us some food for thought on our side, as well. Time to start that with my colleagues here. Thanks, again, for the great software. On Mon, Aug 17, 2020 at 10:58 PM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
FWIW, the policy for all minor point releases is already that we only backport bug fixes and not enhancements, so there's not much more we can trim on that side of things.
We've done LTS releases before with i.e., the Python 2.x support series.
I can pretty much guarantee that we will take on a non-zero maintenance burden if we support an additional Python version. Devs spend a fair bit of time just "keeping CI green" so that new contributions/fixes can flow in without confusion/delays, and that is compounded by the backport CI and separate wheels repo CI.
If you convince Ralf et al. that we should go against the NEP I don't really mind, but it is going to cost us time that could be spent on other things.
On Mon, 17 Aug 2020 at 06:59, Bennet Fauber <bennet@umich.edu> wrote:
Boy, yes, Ilhan, I agree. Too many different roles in my head!
Just to be clear, I also am not concerned about deprecating older NumPy, only the base Python version, and specifically 3.6. It looks like the latest fully released NumPy supports Python 3.6. In my Linux world, we still have a preponderance of RedHat/CentOS 7 installed, and it does not easily provide Python 3.7: python36 is available from EPEL. Our most prevalent Ubuntu is 16.04. We will probably jump to 20.04 next calendar year.
I had not realized that Python itself was going to a 12-month release cycle. That seems to me destructive and predicated on the notion that software is disposable.
I think there is a very good reason why many Linux distributions have separate distributions for longer term stability and more up-to-date features. Red Hat and Fedora, Ubuntu and LTS, Debian Stable and Unstable. Is that viable with SciPy, NumPy, where only the bug fixes get backported, which might reduce the cost of longer maintenance (except for the wheels, as noted)?
Thanks for the discussion on this, no matter where it ends up.
On Sun, Aug 16, 2020 at 7:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a suggestion to consider holding it back. I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others). - for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6? - for the RM and other packagers: how much of a hassle is it to keep an older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now?
Cheers, Evgeni
On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com> wrote:
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the end user side of things.
It looks like NumPy 19.1 supports Python 3.6.
> This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building > with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain the section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user perspective, a better date for deprecation and not too far from the just proposed one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as proficient and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

On Tue, Aug 18, 2020 at 3:58 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
I can pretty much guarantee that we will take on a non-zero maintenance burden if we support an additional Python version. Devs spend a fair bit of time just "keeping CI green" so that new contributions/fixes can flow in without confusion/delays, and that is compounded by the backport CI and separate wheels repo CI.
Yep, this really is the main argument.
If you convince Ralf et al. that we should go against the NEP I don't really mind, but it is going to cost us time that could be spent on other things.
On Mon, 17 Aug 2020 at 06:59, Bennet Fauber <bennet@umich.edu> wrote:
Boy, yes, Ilhan, I agree. Too many different roles in my head!
Just to be clear, I also am not concerned about deprecating older NumPy, only the base Python version, and specifically 3.6. It looks like the latest fully released NumPy supports Python 3.6. In my Linux world, we still have a preponderance of RedHat/CentOS 7 installed, and it does not easily provide Python 3.7: python36 is available from EPEL. Our most prevalent Ubuntu is 16.04. We will probably jump to 20.04 next calendar year.
`pip install scipy --user`, or whatever you do on an old CentOS machine/cluster, is still going to work. As is `apt-get install scipy`. And it will get you the latest supported SciPy for that Python version, which is pretty good. All you're missing out on is a few new features and bug fixes in 1.6.0. If you discover you need those, it's time to learn Conda so it's trivial to install a newer Python version.
I had not realized that Python itself was going to a 12-month release cycle. That seems to me destructive and predicated on the notion that software is disposable.
I have to admit I'm not a fan, but that's because new Python versions typically bring us new burdens and few or no new features we need. Having bug fixes propagate a bit faster is the upside.
I think there is a very good reason why many Linux distributions have separate distributions for longer term stability and more up-to-date features. Red Hat and Fedora, Ubuntu and LTS, Debian Stable and Unstable. Is that viable with SciPy, NumPy, where only the bug fixes get backported, which might reduce the cost of longer maintenance (except for the wheels, as noted)?
Thanks for the discussion on this, no matter where it ends up.
On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <
evgeny.burovskiy@gmail.com> wrote:
No objections for bumping the minimum NumPy version. For bumping the python version.. not quite an objection, more of a
suggestion to consider holding it back.
I guess there are three kinds of stakeholders here: - for end users, upgrading the python version is a fairly major hassle. Three versions of python now means three years back if they switch to a 12-months cadence. Three years of support is definitely on a low side of things. (with my PI hat on, I can certainly relate to what Bennet was saying, even if our lab has it way easier than others).
Python 3.6 was released in Dec 2016, and we're talking about our Dec 2020 release - so 4 years. When it's actually down to 3 years we can review again, but that's not today's decision.
- for the scipy devs, what exactly drives us to drop it? For 3.5 it was of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7 which makes it not fun to stay on 3.6?
f-strings are/were not very interesting imho. If you spend all day doing string processing yes, but for numerical code it really doesn't matter all that much. We had two ways of doing string formatting, now we have three. As Tyler said, it's to reduce the cost of keeping it working, and CI happy. We can spend that time writing or reviewing new features and other improvements instead. Cheers, Ralf
- for the RM and other packagers: how much of a hassle is it to keep an older version? I realize it's more wheels to build for the scipy RM, but that's fairly automated now?
Cheers, Evgeni
On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus@gmail.com>
On 8/16/20 4:23 PM, Bennet Fauber wrote:
This post is by way of trying to articulate some concerns from the
end
user side of things.
It looks like NumPy 19.1 supports Python 3.6.
> This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to be used when building > with Python 3.9 for testing purposes. https://numpy.org/devdocs/release/1.19.1-notes.html
The section of the release notes for NumPy 1.20.0 does not contain
section at the top saying which versions of Python are supported. (https://numpy.org/devdocs/release/1.20.0-notes.html)
For Python itself, I find on their releases, the 3.6.12 schedule was
3.6.12 final: 2020-08-14 (expected) https://www.python.org/dev/peps/pep-0494/#id9
but does not seem to have made it, and Python 3.6.11 was released June 27, 2020. The Python plans for 3.6.13 and beyond are Security fixes only, as needed, until 2021-12. That seems, from my user
a better date for deprecation and not too far from the just
one.
Acknowledging the additional burden of 'supporting' older versions of Python, still it would seem that matching NumPy is not a bad thing.
From a strictly consumer perspective, where much of the work is getting all of the non-NumPy and non-SciPy functionality to work and be stable, upgrading Python can be very disruptive. Time spent getting the 'glue' around analytics to work is time we cannot spend on science. There are large projects that do keep up, but they tend also to be funded well. The many small labs on my campus do not have funding for software development, are unlikely to ever get any, so any work required to fix software because of updates comes from their budget for science and from their scientists who are not as
and therefore not as efficient at adapting to a newer version of Python and all the reinstallation of libraries that attends it.
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using
wrote: the perspective, proposed proficient things
like Jupyter Hub or other web-based services
FWIW, python is changing their release cadence from once every 18 months to once every 12 months[1] so it is likely this problem will get harder.
Matti
[1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note that SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability. <snip> Chuck

Sorry for the top post, but this question doesn't follow immediately from any previous comment. Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support? It might be useful to use dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7. Warren On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note that SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck

On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < warren.weckesser@gmail.com> wrote:
Sorry for the top post, but this question doesn't follow immediately from any previous comment.
Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support?
I think so. Given that there were a few hesitations at least, I'll split it in separate PRs - will do the NumPy one first. It might be useful to use
dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7.
You want to start using them now, or is it more a "nice to have the option"? Cheers, Ralf
Warren
On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of them are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note that SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Time to follow up here I think. At the moment, our wheels repo weekly cron job for POSIX systems includes multiple Python 3.6 matrix entries and they all build/pass tests: https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636 This means a few things: 1) Obviously, we have not yet turned off support for Python 3.6 in the master branch yet 2) At the moment, Python 3.6 does not appear to be causing problems Of course, just because there are no problems with 3.6 now does not mean that there won't be problems when backporting later as master branch evolves. Is the final decision here to turn off 3.6 support before we branch for 1.6.x series? Best wishes, Tyler On Fri, 11 Sep 2020 at 15:42, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < warren.weckesser@gmail.com> wrote:
Sorry for the top post, but this question doesn't follow immediately from any previous comment.
Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support?
I think so. Given that there were a few hesitations at least, I'll split it in separate PRs - will do the NumPy one first.
It might be useful to use
dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7.
You want to start using them now, or is it more a "nice to have the option"?
Cheers, Ralf
Warren
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of
On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote: them
are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why people take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about the Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note that SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

My thoughts would be to drop 3.6 for 1.6, but obviously we need to retain for 1.5.X. At the moment we're supporting 4 Python releases. It would make CI burden a bit less.

On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Time to follow up here I think. At the moment, our wheels repo weekly cron job for POSIX systems includes multiple Python 3.6 matrix entries and they all build/pass tests: https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636
This means a few things: 1) Obviously, we have not yet turned off support for Python 3.6 in the master branch yet 2) At the moment, Python 3.6 does not appear to be causing problems
Of course, just because there are no problems with 3.6 now does not mean that there won't be problems when backporting later as master branch evolves.
Is the final decision here to turn off 3.6 support before we branch for 1.6.x series?
Yes, we should. Timing of this conversation was a bit unfortunate for me, so I didn't get around to opening a PR in September. I can do so tomorrow. Cheers, Ralf
Best wishes, Tyler
On Fri, 11 Sep 2020 at 15:42, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < warren.weckesser@gmail.com> wrote:
Sorry for the top post, but this question doesn't follow immediately from any previous comment.
Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support?
I think so. Given that there were a few hesitations at least, I'll split it in separate PRs - will do the NumPy one first.
It might be useful to use
dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7.
You want to start using them now, or is it more a "nice to have the option"?
Cheers, Ralf
Warren
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of
are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why
take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about
Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when they use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note
On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote: them people the that
SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Hi Ralf, I opened one yesterday: https://github.com/scipy/scipy/pull/13081 Feel free to replace it though. It was based almost purely on the commit where you did the same thing for 3.5 last time. Tyler On Sat, 14 Nov 2020 at 15:39, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Time to follow up here I think. At the moment, our wheels repo weekly cron job for POSIX systems includes multiple Python 3.6 matrix entries and they all build/pass tests: https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636
This means a few things: 1) Obviously, we have not yet turned off support for Python 3.6 in the master branch yet 2) At the moment, Python 3.6 does not appear to be causing problems
Of course, just because there are no problems with 3.6 now does not mean that there won't be problems when backporting later as master branch evolves.
Is the final decision here to turn off 3.6 support before we branch for 1.6.x series?
Yes, we should. Timing of this conversation was a bit unfortunate for me, so I didn't get around to opening a PR in September. I can do so tomorrow.
Cheers, Ralf
Best wishes, Tyler
On Fri, 11 Sep 2020 at 15:42, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < warren.weckesser@gmail.com> wrote:
Sorry for the top post, but this question doesn't follow immediately from any previous comment.
Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support?
I think so. Given that there were a few hesitations at least, I'll split it in separate PRs - will do the NumPy one first.
It might be useful to use
dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7.
You want to start using them now, or is it more a "nice to have the option"?
Cheers, Ralf
Warren
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I have too many personas in my head arguing about this. And many of
are in conflict.
1- I think I had enough arguments online to be known as buying none of that scientific stability views. That doesn't mean I don't get their point, I really really do, but still. It's in the job description. Software is a big part of science just like the papers. (What Matti said) 2- Even a two-year old code is not safe to let it collect dust and constantly requires attention and becomes a liability rather than a tool (what Bennet said). 3- Matlab breaks things all the time with 0 user consideration. None. But admittedly they break in style. We shouldn't be like them (not even close) but the point is when it is a commercial product, we don't hear the uproar we receive. People silently fix things. 4- Just because we update the python version, a lot of packages stop working. This is seriously disheartening. Happens to me all the time (protobuf, influxdb etc). And really annoying if one package has not released the new version yet. 5- Python is releasing too quick. I know Python is not Fortran (thankfully) but this is the other end of the spectrum. With every version one or two of us spent at least 2 weeks of intensive "What did they change on Windows again?" bug hunting. Hence our platform is not reliable and now they want 12 months. (What Ralf said) 6- There are always new users, downloading Python for the first time and it is expected that SciPy stack should work off-the-shelf. Hence we don't have the luxury to wait and see (see 4th item). 7- If there are limited resources for software support, then why
take the risk and update their stuff is something that has been discussed for decades now. And no one seems to converge to a point. (What Matti said) 8- what Evgeni suggested
I'm not sure what to do about this but I am also more worried about
Python version than the NumPy version, my limited anectodal evidence around me suggests that people update NumPy + SciPy together mainly when
use pip with -U (--upgrade) on some package that has SciPy as a dependency. So I feel that it is safe to bump.
Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note
On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote: them people the they that
SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

On Sat, Nov 14, 2020 at 10:47 PM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Hi Ralf,
I opened one yesterday: https://github.com/scipy/scipy/pull/13081
Feel free to replace it though. It was based almost purely on the commit where you did the same thing for 3.5 last time.
Sorry, I've got my notifications disabled. Ignore the noise, your PR looks good! Cheers, Ralf
Tyler
On Sat, 14 Nov 2020 at 15:39, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Time to follow up here I think. At the moment, our wheels repo weekly cron job for POSIX systems includes multiple Python 3.6 matrix entries and they all build/pass tests: https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636
This means a few things: 1) Obviously, we have not yet turned off support for Python 3.6 in the master branch yet 2) At the moment, Python 3.6 does not appear to be causing problems
Of course, just because there are no problems with 3.6 now does not mean that there won't be problems when backporting later as master branch evolves.
Is the final decision here to turn off 3.6 support before we branch for 1.6.x series?
Yes, we should. Timing of this conversation was a bit unfortunate for me, so I didn't get around to opening a PR in September. I can do so tomorrow.
Cheers, Ralf
Best wishes, Tyler
On Fri, 11 Sep 2020 at 15:42, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < warren.weckesser@gmail.com> wrote:
Sorry for the top post, but this question doesn't follow immediately from any previous comment.
Is the conclusion of this thread that with SciPy 1.6 we will definitely drop Python 3.6 support?
I think so. Given that there were a few hesitations at least, I'll split it in separate PRs - will do the NumPy one first.
It might be useful to use
dataclasses (https://docs.python.org/3/library/dataclasses.html), and they are new in Python 3.7.
You want to start using them now, or is it more a "nice to have the option"?
Cheers, Ralf
Warren
On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
> I have too many personas in my head arguing about this. And many of
> are in conflict. > > 1- I think I had enough arguments online to be known as buying none of > that scientific stability views. That doesn't mean I don't get their > point, > I really really do, but still. It's in the job description. Software is a > big part of science just like the papers. (What Matti said) > 2- Even a two-year old code is not safe to let it collect dust and > constantly requires attention and becomes a liability rather than a tool > (what Bennet said). > 3- Matlab breaks things all the time with 0 user consideration. None. But > admittedly they break in style. We shouldn't be like them (not even > close) > but the point is when it is a commercial product, we don't hear the > uproar > we receive. People silently fix things. > 4- Just because we update the python version, a lot of packages stop > working. This is seriously disheartening. Happens to me all the time > (protobuf, influxdb etc). And really annoying if one package has not > released the new version yet. > 5- Python is releasing too quick. I know Python is not Fortran > (thankfully) but this is the other end of the spectrum. With every > version > one or two of us spent at least 2 weeks of intensive "What did they > change > on Windows again?" bug hunting. Hence our platform is not reliable and > now > they want 12 months. (What Ralf said) > 6- There are always new users, downloading Python for the first time and > it is expected that SciPy stack should work off-the-shelf. Hence we don't > have the luxury to wait and see (see 4th item). > 7- If there are limited resources for software support, then why
> take the risk and update their stuff is something that has been discussed > for decades now. And no one seems to converge to a point. (What Matti > said) > 8- what Evgeni suggested > > I'm not sure what to do about this but I am also more worried about
> Python version than the NumPy version, my limited anectodal evidence > around > me suggests that people update NumPy + SciPy together mainly when
> use > pip with -U (--upgrade) on some package that has SciPy as a dependency. > So > I feel that it is safe to bump. > > Just for the NumPy perspective, there is nothing in the forthcoming NumPy 1.20 that doesn't work with Python 3.6, it is just a question of what wheels to provide. OTOH, as Python advances I don't want to worry about someone using newer features, whatever they are. I go back and forth, but feel that 48 months is about the right support window and that argues for dropping Python 3.6. With the faster pace of Python development we will probably want to support the last four versions going forward. Note
On 8/20/20, Charles R Harris <charlesr.harris@gmail.com> wrote: them people the they that
SciPy doesn't need to support all the NumPy versions that support a particular version of Python, just the latest one. The main concern doesn't seem to be compatibility as much as availability.
<snip>
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev

Hi Matti, (am changing the subject line to separate a subtopic).
Is there a forum where we could explore the larger question around best practices in helping projects and teams keep their software up-to-date through rolling updates, test suites, and documentation? A few things come to mind:
- developing a culture of version control and tests when creating software, even if it is grad students racing towards their degrees
- freezing environments in images, which is a whole discipline unto itself, and needed for reproducible science
- offloading the task of maintaining computing environments using things like Jupyter Hub or other web-based services
t's a great point, thanks for starting the conversation! I've asked around (mostly undergrad science students, not comp sci). A common theme is that - it'd be great to have some short best practices how-tos would be helpful, especially for testing, version control and containerization - there's lots of info, and it's overwhelming (hence a short how-to to get started) - the vast majority of learning material on the internet seems off-base and is difficult to connect to the R&D / research projects students are dealing with. Anecdotally, when I was starting (non comp sci background here), one of the eye openers for me was Nelle V's git tutorial at EuroScipy. There are several tutorials from the SciPy community (broadly defined), but they are scattered and not very easy to find (e.g. Matthew Brett's https://matthew-brett.github.io/pydagogue, I'm sure there are others) Just throwing it out there: I wonder if SciPy lecture notes (https://scipy-lectures.org/) could be a place for these sorts of tutorials. Cheers, Evgeni

Hi Evgeni, On Fri, Aug 21, 2020, at 04:12, Evgeni Burovski wrote:
There are several tutorials from the SciPy community (broadly defined), but they are scattered and not very easy to find (e.g. Matthew Brett's https://matthew-brett.github.io/pydagogue, I'm sure there are others)
Just throwing it out there: I wonder if SciPy lecture notes (https://scipy-lectures.org/) could be a place for these sorts of tutorials.
It's true, these are spread all over the place, and it would be good to reconcile. We've also started writing contributor manuals for some projects. See, e.g.,: https://scikit-image.org/docs/stable/core_developer.html (for core developers) and https://skyportal.io/docs/contributing.html for contributing developers. And, of course, SciPy's very extensive https://docs.scipy.org/doc/scipy/reference/dev/contributor/contributor_toc.h... Topics handled includes scope and size of PRs, handling merge conflicts, guidance on how to do a good review, etc. So, if we could figure out which topics are general and relevant to most contributors, we could add those to a central set of documentation that several projects can refer back to (I think that was also the idea behind Matthew Brett's https://github.com/matthew-brett/gitwash). Best regards, Stéfan
participants (10)
-
Andrew Nelson
-
Bennet Fauber
-
Charles R Harris
-
Evgeni Burovski
-
Ilhan Polat
-
Matti Picus
-
Ralf Gommers
-
Stefan van der Walt
-
Tyler Reddy
-
Warren Weckesser