[SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15

Mon Aug 17 22:58:30 EDT 2020

FWIW, the policy for all minor point releases is already that we only
backport bug fixes and not enhancements, so there's not much more we can
trim on that side of things.

We've done LTS releases before with i.e., the Python 2.x support series.

I can pretty much guarantee that we will take on a non-zero maintenance
burden if we support an additional Python version. Devs spend a fair bit of
time just "keeping CI green" so that new contributions/fixes can flow in
without confusion/delays, and that is compounded by the backport CI and
separate wheels repo CI.

If you convince Ralf et al. that we should go against the NEP I don't
really mind, but it is going to cost us time that could be spent on other
things.

On Mon, 17 Aug 2020 at 06:59, Bennet Fauber <bennet at umich.edu> wrote:

> Boy, yes, Ilhan, I agree.  Too many different roles in my head!
>
> Just to be clear, I also am not concerned about deprecating older
> NumPy, only the base Python version, and specifically 3.6.  It looks
> like the latest fully released NumPy supports Python 3.6.  In my Linux
> world, we still have a preponderance of RedHat/CentOS 7 installed, and
> it does not easily provide Python 3.7:  python36 is available from
> EPEL.  Our most prevalent Ubuntu is 16.04.  We will probably jump to
> 20.04 next calendar year.
>
> I had not realized that Python itself was going to a 12-month release
> cycle.  That seems to me destructive and predicated on the notion that
> software is disposable.
>
> I think there is a very good reason why many Linux distributions have
> separate distributions for longer term stability and more up-to-date
> features.  Red Hat and Fedora, Ubuntu and LTS, Debian Stable and
> Unstable.  Is that viable with SciPy, NumPy, where only the bug fixes
> get backported, which might reduce the cost of longer maintenance
> (except for the wheels, as noted)?
>
> Thanks for the discussion on this, no matter where it ends up.
>
>
>
> On Sun, Aug 16, 2020 at 7:17 PM Ilhan Polat <ilhanpolat at gmail.com> wrote:
> >
> > I have too many personas in my head arguing about this. And many of them
> are in conflict.
> >
> > 1- I think I had enough arguments online to be known as buying none of
> that scientific stability views. That doesn't mean I don't get their point,
> I really really do, but still. It's in the job description. Software is a
> big part of science just like the papers. (What Matti said)
> > 2- Even a two-year old code is not safe to let it collect dust and
> constantly requires attention and becomes a liability rather than a tool
> (what Bennet said).
> > 3- Matlab breaks things all the time with 0 user consideration. None.
> But admittedly they break in style. We shouldn't be like them (not even
> close) but the point is when it is a commercial product, we don't hear the
> uproar we receive. People silently fix things.
> > 4- Just because we update the python version, a lot of packages stop
> working. This is seriously disheartening. Happens to me all the time
> (protobuf, influxdb etc). And really annoying if one package has not
> released the new version yet.
> > 5- Python is releasing too quick. I know Python is not Fortran
> (thankfully) but this is the other end of the spectrum. With every version
> one or two of us spent at least 2 weeks of intensive "What did they change
> on Windows again?" bug hunting. Hence our platform is not reliable and now
> they want  12 months.  (What Ralf said)
> > 6- There are always new users, downloading Python for the first time and
> it is expected that SciPy stack should work off-the-shelf. Hence we don't
> have the luxury to wait and see (see 4th item).
> > 7- If there are limited resources for software support, then why people
> take the risk and update their stuff is something that has been discussed
> for decades now. And no one seems to converge to a point. (What Matti said)
> > 8-  what Evgeni suggested
> >
> > I'm not sure what to do about this but I am also more worried about the
> Python version than the NumPy version, my limited anectodal evidence around
> me suggests that people update NumPy + SciPy together mainly when they use
> pip with -U (--upgrade) on some package that has SciPy as a dependency. So
> I feel that it is safe to bump.
> >
> >
> >
> >
> >
> > On Mon, Aug 17, 2020 at 12:19 AM Evgeni Burovski <
> evgeny.burovskiy at gmail.com> wrote:
> >>
> >> No objections for bumping the minimum NumPy version.
> >> For bumping the python version.. not quite an objection, more of a
> suggestion to consider holding it back.
> >> I guess there are three kinds of stakeholders here:
> >> - for end users, upgrading the python version is a fairly major hassle.
> Three versions of python now means three years back if they switch to a
> 12-months cadence. Three years of support is definitely on a low side of
> things. (with my PI hat on, I can certainly relate to what Bennet was
> saying, even if our lab has it way easier than others).
> >> - for the scipy devs, what exactly drives us to drop it? For 3.5 it was
> of course the matmul operator, for 3.6 it's f-strings, what's there in 3.7
> which makes it not fun to stay on 3.6?
> >> - for the RM and other packagers: how much of a hassle is it to keep an
> older version? I realize it's more wheels to build for the scipy RM, but
> that's fairly automated now?
> >>
> >>
> >>
> >> Cheers,
> >> Evgeni
> >>
> >>
> >> On Sun, Aug 16, 2020 at 6:22 PM Matti Picus <matti.picus at gmail.com>
> wrote:
> >> >
> >> >
> >> > On 8/16/20 4:23 PM, Bennet Fauber wrote:
> >> > > This post is by way of trying to articulate some concerns from the
> end
> >> > > user side of things.
> >> > >
> >> > > It looks like NumPy 19.1 supports Python 3.6.
> >> > >
> >> > >> This release supports Python 3.6-3.8. Cython >= 0.29.21 needs to
> be used when building
> >> > >> with Python 3.9 for testing purposes.
> >> > > https://numpy.org/devdocs/release/1.19.1-notes.html
> >> > >
> >> > > The section of the release notes for NumPy 1.20.0 does not contain
> the
> >> > > section at the top saying which versions of Python are supported.
> >> > > (https://numpy.org/devdocs/release/1.20.0-notes.html)
> >> > >
> >> > > For Python itself,  I find on their releases, the 3.6.12 schedule
> was
> >> > >
> >> > >      3.6.12 final: 2020-08-14 (expected)
> >> > > https://www.python.org/dev/peps/pep-0494/#id9
> >> > >
> >> > > but does not seem to have made it, and Python 3.6.11 was released
> June
> >> > > 27, 2020.  The Python plans for 3.6.13 and beyond are Security fixes
> >> > > only, as needed, until 2021-12.  That seems, from my user
> perspective,
> >> > > a better date for deprecation and not too far from the just proposed
> >> > > one.
> >> > >
> >> > > Acknowledging the additional burden of 'supporting' older versions
> of
> >> > > Python, still it would seem that matching NumPy is not a bad thing.
> >> > >
> >> > >  From a strictly consumer perspective, where much of the work is
> >> > > getting all of the non-NumPy and non-SciPy functionality to work and
> >> > > be stable, upgrading Python can be very disruptive.  Time spent
> >> > > getting the 'glue' around analytics to work is time we cannot spend
> on
> >> > > science.  There are large projects that do keep up, but they tend
> also
> >> > > to be funded well.  The many small labs on my campus do not have
> >> > > funding for software development, are unlikely to ever get any, so
> any
> >> > > work required to fix software because of updates comes from their
> >> > > budget for science and from their scientists who are not as
> proficient
> >> > > and therefore not as efficient at adapting to a newer version of
> >> > > Python and all the reinstallation of libraries that attends it.
> >> > >
> >> >
> >> > Is there a forum where we could explore the larger question around
> best
> >> > practices in helping projects and teams keep their software up-to-date
> >> > through rolling updates, test suites, and documentation? A few things
> >> > come to mind:
> >> >
> >> > - developing a culture of version control and tests when creating
> >> > software, even if it is grad students racing towards their degrees
> >> >
> >> > - freezing environments in images, which is a whole discipline unto
> >> > itself, and needed for reproducible science
> >> >
> >> > - offloading the task of maintaining computing environments using
> things
> >> > like Jupyter Hub or other web-based services
> >> >
> >> >
> >> > FWIW, python is changing their release cadence from once every 18
> months
> >> > to once every 12 months[1] so it is likely this problem will get
> harder.
> >> >
> >> > Matti
> >> >
> >> > [1] https://www.python.org/dev/peps/pep-0602/#annual-release-cadence
> >> > _______________________________________________
> >> > SciPy-Dev mailing list
> >> > SciPy-Dev at python.org
> >> > https://mail.python.org/mailman/listinfo/scipy-dev
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200817/59b4f3e6/attachment-0001.html>