RFC: a proposal to deprecate interpolate.interp2d
Hi, TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` I believe it has both interface and structural problems which cannot be fixed in a satisfactory backwards compatible way; all its functionality is available otherwise via equivalent and more predictable routines (*BivariateSpline and bisplrep); for interpolation specifically we now have better behaved routines (interpn and griddata). Cons for deprecation: - The routine is a direct analog of the MATLAB routine of the same name. - It has been there "forever". - It is probably widely used. This list of reasons make it sound like there is no way we can deprecate it; please read on however. Pros for deprecation: 1. The internal implementation of `interp2d` is not really meant for interpolation, and is known to produce wrong answers in some cases. 2. If it's gone, no functionality is missing from scipy.interpolate: it is a third equivalent way of wrapping FITPACK routines, along with bisplrep/bisplev and *BivariateSpline classes. 3. We now have better, more controllable, more predictable, and easier-to-understand routines for almost all interp2d functionality. 4. `interp2d` interface is confusing to users and error-prone and cannot be fixed while keeping backwards compatibility 5. Being a MATLAB name clone, it naturally attracts new users, who would otherwise use better, more suitable or more predictable routines. Interp2d has at least five open issues: see the FITPACK column of if it's gone, no functionality is missing from scipy.interpolate: it is a third equivalent way of wrapping FITPACK routines, along with bisplrep and *BivariateSpline classes. `interp2d` has at least five open issues, see e.g. the FITPACK column of https://github.com/orgs/scipy/projects/1. I'll now expand on the "pros" points above and try to summarize the open issues. 1. `interp2d` is a wrapper around the FITPACK spline-fitting routines. These are mainly targeted for spline *smoothing*, i.e. fitting a smoothing spline to noisy data. They do so by automagically selecting the spline knots to minimize a certain quality metric which tells how close the fit is to the data. There is a single control knob, the "smoothing parameter s", which can be set to zero to require the interpolation conditions, which influences the knot selection. When this knot construction procedure fails --- and it's most prone to failing for linear interpolation, which is the default for interp2d --- it returns a wrong result, either silently or with an extremely obscure warning. For details on this, see the discussion https://github.com/scipy/scipy/issues/2167. While that issue is about `BivariateSpline`s, interp2d has exactly the same set of problems and no way for a user to control the result. Also note https://github.com/scipy/scipy/issues/11309 , which shows essentially the same problem, only compounded by peculiarities of the interp2d interface. In passing, note a helpful code comment, "# TODO: surfit is really not meant for interpolation!", https://github.com/scipy/scipy/blob/main/scipy/interpolate/_interpolate.py#L..., which git blame attributes to commit 7adfbbda57e from 2012 2. We ship at least three ways of wrapping 2D FITPACK routines: the functional interface, `bisplrep / bisplev`, the OOP interface `*BivariateSpline`s and interp2d. The main difference is the set of defaults--- which adds to user confusion, and that interp2d switches between essentially SmoothBivariateSpline and RectBivariateSpline, depending on the shapes of the input arrays. All in all, exactly zero functionality is missing from scipy.interpolate if interp2d is gone. 3. One confusing aspect of the FITPACK procedure is that there is no guarantee that the spline knots coincide with the data points. This makes sense for fitting and smoothing; for interpolation however, most users (I'm speculating here) expect that these coincide. Elsewhere in scipy.interpolate, the currently recommended way is - for regular grids in N-D, use RegularGridInterpolator. This does guarantee that knots=data points. - for scattered data in N-D, use `griddata`. Internally it is triangulation-bases via QHull. In 2D specifically, it allows constructing C1-continuous splines in addition to linear and nearest-neighbor interpolators. 4. First, as mentioned above, `interp2d` switches between different algorithms depending on the input shapes, https://github.com/scipy/scipy/issues/11309. For 1D input arrays it does scattered data fitting, equivalent to the SmoothBivariateSpline; for meshgrid inputs, it's equivalent to RectBivariateSpline. Second, also as mentioned, there is no guarantee that the spline knots coincide with the data. This breaks e.g. a natural expectation that a linear interpolator is bilinear in data. Third, it has an unusual behavior with respect to scalar inputs, see Warren's detailed comment in https://github.com/scipy/scipy/issues/16396#issuecomment-1154175268, which contains the note "I suspect this breaks anyone's mental model for what should happen" Fourth, it does not handle data well with increasing/decreasing ordering, see https://github.com/scipy/scipy/issues/16492. There is also an ample opportunity to silently get wrong results. Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone. So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries. Thoughts? Evgeni
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation period? Matti On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
Hi Matti, Yes, absolutely! If we go ahead with this deprecation, I'll add a short note to the deprecation message and a longer discussion in the release notes. Roughly (untested code below, will flesh out in the relnotes): Almost direct replacements are : - interp2d(x, y, z, kind='cubic') ---> `SmoothBivariateSpline(x, y, z, kx=3, ky=3, s=0)` or `RectBivariateSpline(x, y, z, kx=3, ky=3, s=0)` depending on the input mode (grid or scattered) - for kind='linear', if there is a `RuntimeWarning: No more knots can be added` : use `tck = bisplrep(x, y, z, kx=1, ky=1, s=0)` and possibly adjust `mxest, myest` parameters However the preferred replacement for regular grids is `RegularGridInterpolator((x, y), z, method=...)` --- note the tuple argument (x, y). One reason this is messy is that interp2d itself tries to do too many things IMO. Cheers, Evgeni On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
Thanks for digging into this Evgeni! On Tue, Oct 4, 2022 at 9:04 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Hi Matti,
Yes, absolutely! If we go ahead with this deprecation, I'll add a short note to the deprecation message and a longer discussion in the release notes.
Roughly (untested code below, will flesh out in the relnotes):
Almost direct replacements are : - interp2d(x, y, z, kind='cubic') ---> `SmoothBivariateSpline(x, y, z, kx=3, ky=3, s=0)` or `RectBivariateSpline(x, y, z, kx=3, ky=3, s=0)` depending on the input mode (grid or scattered)
- for kind='linear', if there is a `RuntimeWarning: No more knots can be added` : use `tck = bisplrep(x, y, z, kx=1, ky=1, s=0)` and possibly adjust `mxest, myest` parameters
However the preferred replacement for regular grids is `RegularGridInterpolator((x, y), z, method=...)` --- note the tuple argument (x, y).
One reason this is messy is that interp2d itself tries to do too many things IMO.
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions. My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc. Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator)? Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation? Cheers, Ralf
Cheers,
Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator)?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)). `interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy (https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6...). `interpnd` is a private module which should probably get a leading underscore. In [17]: import scipy.interpolate as si In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings'] Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
On Tue, Oct 4, 2022 at 10:54 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly basic,
- 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein
using the recommended solver based on: polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator )?
This is a great idea indeed. Will draft one together with gh-16707.
Thanks!
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy ( https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6... ).
That sounds good to me.
`interpnd` is a private module which should probably get a leading underscore.
Sorry, I meant `interpn`: http://scipy.github.io/devdocs/reference/generated/scipy.interpolate.interpn... Cheers, Ralf
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
Sorry, I meant `interpn`: http://scipy.github.io/devdocs/reference/generated/scipy.interpolate.interpn...
`interpn` is in good shape. It's essentially a keystroke-saving wrapper over the RegularGridInterpolator, `interpn(point, values, xi ...)` == `res = RegularGridInterpolator(points, values, ...); res(xi)`. Were we starting from scratch, I'd argue against these sorts of wrappers but that's moot now, we don't want gratuitous breakage. Cheers, Evgeni
On Wed, Oct 5, 2022 at 7:34 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
Sorry, I meant `interpn`: http://scipy.github.io/devdocs/reference/generated/scipy.interpolate.interpn...
`interpn` is in good shape.
It's essentially a keystroke-saving wrapper over the RegularGridInterpolator, `interpn(point, values, xi ...)` == `res = RegularGridInterpolator(points, values, ...); res(xi)`. Were we starting from scratch, I'd argue against these sorts of wrappers but that's moot now, we don't want gratuitous breakage.
Thanks. If we have them, I think we should either not recommend them in that table we discussed earlier, or have a uniform set of them. The former sounds appropriate here, seems like we should recommend using RegularGridInterpolator directly. Cheers, Ralf
It would be extremely helpful to include a userguide with change cases - a bit like already explained in your arguments above. It will help cut down the time to make changes to old (barely remembered) code. Thanks! Paul On Tue, Oct 4, 2022 at 9:54 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly basic,
- 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein
using the recommended solver based on: polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator )?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy ( https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6... ).
`interpnd` is a private module which should probably get a leading underscore.
In [17]: import scipy.interpolate as si
In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings']
Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour?
that could be added to the warning message during the deprecation
Perhaps period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way
for
2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
I agree that the "legacy" designation is appropriate. On Tue, Oct 4, 2022 at 2:42 AM Paul Kuin <npkuin@gmail.com> wrote:
It would be extremely helpful to include a userguide with change cases - a bit like already explained in your arguments above. It will help cut down the time to make changes to old (barely remembered) code.
Thanks! Paul
On Tue, Oct 4, 2022 at 9:54 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly
- 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein
basic, using the recommended solver based on: polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator )?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy ( https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6... ).
`interpnd` is a private module which should probably get a leading underscore.
In [17]: import scipy.interpolate as si
In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings']
Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the
alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation
closest period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way
2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many
for problems,
and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: mhaberla@calpoly.edu
-- Matt Haberland Assistant Professor BioResource and Agricultural Engineering 08A-3K, Cal Poly
Dear Paul, Absolutely! Old barely remembered scripts are one of the main concerns here, and we certainly need to make the transition as smooth as possible. Here's a quick userguide, please let me know if that's what you're looking for: https://gist.github.com/ev-br/8544371b40f414b7eaf3fe6217209bff Cheers, Evgeni On Tue, Oct 4, 2022 at 12:41 PM Paul Kuin <npkuin@gmail.com> wrote:
It would be extremely helpful to include a userguide with change cases - a bit like already explained in your arguments above. It will help cut down the time to make changes to old (barely remembered) code.
Thanks! Paul
On Tue, Oct 4, 2022 at 9:54 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator)?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy (https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6...).
`interpnd` is a private module which should probably get a leading underscore.
In [17]: import scipy.interpolate as si
In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings']
Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the closest alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation period?
Matti
On 4/10/22 07:59, Evgeni Burovski wrote:
Hi,
TL;DR : I propose to deprecate and eventually remove the `interp2d` routine from `scipy.interpolate` ... Concluding, The routine's name seems to hint that it is _the_ way for 2D interpolation, even more so for users with prior Matlab experience. It is however not the recommended routine, it has too many problems, and scipy.interpolate would, I believe, be better with it gone.
So the proposal is to deprecate it and add ample documentation for users to migrate to either equivalent or better functionality. I'm ready to do the legwork, and I'm planning to be around to help with user queries.
Thoughts?
Evgeni _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: matti.picus@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
Dear Evgeni, Yes, I forgot we have notebooks now! Seriously, I use these 1d and 2d methods all the time, and have used it for calibration interpolations. That in itself begs the question if the new methods are numerically identical to within a percent or less. I assume of course that they are, since they seem to be based on the same core codes. Anyway, I'll keep an eye on this and run some checks once the new set of routines has been released. Thanks again, Paul On Thu, Oct 6, 2022 at 1:07 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Dear Paul,
Absolutely! Old barely remembered scripts are one of the main concerns here, and we certainly need to make the transition as smooth as possible. Here's a quick userguide, please let me know if that's what you're looking for:
https://gist.github.com/ev-br/8544371b40f414b7eaf3fe6217209bff
Cheers,
Evgeni
On Tue, Oct 4, 2022 at 12:41 PM Paul Kuin <npkuin@gmail.com> wrote:
It would be extremely helpful to include a userguide with change cases -
a bit like already explained in your arguments above. It will help cut down the time to make changes to old (barely remembered) code.
Thanks! Paul
On Tue, Oct 4, 2022 at 9:54 AM Evgeni Burovski <
That is indeed a little messy. It would be very useful I think to
have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly
basic, using the recommended solver based on:
- 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet
or in a gist (use something like https://tableconvert.com/markdown-generator)?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me.
What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy (
https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6... ).
`interpnd` is a private module which should probably get a leading
underscore.
In [17]: import scipy.interpolate as si
In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings']
Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com>
wrote:
Thanks for the careful analysis and volunteering to do the work. Deprecating the problematic interp2d seems to make sense but makes migration from matlab a little less obvious. What would be the
closest
alternative for people looking for matlab-equivalent behaviour? Perhaps that could be added to the warning message during the deprecation
Matti
On 4/10/22 07:59, Evgeni Burovski wrote: > Hi, > > TL;DR : I propose to deprecate and eventually remove the
`interp2d`
> routine from `scipy.interpolate` > ... > Concluding, The routine's name seems to hint that it is _the_ way for > 2D interpolation, even more so for users with prior Matlab experience. > It is however not the recommended routine, it has too many
evgeny.burovskiy@gmail.com> wrote: polynomials, etc. period? problems,
> and scipy.interpolate would, I believe, be better with it gone. > > So the proposal is to deprecate it and add ample documentation for > users to migrate to either equivalent or better functionality. I'm > ready to do the legwork, and I'm planning to be around to help with > user queries. > > Thoughts? > > Evgeni > _______________________________________________ > SciPy-Dev mailing list -- scipy-dev@python.org > To unsubscribe send an email to scipy-dev-leave@python.org > https://mail.python.org/mailman3/lists/scipy-dev.python.org/ > Member address: matti.picus@gmail.com _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
Dear Paul, There is nothing new to be released here. The only expected change is that interp2d will start emitting DeprecationWarnings for a while and then disappear. The other routines in this gist notebook will stay intact, and they are not new either --- FITPACK wrappers have been in scipy forever (tm). I believe the results with and without interp2d are exactly equivalent (up to machine precision maybe), so there is nothing stopping you and other users from migrating away from the interp2d syntactic sugar to core FITPACK wrappers --- today or tomorrow. If it turns out that there are datasets where results differ, I'd like to see those. For new code, I'd recommend more recent routines instead of FITPACK, but that's a separate story anyway. Hope this helps, Evgeni On Thu, Oct 6, 2022 at 3:39 PM Paul Kuin <npkuin@gmail.com> wrote:
Dear Evgeni,
Yes, I forgot we have notebooks now! Seriously, I use these 1d and 2d methods all the time, and have used it for calibration interpolations. That in itself begs the question if the new methods are numerically identical to within a percent or less. I assume of course that they are, since they seem to be based on the same core codes. Anyway, I'll keep an eye on this and run some checks once the new set of routines has been released.
Thanks again,
Paul
On Thu, Oct 6, 2022 at 1:07 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Dear Paul,
Absolutely! Old barely remembered scripts are one of the main concerns here, and we certainly need to make the transition as smooth as possible. Here's a quick userguide, please let me know if that's what you're looking for:
https://gist.github.com/ev-br/8544371b40f414b7eaf3fe6217209bff
Cheers,
Evgeni
On Tue, Oct 4, 2022 at 12:41 PM Paul Kuin <npkuin@gmail.com> wrote:
It would be extremely helpful to include a userguide with change cases - a bit like already explained in your arguments above. It will help cut down the time to make changes to old (barely remembered) code.
Thanks! Paul
On Tue, Oct 4, 2022 at 9:54 AM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
That is indeed a little messy. It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
Agreed, scipy.interpolate has a few gaps in functionality (the largest one being smoothing splines), and a rather large issue of API having grown organically over several generations of devs. Newer parts are (mostly) consistent, but that's about it.
My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc.
Agreed. There is a start at https://github.com/scipy/scipy/pull/16707, which still needs a major revision.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator)?
This is a great idea indeed. Will draft one together with gh-16707.
Your reasoning for deprecating `interp2d` sounds convincing to me. What about `interp1d` and `interpnd`, are they in a similar situation?
I think these two are different (in different ways :-)).
`interp1d` is brittle and ugly but established, widely used and is not broken beyond repair (which interp2d is, I'd argue). And it does not need much maintenance effort anymore. So I'd say it's a prime candidate for formally declaring as being legacy (https://mail.python.org/archives/list/scipy-dev@python.org/thread/Z33JO27EX6...).
`interpnd` is a private module which should probably get a leading underscore.
In [17]: import scipy.interpolate as si
In [18]: [_ for _ in dir(si.interpnd) if not _.startswith('_')] Out[18]: ['CloughTocher2DInterpolator', 'GradientEstimationWarning', 'LinearNDInterpolator', 'NDInterpolatorBase', 'estimate_gradients_2d_global', 'np', 'qhull', 'warnings']
Cheers, Evgeni
On Tue, Oct 4, 2022 at 9:19 AM Matti Picus <matti.picus@gmail.com> wrote: > > Thanks for the careful analysis and volunteering to do the work. > Deprecating the problematic interp2d seems to make sense but makes > migration from matlab a little less obvious. What would be the closest > alternative for people looking for matlab-equivalent behaviour? Perhaps > that could be added to the warning message during the deprecation period? > > > Matti > > > On 4/10/22 07:59, Evgeni Burovski wrote: > > Hi, > > > > TL;DR : I propose to deprecate and eventually remove the `interp2d` > > routine from `scipy.interpolate` > > ... > > Concluding, The routine's name seems to hint that it is _the_ way for > > 2D interpolation, even more so for users with prior Matlab experience. > > It is however not the recommended routine, it has too many problems, > > and scipy.interpolate would, I believe, be better with it gone. > > > > So the proposal is to deprecate it and add ample documentation for > > users to migrate to either equivalent or better functionality. I'm > > ready to do the legwork, and I'm planning to be around to help with > > user queries. > > > > Thoughts? > > > > Evgeni > > _______________________________________________ > > SciPy-Dev mailing list -- scipy-dev@python.org > > To unsubscribe send an email to scipy-dev-leave@python.org > > https://mail.python.org/mailman3/lists/scipy-dev.python.org/ > > Member address: matti.picus@gmail.com > _______________________________________________ > SciPy-Dev mailing list -- scipy-dev@python.org > To unsubscribe send an email to scipy-dev-leave@python.org > https://mail.python.org/mailman3/lists/scipy-dev.python.org/ > Member address: evgeny.burovskiy@gmail.com _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: ralf.gommers@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: npkuin@gmail.com
_______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
Hi, On Thu, Oct 6, 2022 at 1:57 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Dear Paul,
There is nothing new to be released here. The only expected change is that interp2d will start emitting DeprecationWarnings for a while and then disappear.
Sorry - just to say - that sounds reasonable to me - but I wonder whether we should leave a stub in there for a long time, to give an informative error and point people to the correct code to port to the newer routines. I bet there are lots of old scripts using interp2d that people will try running for years to come, and it would be reassuring to know that they will be minimally confused with the resulting error. Cheers, Matthew
Hi Matthew,
There is nothing new to be released here. The only expected change is that interp2d will start emitting DeprecationWarnings for a while and then disappear.
Sorry - just to say - that sounds reasonable to me - but I wonder whether we should leave a stub in there for a long time, to give an informative error and point people to the correct code to port to the newer routines. I bet there are lots of old scripts using interp2d that people will try running for years to come, and it would be reassuring to know that they will be minimally confused with the resulting error.
So, to check if I understood correctly, you are suggesting to implement essentially a three-step deprecation: 1. in release N, interp2d starts emitting DeprecationWarnings; 2. in release N+2, interp2d implementation is replaced by a stub which gives a hard error with an explanation and a pointer to a porting guide; 3. in release N+X (X >> 2), the stub is finally gone, and an attempt to `from scipy.interpolate import interp2d` gives a usual ImportError? Evgeni
Hi, On Sat, Oct 8, 2022 at 6:13 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Hi Matthew,
There is nothing new to be released here. The only expected change is that interp2d will start emitting DeprecationWarnings for a while and then disappear.
Sorry - just to say - that sounds reasonable to me - but I wonder whether we should leave a stub in there for a long time, to give an informative error and point people to the correct code to port to the newer routines. I bet there are lots of old scripts using interp2d that people will try running for years to come, and it would be reassuring to know that they will be minimally confused with the resulting error.
So, to check if I understood correctly, you are suggesting to implement essentially a three-step deprecation:
1. in release N, interp2d starts emitting DeprecationWarnings; 2. in release N+2, interp2d implementation is replaced by a stub which gives a hard error with an explanation and a pointer to a porting guide; 3. in release N+X (X >> 2), the stub is finally gone, and an attempt to `from scipy.interpolate import interp2d` gives a usual ImportError?
Yes, right- with a preference for a fairly large X, given that the maintenance burden is pretty low. Cheers, Matthew
Hi, Just to close the loop: https://github.com/scipy/scipy/pull/17180 is merged, and interp2d emits DeprecationWarnings starting from SciPy 1.10. The current plan is to follow the three-step process Matthew suggested above and make it raise a hard error in SciPy 1.12. Thanks all for your comments and suggestions! If you have a problem porting your interp2d usage to other routines, please feel free to ping me either on GitHub or in this email thread. Evgeni On Sat, Oct 8, 2022 at 10:36 PM Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Sat, Oct 8, 2022 at 6:13 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Hi Matthew,
There is nothing new to be released here. The only expected change is that interp2d will start emitting DeprecationWarnings for a while and then disappear.
Sorry - just to say - that sounds reasonable to me - but I wonder whether we should leave a stub in there for a long time, to give an informative error and point people to the correct code to port to the newer routines. I bet there are lots of old scripts using interp2d that people will try running for years to come, and it would be reassuring to know that they will be minimally confused with the resulting error.
So, to check if I understood correctly, you are suggesting to implement essentially a three-step deprecation:
1. in release N, interp2d starts emitting DeprecationWarnings; 2. in release N+2, interp2d implementation is replaced by a stub which gives a hard error with an explanation and a pointer to a porting guide; 3. in release N+X (X >> 2), the stub is finally gone, and an attempt to `from scipy.interpolate import interp2d` gives a usual ImportError?
Yes, right- with a preference for a fairly large X, given that the maintenance burden is pretty low.
Cheers,
Matthew _______________________________________________ SciPy-Dev mailing list -- scipy-dev@python.org To unsubscribe send an email to scipy-dev-leave@python.org https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: evgeny.burovskiy@gmail.com
Hi, On Tue, Oct 4, 2022 at 11:02 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator)?
Here's a(n opinionated) table of this sort for interpolation (not smoothing): https://gist.github.com/ev-br/fcd25570c3f332faefd521e6d3c87510 How to integrate it into the main scipy docs remains to be seen. Cheers, Evgeni
On Tue, Oct 18, 2022 at 12:36 PM Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:
Hi,
On Tue, Oct 4, 2022 at 11:02 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
It would be very useful I think to have a table showing the recommended functions/classes for each form of interpolation. That may show where we have API design issues - my impression is that we do have those, more so than gaps in functionality. Or vice versa the properties of each function, similar to the table at http://scipy.github.io/devdocs/reference/optimize.html#scalar-functions.
My impression is that the majority of users want something fairly basic, using the recommended solver based on: - 1-D / 2-D / n-D - structured or unstructured grid - exact interpolation or approximation (smoothing polynomials/splines) - type of interpolation: nearest, linear, cubic, quintic - (and maybe) derivatives: level of continuity or ability to estimate derivatives
And then there's a whole host of other more special-purpose or advanced functionality: RBF, Akima, barycentric, Clough-Tocher, Bernstein polynomials, etc.
Would this make sense to sketch a table like that, in a spreadsheet or in a gist (use something like https://tableconvert.com/markdown-generator )?
Here's a(n opinionated) table of this sort for interpolation (not smoothing): https://gist.github.com/ev-br/fcd25570c3f332faefd521e6d3c87510 How to integrate it into the main scipy docs remains to be seen.
Thanks for working on this Evgeni. That actually looks worse than I expected. There's very little consistency in either coverage or API syntax. Part of the reason for that may be a consequence of opinionated choices. E.g., for unstructured grids, `griddata(..., method=)` covers all linear, nearest and 1D/2D cubic - so while only listed as an alias, it's equivalent to all three classes listed + interp1d. I was also surprised not to see BSpline, since it's a fairly new addition. Cheers, Ralf
participants (6)
-
Evgeni Burovski -
Matt Haberland -
Matthew Brett -
Matti Picus -
Paul Kuin -
Ralf Gommers