curve_fit() should require initial values for parameters
Hi All, First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize. I am a very big fan, and very appreciative of the work done here. With lmfit we have tried to take the "rough edges" from optimization and curve-fitting with python, but we're very much in favor of building wrappers on top of the core of scipy.optimize. Still, many people use `curve_fit` because it comes built-in with scipy, is well advertised, and is well-suited for simple uses. It is clearly aimed at novices and tries to hide many of the complexities of using optimization routines for curve fitting. I try to help people with questions about using `curve_fit` and `scipy.optimize` as well as `lmfit`. In trying to help people using `curve_fit`, I've noticed a disturbing trend. When a novice or even experienced user asks for help using `curve_fit` because a fit "doesn't work" there is a very good chance that they did not specify `p0` for the initial values and that the default behavior of setting all starting values to 1.0 will cause their fit to fail to converge, or really to even start. This failure is understandable to an experienced user, but apparently not to the novice. Curve-fitting problems are by nature local solvers and are always sensitive to initial values. For some problems, parameter values of 1.0 are simply inappropriate, and the numerical derivatives for some parameters near values of 1.0 will be 0. Indeed, there is no value that is always a reasonable starting value for all parameters. FWIW, in lmfit, we simply refuse to let a user run a curve-fitting problem without initial values. I believe that most other curve-fitting interfaces also require initial values for all parameters. Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value. I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often. Cheers, --Matt Newville
For what it's worth, I agree that we should remove the default. There's no generic value that accidentally works often enough to justify the time wasted and novice confusion when it fails. I also agree that it is going to take a reasonably long deprecation cycle. On Wed, Jan 23, 2019 at 5:26 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi All,
First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize. I am a very big fan, and very appreciative of the work done here. With lmfit we have tried to take the "rough edges" from optimization and curve-fitting with python, but we're very much in favor of building wrappers on top of the core of scipy.optimize.
Still, many people use `curve_fit` because it comes built-in with scipy, is well advertised, and is well-suited for simple uses. It is clearly aimed at novices and tries to hide many of the complexities of using optimization routines for curve fitting. I try to help people with questions about using `curve_fit` and `scipy.optimize` as well as `lmfit`. In trying to help people using `curve_fit`, I've noticed a disturbing trend. When a novice or even experienced user asks for help using `curve_fit` because a fit "doesn't work" there is a very good chance that they did not specify `p0` for the initial values and that the default behavior of setting all starting values to 1.0 will cause their fit to fail to converge, or really to even start.
This failure is understandable to an experienced user, but apparently not to the novice. Curve-fitting problems are by nature local solvers and are always sensitive to initial values. For some problems, parameter values of 1.0 are simply inappropriate, and the numerical derivatives for some parameters near values of 1.0 will be 0. Indeed, there is no value that is always a reasonable starting value for all parameters. FWIW, in lmfit, we simply refuse to let a user run a curve-fitting problem without initial values. I believe that most other curve-fitting interfaces also require initial values for all parameters.
Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value.
I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often.
Cheers,
--Matt Newville
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
-- Robert Kern
On Wed, Jan 23, 2019 at 11:12 PM Robert Kern <robert.kern@gmail.com> wrote:
For what it's worth, I agree that we should remove the default. There's no generic value that accidentally works often enough to justify the time wasted and novice confusion when it fails.
I also agree that it is going to take a reasonably long deprecation cycle.
On Wed, Jan 23, 2019 at 5:26 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi All,
First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize. I am a very big fan, and very appreciative of the work done here. With lmfit we have tried to take the "rough edges" from optimization and curve-fitting with python, but we're very much in favor of building wrappers on top of the core of scipy.optimize.
Still, many people use `curve_fit` because it comes built-in with scipy, is well advertised, and is well-suited for simple uses. It is clearly aimed at novices and tries to hide many of the complexities of using optimization routines for curve fitting. I try to help people with questions about using `curve_fit` and `scipy.optimize` as well as `lmfit`. In trying to help people using `curve_fit`, I've noticed a disturbing trend. When a novice or even experienced user asks for help using `curve_fit` because a fit "doesn't work" there is a very good chance that they did not specify `p0` for the initial values and that the default behavior of setting all starting values to 1.0 will cause their fit to fail to converge, or really to even start.
This failure is understandable to an experienced user, but apparently not to the novice. Curve-fitting problems are by nature local solvers and are always sensitive to initial values. For some problems, parameter values of 1.0 are simply inappropriate, and the numerical derivatives for some parameters near values of 1.0 will be 0. Indeed, there is no value that is always a reasonable starting value for all parameters. FWIW, in lmfit, we simply refuse to let a user run a curve-fitting problem without initial values. I believe that most other curve-fitting interfaces also require initial values for all parameters.
Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value.
I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often.
I think making initial values compulsory is too much of a break with tradition. IMO, a warning and better documentation would be more appropriate. https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified. I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided. I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems. I have no idea what the overall selection of use cases is. A brief skimming of some github searches shows that many users don't specify the initial values. (A semi-random search result https://github.com/jmunroe/phys2820-fall2018/blob/e270b1533130b2b7acd0ec5da3... ) (Asides: The feedback I get about statsmodels are almost only for cases that "don't work", e.g. badly conditioned data, bad scaling of the data, corner case. Based on this feedback statsmodels optimization looks pretty bad, but this does not reflect that it works well in, say, 90% of the cases. However, unlike curve_fit, statsmodels has mostly models with predefined nonlinear functions which makes it easier to fit. )
Cheers,
--Matt Newville
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
-- Robert Kern _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Thu, Jan 24, 2019 at 10:26 AM <josef.pktd@gmail.com> wrote:
On Wed, Jan 23, 2019 at 11:12 PM Robert Kern <robert.kern@gmail.com> wrote:
For what it's worth, I agree that we should remove the default. There's no generic value that accidentally works often enough to justify the time wasted and novice confusion when it fails.
I also agree that it is going to take a reasonably long deprecation cycle.
On Wed, Jan 23, 2019 at 5:26 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi All,
First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize. I am a very big fan, and very appreciative of the work done here. With lmfit we have tried to take the "rough edges" from optimization and curve-fitting with python, but we're very much in favor of building wrappers on top of the core of scipy.optimize.
Still, many people use `curve_fit` because it comes built-in with scipy, is well advertised, and is well-suited for simple uses. It is clearly aimed at novices and tries to hide many of the complexities of using optimization routines for curve fitting. I try to help people with questions about using `curve_fit` and `scipy.optimize` as well as `lmfit`. In trying to help people using `curve_fit`, I've noticed a disturbing trend. When a novice or even experienced user asks for help using `curve_fit` because a fit "doesn't work" there is a very good chance that they did not specify `p0` for the initial values and that the default behavior of setting all starting values to 1.0 will cause their fit to fail to converge, or really to even start.
This failure is understandable to an experienced user, but apparently not to the novice. Curve-fitting problems are by nature local solvers and are always sensitive to initial values. For some problems, parameter values of 1.0 are simply inappropriate, and the numerical derivatives for some parameters near values of 1.0 will be 0. Indeed, there is no value that is always a reasonable starting value for all parameters. FWIW, in lmfit, we simply refuse to let a user run a curve-fitting problem without initial values. I believe that most other curve-fitting interfaces also require initial values for all parameters.
Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value.
I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often.
I think making initial values compulsory is too much of a break with tradition.
Well, it may be a break with the tradition of using scipy.optimize.curve_fit, but I do not think it is a break with the tradition of curve fitting. Indeed, what curve_fit does when a user leaves `p0=None` is *not* to leave the initial values unspecified -- the underlying optimization routine would simply not accept that -- but rather to silently select values that are all '1.0'. I am not aware of any other curve-fitting code or use of non-linear optimization that does this. So, in a sense it is a "traditional". It is also wrong. IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
I think trying to guess starting values would require an understanding of the function calculating the model, and is not generally solvable. For sure, if one knows the function, initial guesses are possible. Lmfit has this capability for a few commonly used model functions and those initial guess are often very good. But it cannot be done in general.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems. I have no idea what the overall selection of use cases is. A brief skimming of some github searches shows that many users don't specify the initial values. (A semi-random search result https://github.com/jmunroe/phys2820-fall2018/blob/e270b1533130b2b7acd0ec5da3... )
(Asides: The feedback I get about statsmodels are almost only for cases that "don't work", e.g. badly conditioned data, bad scaling of the data, corner case. Based on this feedback statsmodels optimization looks pretty bad, but this does not reflect that it works well in, say, 90% of the cases. However, unlike curve_fit, statsmodels has mostly models with predefined nonlinear functions which makes it easier to fit. )
I do not know what the usage of `curve_fit` is. Apparently some users get tripped up by not specifying initial values. But that is in the nature of curve fitting -- initial values are necessary. Claiming that they do not matter or are an unnecessary burden is just not correct. It seems like the first step is to change `curve_fit` to raise a warning or print a message (not sure which is preferred) when `p0` is `None`, but continue guessing `1.0`, at least for the time being. Eventually, this could be changed to raise an exception if `p0` is `None`. Perhaps a middle step would be to change it to not guess `1.0` but a number comprised of a uniformly selected random number between [-1, 1] for the mantissa and a uniformly selected random integer between [-20, 20] for the exponent, as long as bounds are respected? Cheers, --Matt Newville
On Thu, Jan 24, 2019 at 1:16 PM Matt Newville <newville@cars.uchicago.edu> wrote:
On Thu, Jan 24, 2019 at 10:26 AM <josef.pktd@gmail.com> wrote:
On Wed, Jan 23, 2019 at 11:12 PM Robert Kern <robert.kern@gmail.com> wrote:
For what it's worth, I agree that we should remove the default. There's no generic value that accidentally works often enough to justify the time wasted and novice confusion when it fails.
I also agree that it is going to take a reasonably long deprecation cycle.
On Wed, Jan 23, 2019 at 5:26 PM Matt Newville < newville@cars.uchicago.edu> wrote:
Hi All,
First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize. I am a very big fan, and very appreciative of the work done here. With lmfit we have tried to take the "rough edges" from optimization and curve-fitting with python, but we're very much in favor of building wrappers on top of the core of scipy.optimize.
Still, many people use `curve_fit` because it comes built-in with scipy, is well advertised, and is well-suited for simple uses. It is clearly aimed at novices and tries to hide many of the complexities of using optimization routines for curve fitting. I try to help people with questions about using `curve_fit` and `scipy.optimize` as well as `lmfit`. In trying to help people using `curve_fit`, I've noticed a disturbing trend. When a novice or even experienced user asks for help using `curve_fit` because a fit "doesn't work" there is a very good chance that they did not specify `p0` for the initial values and that the default behavior of setting all starting values to 1.0 will cause their fit to fail to converge, or really to even start.
This failure is understandable to an experienced user, but apparently not to the novice. Curve-fitting problems are by nature local solvers and are always sensitive to initial values. For some problems, parameter values of 1.0 are simply inappropriate, and the numerical derivatives for some parameters near values of 1.0 will be 0. Indeed, there is no value that is always a reasonable starting value for all parameters. FWIW, in lmfit, we simply refuse to let a user run a curve-fitting problem without initial values. I believe that most other curve-fitting interfaces also require initial values for all parameters.
Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value.
I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often.
I think making initial values compulsory is too much of a break with tradition.
Well, it may be a break with the tradition of using scipy.optimize.curve_fit, but I do not think it is a break with the tradition of curve fitting. Indeed, what curve_fit does when a user leaves `p0=None` is *not* to leave the initial values unspecified -- the underlying optimization routine would simply not accept that -- but rather to silently select values that are all '1.0'. I am not aware of any other curve-fitting code or use of non-linear optimization that does this. So, in a sense it is a "traditional". It is also wrong.
scipy.stats distribution fit also default to ones if not overridden by the specific distribution. statsmodels only has a few models with arbitrary user functions, but I usually try to set a default that works at least in some common cases. def fitstart(self): #might not make sense for more general functions return np.zeros(self.exog.shape[1]) curve_fit was added to scipy as a convenience function, in contrast to the "serious" optimizers. For that I think putting in more effort to reduce the work by users is useful. (Note, I was never a fan of using `inspect` which is needed to know how many default starting values to create.)
IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
I think trying to guess starting values would require an understanding of the function calculating the model, and is not generally solvable. For sure, if one knows the function, initial guesses are possible. Lmfit has this capability for a few commonly used model functions and those initial guess are often very good. But it cannot be done in general.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems. I have no idea what the overall selection of use cases is. A brief skimming of some github searches shows that many users don't specify the initial values. (A semi-random search result https://github.com/jmunroe/phys2820-fall2018/blob/e270b1533130b2b7acd0ec5da3... )
(Asides: The feedback I get about statsmodels are almost only for cases that "don't work", e.g. badly conditioned data, bad scaling of the data, corner case. Based on this feedback statsmodels optimization looks pretty bad, but this does not reflect that it works well in, say, 90% of the cases. However, unlike curve_fit, statsmodels has mostly models with predefined nonlinear functions which makes it easier to fit. )
I do not know what the usage of `curve_fit` is. Apparently some users get tripped up by not specifying initial values. But that is in the nature of curve fitting -- initial values are necessary. Claiming that they do not matter or are an unnecessary burden is just not correct.
It seems like the first step is to change `curve_fit` to raise a warning or print a message (not sure which is preferred) when `p0` is `None`, but continue guessing `1.0`, at least for the time being. Eventually, this could be changed to raise an exception if `p0` is `None`.
Perhaps a middle step would be to change it to not guess `1.0` but a number comprised of a uniformly selected random number between [-1, 1] for the mantissa and a uniformly selected random integer between [-20, 20] for the exponent, as long as bounds are respected?
Cheers,
--Matt Newville _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Josef, On Thu, 24 Jan 2019 11:26:09 -0500, josef.pktd@gmail.com wrote:
I think making initial values compulsory is too much of a break with tradition. IMO, a warning and better documentation would be more appropriate. https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
Isn't the greater danger that convergence succeeds, with p0 unspecified, and the resulting model not being at all what the user had in mind?
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
This is the type of magic I hope we can avoid. Having different execution paths based on some vaguely defined notion of perceived failure seems dangerous at best.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems.
I agree; and I think this can be accomplished by better documentation, helpful warnings, and assisting the user in choosing correct parameters. Best regards, Stéfan
On Thu, Jan 24, 2019 at 1:46 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hi Josef,
On Thu, 24 Jan 2019 11:26:09 -0500, josef.pktd@gmail.com wrote:
I think making initial values compulsory is too much of a break with tradition. IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi...
does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
Isn't the greater danger that convergence succeeds, with p0 unspecified, and the resulting model not being at all what the user had in mind?
Unless the optimization problem is globally convex, the user always needs to check the results.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
This is the type of magic I hope we can avoid. Having different execution paths based on some vaguely defined notion of perceived failure seems dangerous at best.
I there is no guarantee for a global optimum, it's still what either the program or the user has to do. E.g. for statsmodels (very rough guess on numbers) 90% of the cases work fine 10% of the cases the data is not appropriate, singular, ill conditioned or otherwise "not nice" 10% of the cases the optimizer has problems and does not converge. In this last case either the program or the user needs to work more: We can try different optimizers, e.g. start with nelder-mead before switching to a gradient optimizer. Or, switch to global optimizer from scipy, if the underlying model is complex and might not be well behaved. or pure man's global optimizer: try out many different random or semi-random starting values. (and if all fails go back to the drawing board and try to find a parameterization that is better behaved.) statsmodels is switching optimizers in some cases, but in most cases it is up to the user to change the optimizers after convergence failure. However, we did select default optimizers by which scipy optimizer seems to work well for the various cases. Stata is also switching optimizers in some cases, and AFAIR has in some cases and option to "try harder". statsmodels is still missing an automatic "try harder" option, that automatically switches optimizers on convergence failure.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems.
I agree; and I think this can be accomplished by better documentation, helpful warnings, and assisting the user in choosing correct parameters.
The main question for me is whether the warnings and improved documentation are enough, or whether curve_fit needs to force every user to specify the starting values. i.e. I think Try automatic first, and if that does not succeed, then the user has to think again, is more convenient, than "you have to think about your problem first, don't just hit the button". Josef
Best regards, Stéfan _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Thu, 24 Jan 2019 14:19:09 -0500, josef.pktd@gmail.com wrote:
i.e. I think Try automatic first, and if that does not succeed, then the user has to think again, is more convenient, than "you have to think about your problem first, don't just hit the button".
My question is: can you ever know with certainty when you did not succeed? In many cases, you'll probably think you did fine? But, consider, e.g., an alternative implementation that tries 5 different fitting methods, and then picks the solution with the lowest error. That, while not computationally optimal, is a process you can explain clearly. In this case, how do you communicate to the user through which steps there data went to obtained the returned result? This is what I mean by avoiding the magic; making sure the user knows where their results came from. I am not opposed to convenience, as long as there is clear communication. Stéfan
On Thu, Jan 24, 2019 at 3:01 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Thu, 24 Jan 2019 14:19:09 -0500, josef.pktd@gmail.com wrote:
i.e. I think Try automatic first, and if that does not succeed, then the user has to think again, is more convenient, than "you have to think about your problem first, don't just hit the button".
My question is: can you ever know with certainty when you did not succeed? In many cases, you'll probably think you did fine?
But, consider, e.g., an alternative implementation that tries 5 different fitting methods, and then picks the solution with the lowest error. That, while not computationally optimal, is a process you can explain clearly.
In this case, how do you communicate to the user through which steps there data went to obtained the returned result? This is what I mean by avoiding the magic; making sure the user knows where their results came from. I am not opposed to convenience, as long as there is clear communication.
Trying out different starting values and check convergence is similar to what global optimizers like basinhopping do. curve_fit has optional returns, which are not mentioned in the docstring, no description of `full_output`. Any systematic method, like "try again if convergence failed" can be described. The actual search path, which methods have been tried, can be included in some `fit_history` in the `full_output`. (E.g. statsmodels includes in most models some extra information about the last optimization method used that includes which starting values were used, which optimizer and similar. AFAIR, we don't return information about earlier, preliminary optimization steps to get good starting values for the final optimization.) I never tried whether leastsq can be used inside basinhopping and I have not tried the new global optimizers yet. There would be options to make it work for more cases without requiring users to specify starting values and possibly to switch to fancier optimizers. (I'm mainly arguing this position because curve_fit was initially sold as a function that is more convenient to use than leastsq, i.e. it should put less demand on the user than just using the underlying optimizers directly.) Josef
Stéfan _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Thu, 24 Jan 2019 15:35:14 -0500, josef.pktd@gmail.com wrote:
Any systematic method, like "try again if convergence failed" can be described. The actual search path, which methods have been tried, can be included in some `fit_history` in the `full_output`.
I could go along with that, if the user then has a clear path towards restarting the procedure using only the latest method + initial parameter. And then, as you say, the function could be marketed as more of a black box solution. I don't usually recommend that people use black boxes, but I guess I can be convinced that they have a place in certain use cases. Best regards, Stéfan
Hi All, On Thu, Jan 24, 2019 at 1:20 PM <josef.pktd@gmail.com> wrote:
On Thu, Jan 24, 2019 at 1:46 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hi Josef,
On Thu, 24 Jan 2019 11:26:09 -0500, josef.pktd@gmail.com wrote:
I think making initial values compulsory is too much of a break with tradition. IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi...
does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
Isn't the greater danger that convergence succeeds, with p0 unspecified, and the resulting model not being at all what the user had in mind?
Unless the optimization problem is globally convex, the user always needs to check the results.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
This is the type of magic I hope we can avoid. Having different execution paths based on some vaguely defined notion of perceived failure seems dangerous at best.
I there is no guarantee for a global optimum, it's still what either the program or the user has to do.
E.g. for statsmodels (very rough guess on numbers) 90% of the cases work fine 10% of the cases the data is not appropriate, singular, ill conditioned or otherwise "not nice" 10% of the cases the optimizer has problems and does not converge.
In this last case either the program or the user needs to work more: We can try different optimizers, e.g. start with nelder-mead before switching to a gradient optimizer. Or, switch to global optimizer from scipy, if the underlying model is complex and might not be well behaved. or pure man's global optimizer: try out many different random or semi-random starting values. (and if all fails go back to the drawing board and try to find a parameterization that is better behaved.)
statsmodels is switching optimizers in some cases, but in most cases it is up to the user to change the optimizers after convergence failure. However, we did select default optimizers by which scipy optimizer seems to work well for the various cases. Stata is also switching optimizers in some cases, and AFAIR has in some cases and option to "try harder". statsmodels is still missing an automatic "try harder" option, that automatically switches optimizers on convergence failure.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems.
I agree; and I think this can be accomplished by better documentation, helpful warnings, and assisting the user in choosing correct parameters.
The main question for me is whether the warnings and improved documentation are enough, or whether curve_fit needs to force every user to specify the starting values.
I may not be understanding what you say about statsmodel. Is that using or related to `curve_fit()`? Perhaps it works well in many cases for you because of the limited range of the probability distribution functions being fitted? My view on this starts with the fact that Initial values are actually required in non-linear optimization. In a sense, not "forcing every user to specify starting values" and silently replacing `None` with `np.ones(n_variables)` is misinforming the user. I cannot think of any reason to recommend this behavior. It will certainly fail spectacularly sometimes. I would not try to guess (or probably believe anyone else's guess ;)) how often this would happen, but I can tell you that for essentially all of the fitting I do and my applications do for other users, giving initial values of 1 for all parameters would fail in such a way as to not move past the initial values (that is "not work" in a way that might easily confuse a novice). Again, I do not use `curve_fit()`, but clearly `p0=None` fails often enough to cause confusion. i.e. I think
Try automatic first, and if that does not succeed, then the user has to think again, is more convenient, than "you have to think about your problem first, don't just hit the button".
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit. The user of `curve_fit` already has to provide data (about which they presumably know something) and write a function that models that data. I think that qualifies as "has to think about their problem". They should be able to make some guess ("prior belief") of the parameter values. Hopefully they will run their modelling function with some sensible values for the parameters before running `curve_fit` to make sure that their function runs correctly. Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant. But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient. I can believe a deprecation time would be reasonable, but I would hope this behavior could be removed. --Matt Newville
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice. I don't know anything about the curve fit I will get in the end. So I don't need to pretend that I know a good starting value. Maybe for 3 parameter functions, fine I can come up with an argument but you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark. I am pretty confident that if we force this argument most of the people you want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous? On Mon, Jan 28, 2019 at 3:07 AM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi All,
On Thu, Jan 24, 2019 at 1:20 PM <josef.pktd@gmail.com> wrote:
On Thu, Jan 24, 2019 at 1:46 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hi Josef,
On Thu, 24 Jan 2019 11:26:09 -0500, josef.pktd@gmail.com wrote:
I think making initial values compulsory is too much of a break with tradition. IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi...
does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
Isn't the greater danger that convergence succeeds, with p0 unspecified, and the resulting model not being at all what the user had in mind?
Unless the optimization problem is globally convex, the user always needs to check the results.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
This is the type of magic I hope we can avoid. Having different execution paths based on some vaguely defined notion of perceived failure seems dangerous at best.
I there is no guarantee for a global optimum, it's still what either the program or the user has to do.
E.g. for statsmodels (very rough guess on numbers) 90% of the cases work fine 10% of the cases the data is not appropriate, singular, ill conditioned or otherwise "not nice" 10% of the cases the optimizer has problems and does not converge.
In this last case either the program or the user needs to work more: We can try different optimizers, e.g. start with nelder-mead before switching to a gradient optimizer. Or, switch to global optimizer from scipy, if the underlying model is complex and might not be well behaved. or pure man's global optimizer: try out many different random or semi-random starting values. (and if all fails go back to the drawing board and try to find a parameterization that is better behaved.)
statsmodels is switching optimizers in some cases, but in most cases it is up to the user to change the optimizers after convergence failure. However, we did select default optimizers by which scipy optimizer seems to work well for the various cases. Stata is also switching optimizers in some cases, and AFAIR has in some cases and option to "try harder". statsmodels is still missing an automatic "try harder" option, that automatically switches optimizers on convergence failure.
I'm not a user of curve_fit, but I guess there might be a strong selection bias in use cases when helping out users that run into problems.
I agree; and I think this can be accomplished by better documentation, helpful warnings, and assisting the user in choosing correct parameters.
The main question for me is whether the warnings and improved documentation are enough, or whether curve_fit needs to force every user to specify the starting values.
I may not be understanding what you say about statsmodel. Is that using or related to `curve_fit()`? Perhaps it works well in many cases for you because of the limited range of the probability distribution functions being fitted?
My view on this starts with the fact that Initial values are actually required in non-linear optimization. In a sense, not "forcing every user to specify starting values" and silently replacing `None` with `np.ones(n_variables)` is misinforming the user. I cannot think of any reason to recommend this behavior. It will certainly fail spectacularly sometimes. I would not try to guess (or probably believe anyone else's guess ;)) how often this would happen, but I can tell you that for essentially all of the fitting I do and my applications do for other users, giving initial values of 1 for all parameters would fail in such a way as to not move past the initial values (that is "not work" in a way that might easily confuse a novice). Again, I do not use `curve_fit()`, but clearly `p0=None` fails often enough to cause confusion.
i.e. I think
Try automatic first, and if that does not succeed, then the user has to think again, is more convenient, than "you have to think about your problem first, don't just hit the button".
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
The user of `curve_fit` already has to provide data (about which they presumably know something) and write a function that models that data. I think that qualifies as "has to think about their problem". They should be able to make some guess ("prior belief") of the parameter values. Hopefully they will run their modelling function with some sensible values for the parameters before running `curve_fit` to make sure that their function runs correctly.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant. But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient. I can believe a deprecation time would be reasonable, but I would hope this behavior could be removed.
--Matt Newville
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Tue, 29 Jan 2019 17:53:37 +0100, Ilhan Polat wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
This entire thread is about why np.ones is not a good choice. It is not as good as any other in any particular case, and is patently wrong for most problems. Sure, each problem has its own "better" and "worse" initial values, so if you are trying to optimize over all possible cases, it may be as good a choice as any other. But the question here is exactly about whether that is a sensible thing to do. Matt argues that it is not, and I find his argument convincing. The current API leads the user to believe that, without specifying p0, they are likely to get a reasonable answer out. We can help our users by signaling to them that this is simply not the case. Best regards, Stéfan
Sorry, I can't see this helps our users by forcing them entering values that they have no idea about would educate them about nonconvexity of the problem. This is properly documented in the argument docs and we can expand it and if they don't read the docs we are done. My opinion is that we are trying to solve a problem that doesn't exist. On Wed, Jan 30, 2019 at 2:30 AM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Tue, 29 Jan 2019 17:53:37 +0100, Ilhan Polat wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
This entire thread is about why np.ones is not a good choice. It is not as good as any other in any particular case, and is patently wrong for most problems. Sure, each problem has its own "better" and "worse" initial values, so if you are trying to optimize over all possible cases, it may be as good a choice as any other. But the question here is exactly about whether that is a sensible thing to do. Matt argues that it is not, and I find his argument convincing.
The current API leads the user to believe that, without specifying p0, they are likely to get a reasonable answer out. We can help our users by signaling to them that this is simply not the case.
Best regards, Stéfan _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Ilhan, On Tue, Jan 29, 2019 at 10:54 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
Well, I agree that `np.ones(n_variables)` is as good as any other choice. All default choices are horrible and not defensible. Mathematically, algorithmically, and conceptually, initial values ARE REQUIRED for non-linear least squares optimization. The codes underlying `curve_fit()` (including `leastsq` and the mess that is `least_square`) do not permit the user to not provide initial values. The algorithms used simply do not make sense without initial values. Programmatically, an optional keyword argument to a function, say with a default value of `None`, implies that there is a sensible default for that value. Thus, default fitting tolerances might be 1.e-7 (or, perhaps square root of machine precision) or the default value for "method to calculate jacobian" might be `None` to mean "calculate by finite difference". For these optional inputs, the function (say, `curve_fit()`) has a sensible default value that will work independently from the other input. That notion of "independent, sensible default value" is not ever possible for the initial values `p0` for `curve_fit`. Sensible initial values always depend on the data to be fit and the function modeling the data. Change the data values dramatically and `p0` must change. Change the definition of the function (or even the order of the arguments), and `p0` must change. There is not and cannot be a sensible default. Telling the user that `p0` is optional and can be `None` (as the current documentation does clearly state) is utterly and profoundly wrong. It is mathematically indefensible. It is horrible API design. It harms the integretiy of `scipy.optimize` to tell the user this. I don't know anything about the curve fit I will get in the end. So I
don't need to pretend that I know a good starting value.
It is not possible to do curve-fitting or non-linear least-squares minimization when you "don't know anything". The user MUST provide data to be modeled and MUST provide a function to model that data. It is "pretending" to think that this is sufficient. The user also must provide initial values. Maybe for 3 parameter functions, fine I can come up with an argument but
you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
It is not possible to do curve-fitting or non-linear least-squares minimization when one is completely ignorant of the the function.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark.
It is not an extra keyword argument. It is required input for the problem. `curve_fit()` is converting your feigned (or perhaps obstinate) ignorance to a set of starting values for you. But starting values simply cannot be independent of the input model function or input data. I am pretty confident that if we force this argument most of the people you
want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
I refuse to speculate on what "most users" will do, and I also refuse to accept your speculation on this without evidence. There a great many applications of curve-fitting for which `np.ones(n_variables)` and `np.zeros(n_variables)` will completely fail -- the fit will never move from the starting point. For the kinds of fits done in the programs I support, either of these would mean that essentially all fits would never move from its starting point, as at least one parameter being 0 or 1 would essentially always mean the model function was 0 over the full data range. But, again, I don't use `curve_fit` but other tools built on top of `scipy.optimize()`. Generally, the model function and data together imply sensible or at least guessable default values, but these cannot independent of model or data. In lmfit we do not permit the user to not supply default starting values -- default parameter values are `None` which will quickly raise a ValueError. I don't recall ever getting asked to change this. Because it should be obvious to all users that each parameter requires an initial value. Where appropriate and possible, we do provide methods for model functions to make initial guesses based on data. But again, the starting values always depend on model function and data. Default arguments *do* matter from a UX perspective when defaults are sensible. `curve_fit` has three required positional arguments: a model function (that must be annoying to have to provide), "y" data to be fit (well, I guess I have that), and "x" data. Why are those all required? Why not allow `func=None` to be a function that calculates a sine wave? Why not allow `y=None` to mean `np.ones(1000)`? Why not allow `x=None` to mean `np.arange(len(y))`? Wouldn't that be friendlier to the user?
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous?
Nope, just trying to help `curve_fit()` work better. --Matt
I appologize for the anecdotal nature of my experience, but I have used curve_fit() since it was first introduced and I just took a tour of almost all the times I've used it (well over 100 times, all told) and not once did I fail to provide initial parameters - even if I merely set them to all ones which it turns out I did more than I care to admit. This covers "production-level" code used by others as well as one-off throw-away jupyter notebooks. It may be that it never occured to me that p0 was optional. Anyways, I'm all for correctness and quality over backward-compatibility, even in the libraries and modules I consume, so I'd vote for making p0 required. -- Johann <https://sites.google.com/site/theodoregoetz/> On Wed, Jan 30, 2019 at 8:05 AM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi Ilhan,
On Tue, Jan 29, 2019 at 10:54 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
Well, I agree that `np.ones(n_variables)` is as good as any other choice. All default choices are horrible and not defensible.
Mathematically, algorithmically, and conceptually, initial values ARE REQUIRED for non-linear least squares optimization. The codes underlying `curve_fit()` (including `leastsq` and the mess that is `least_square`) do not permit the user to not provide initial values. The algorithms used simply do not make sense without initial values.
Programmatically, an optional keyword argument to a function, say with a default value of `None`, implies that there is a sensible default for that value. Thus, default fitting tolerances might be 1.e-7 (or, perhaps square root of machine precision) or the default value for "method to calculate jacobian" might be `None` to mean "calculate by finite difference". For these optional inputs, the function (say, `curve_fit()`) has a sensible default value that will work independently from the other input.
That notion of "independent, sensible default value" is not ever possible for the initial values `p0` for `curve_fit`. Sensible initial values always depend on the data to be fit and the function modeling the data. Change the data values dramatically and `p0` must change. Change the definition of the function (or even the order of the arguments), and `p0` must change. There is not and cannot be a sensible default.
Telling the user that `p0` is optional and can be `None` (as the current documentation does clearly state) is utterly and profoundly wrong. It is mathematically indefensible. It is horrible API design. It harms the integretiy of `scipy.optimize` to tell the user this.
I don't know anything about the curve fit I will get in the end. So I
don't need to pretend that I know a good starting value.
It is not possible to do curve-fitting or non-linear least-squares minimization when you "don't know anything". The user MUST provide data to be modeled and MUST provide a function to model that data. It is "pretending" to think that this is sufficient. The user also must provide initial values.
Maybe for 3 parameter functions, fine I can come up with an argument but
you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
It is not possible to do curve-fitting or non-linear least-squares minimization when one is completely ignorant of the the function.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark.
It is not an extra keyword argument. It is required input for the problem. `curve_fit()` is converting your feigned (or perhaps obstinate) ignorance to a set of starting values for you. But starting values simply cannot be independent of the input model function or input data.
I am pretty confident that if we force this argument most of the people
you want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
I refuse to speculate on what "most users" will do, and I also refuse to accept your speculation on this without evidence. There a great many applications of curve-fitting for which `np.ones(n_variables)` and `np.zeros(n_variables)` will completely fail -- the fit will never move from the starting point. For the kinds of fits done in the programs I support, either of these would mean that essentially all fits would never move from its starting point, as at least one parameter being 0 or 1 would essentially always mean the model function was 0 over the full data range.
But, again, I don't use `curve_fit` but other tools built on top of `scipy.optimize()`. Generally, the model function and data together imply sensible or at least guessable default values, but these cannot independent of model or data. In lmfit we do not permit the user to not supply default starting values -- default parameter values are `None` which will quickly raise a ValueError. I don't recall ever getting asked to change this. Because it should be obvious to all users that each parameter requires an initial value. Where appropriate and possible, we do provide methods for model functions to make initial guesses based on data. But again, the starting values always depend on model function and data.
Default arguments *do* matter from a UX perspective when defaults are sensible. `curve_fit` has three required positional arguments: a model function (that must be annoying to have to provide), "y" data to be fit (well, I guess I have that), and "x" data. Why are those all required? Why not allow `func=None` to be a function that calculates a sine wave? Why not allow `y=None` to mean `np.ones(1000)`? Why not allow `x=None` to mean `np.arange(len(y))`? Wouldn't that be friendlier to the user?
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous?
Nope, just trying to help `curve_fit()` work better.
--Matt
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit https://github.com/scipy/scipy/search?q=curve_fit&type=Issues As far as I know there was no complaint so far. Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice. That's just being pedantic about the API. Making the function api more annoying than it has to be is for me a wrong choice or even unpythonic if you are among that kind of crowd. A wrongness that matlab and alike software continuously annoy with their clunky UI. Users are not ours to educate. And if something is None then it means the code will make up some values not the correct ones; as clearly documented, some values. If you have better ones provide them. Having said that, I am getting a lot of "horrible, utterly wrong, obstinate, disservice" etc. that makes me uncomfortable and for me it's past beyond the discussion. I am not good at interwebz so I'd better stop here. We are talking about a simple function argument being required or optional that is essentially a made-up array. Thus I would like to reserve these words for more important occasions It's just a python library I am contributing to so I don't want to be involved in this particular issue any more. Since I am clearly biased on one side, I leave it to other members to decide, I am fine with any outcome. Best, On Wed, Jan 30, 2019 at 5:05 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi Ilhan,
On Tue, Jan 29, 2019 at 10:54 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
Well, I agree that `np.ones(n_variables)` is as good as any other choice. All default choices are horrible and not defensible.
Mathematically, algorithmically, and conceptually, initial values ARE REQUIRED for non-linear least squares optimization. The codes underlying `curve_fit()` (including `leastsq` and the mess that is `least_square`) do not permit the user to not provide initial values. The algorithms used simply do not make sense without initial values.
Programmatically, an optional keyword argument to a function, say with a default value of `None`, implies that there is a sensible default for that value. Thus, default fitting tolerances might be 1.e-7 (or, perhaps square root of machine precision) or the default value for "method to calculate jacobian" might be `None` to mean "calculate by finite difference". For these optional inputs, the function (say, `curve_fit()`) has a sensible default value that will work independently from the other input.
That notion of "independent, sensible default value" is not ever possible for the initial values `p0` for `curve_fit`. Sensible initial values always depend on the data to be fit and the function modeling the data. Change the data values dramatically and `p0` must change. Change the definition of the function (or even the order of the arguments), and `p0` must change. There is not and cannot be a sensible default.
Telling the user that `p0` is optional and can be `None` (as the current documentation does clearly state) is utterly and profoundly wrong. It is mathematically indefensible. It is horrible API design. It harms the integretiy of `scipy.optimize` to tell the user this.
I don't know anything about the curve fit I will get in the end. So I
don't need to pretend that I know a good starting value.
It is not possible to do curve-fitting or non-linear least-squares minimization when you "don't know anything". The user MUST provide data to be modeled and MUST provide a function to model that data. It is "pretending" to think that this is sufficient. The user also must provide initial values.
Maybe for 3 parameter functions, fine I can come up with an argument but
you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
It is not possible to do curve-fitting or non-linear least-squares minimization when one is completely ignorant of the the function.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark.
It is not an extra keyword argument. It is required input for the problem. `curve_fit()` is converting your feigned (or perhaps obstinate) ignorance to a set of starting values for you. But starting values simply cannot be independent of the input model function or input data.
I am pretty confident that if we force this argument most of the people
you want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
I refuse to speculate on what "most users" will do, and I also refuse to accept your speculation on this without evidence. There a great many applications of curve-fitting for which `np.ones(n_variables)` and `np.zeros(n_variables)` will completely fail -- the fit will never move from the starting point. For the kinds of fits done in the programs I support, either of these would mean that essentially all fits would never move from its starting point, as at least one parameter being 0 or 1 would essentially always mean the model function was 0 over the full data range.
But, again, I don't use `curve_fit` but other tools built on top of `scipy.optimize()`. Generally, the model function and data together imply sensible or at least guessable default values, but these cannot independent of model or data. In lmfit we do not permit the user to not supply default starting values -- default parameter values are `None` which will quickly raise a ValueError. I don't recall ever getting asked to change this. Because it should be obvious to all users that each parameter requires an initial value. Where appropriate and possible, we do provide methods for model functions to make initial guesses based on data. But again, the starting values always depend on model function and data.
Default arguments *do* matter from a UX perspective when defaults are sensible. `curve_fit` has three required positional arguments: a model function (that must be annoying to have to provide), "y" data to be fit (well, I guess I have that), and "x" data. Why are those all required? Why not allow `func=None` to be a function that calculates a sine wave? Why not allow `y=None` to mean `np.ones(1000)`? Why not allow `x=None` to mean `np.arange(len(y))`? Wouldn't that be friendlier to the user?
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous?
Nope, just trying to help `curve_fit()` work better.
--Matt
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Wed, Jan 30, 2019 at 1:48 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit
https://github.com/scipy/scipy/search?q=curve_fit&type=Issues
As far as I know there was no complaint so far. Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice. That's just being pedantic about the API. Making the function api more annoying than it has to be is for me a wrong choice or even unpythonic if you are among that kind of crowd. A wrongness that matlab and alike software continuously annoy with their clunky UI. Users are not ours to educate. And if something is None then it means the code will make up some values not the correct ones; as clearly documented, some values. If you have better ones provide them.
Having said that, I am getting a lot of "horrible, utterly wrong, obstinate, disservice" etc. that makes me uncomfortable and for me it's past beyond the discussion. I am not good at interwebz so I'd better stop here. We are talking about a simple function argument being required or optional that is essentially a made-up array. Thus I would like to reserve these words for more important occasions It's just a python library I am contributing to so I don't want to be involved in this particular issue any more. Since I am clearly biased on one side, I leave it to other members to decide, I am fine with any outcome.
Best,
Except that I am not a user of curve_fit, I agree completely with Ilhan. Actually, I think `ones` is one of the most reasonable default choices. Most cases are not in an arbitrary space of functions. The parameterization is often chosen to have nice positive values for interpretability. For example, I think that all (or almost all) parameters in scipy.stats distributions are positive or nonnegative (except for loc where any number is possible). Based on a brief browsing of recent stackoverflow questions, it looks like there are many possible problems with curve_fit which is an inherent problem with nonlinear optimization in general. But I think that specifying starting values if the results don't look nice, should be an obvious solution to users (especially with improved docstring for curve_fit). Many other problems on stackoverflow seem to be that users don't use a good parameterization of their function. Starting values is just one possible source of problems, and a user needs to be willing to investigate those when the first try doesn't work. (*) In the rest of scipy.optimize (and in related functions in statsmodels) there is no default starting values also because there is no information about the number of parameters (or length or the parameter) available. curve_fit using inspect and args was designed for making automatic starting values possible. "If at first you don't succeed, try, try again." (I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.) Currently no user is prevented from specifying starting values. After the change everyone is forced to add this additional step, just because some users are surprised that nonlinear optimization doesn't always work (out of the box). Josef
On Wed, Jan 30, 2019 at 5:05 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi Ilhan,
On Tue, Jan 29, 2019 at 10:54 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
Well, I agree that `np.ones(n_variables)` is as good as any other choice. All default choices are horrible and not defensible.
Mathematically, algorithmically, and conceptually, initial values ARE REQUIRED for non-linear least squares optimization. The codes underlying `curve_fit()` (including `leastsq` and the mess that is `least_square`) do not permit the user to not provide initial values. The algorithms used simply do not make sense without initial values.
Programmatically, an optional keyword argument to a function, say with a default value of `None`, implies that there is a sensible default for that value. Thus, default fitting tolerances might be 1.e-7 (or, perhaps square root of machine precision) or the default value for "method to calculate jacobian" might be `None` to mean "calculate by finite difference". For these optional inputs, the function (say, `curve_fit()`) has a sensible default value that will work independently from the other input.
That notion of "independent, sensible default value" is not ever possible for the initial values `p0` for `curve_fit`. Sensible initial values always depend on the data to be fit and the function modeling the data. Change the data values dramatically and `p0` must change. Change the definition of the function (or even the order of the arguments), and `p0` must change. There is not and cannot be a sensible default.
Telling the user that `p0` is optional and can be `None` (as the current documentation does clearly state) is utterly and profoundly wrong. It is mathematically indefensible. It is horrible API design. It harms the integretiy of `scipy.optimize` to tell the user this.
I don't know anything about the curve fit I will get in the end. So I
don't need to pretend that I know a good starting value.
It is not possible to do curve-fitting or non-linear least-squares minimization when you "don't know anything". The user MUST provide data to be modeled and MUST provide a function to model that data. It is "pretending" to think that this is sufficient. The user also must provide initial values.
Maybe for 3 parameter functions, fine I can come up with an argument but
you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
It is not possible to do curve-fitting or non-linear least-squares minimization when one is completely ignorant of the the function.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark.
It is not an extra keyword argument. It is required input for the problem. `curve_fit()` is converting your feigned (or perhaps obstinate) ignorance to a set of starting values for you. But starting values simply cannot be independent of the input model function or input data.
I am pretty confident that if we force this argument most of the people
you want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
I refuse to speculate on what "most users" will do, and I also refuse to accept your speculation on this without evidence. There a great many applications of curve-fitting for which `np.ones(n_variables)` and `np.zeros(n_variables)` will completely fail -- the fit will never move from the starting point. For the kinds of fits done in the programs I support, either of these would mean that essentially all fits would never move from its starting point, as at least one parameter being 0 or 1 would essentially always mean the model function was 0 over the full data range.
But, again, I don't use `curve_fit` but other tools built on top of `scipy.optimize()`. Generally, the model function and data together imply sensible or at least guessable default values, but these cannot independent of model or data. In lmfit we do not permit the user to not supply default starting values -- default parameter values are `None` which will quickly raise a ValueError. I don't recall ever getting asked to change this. Because it should be obvious to all users that each parameter requires an initial value. Where appropriate and possible, we do provide methods for model functions to make initial guesses based on data. But again, the starting values always depend on model function and data.
Default arguments *do* matter from a UX perspective when defaults are sensible. `curve_fit` has three required positional arguments: a model function (that must be annoying to have to provide), "y" data to be fit (well, I guess I have that), and "x" data. Why are those all required? Why not allow `func=None` to be a function that calculates a sine wave? Why not allow `y=None` to mean `np.ones(1000)`? Why not allow `x=None` to mean `np.arange(len(y))`? Wouldn't that be friendlier to the user?
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous?
Nope, just trying to help `curve_fit()` work better.
--Matt
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Wed, Jan 30, 2019 at 2:27 PM <josef.pktd@gmail.com> wrote:
On Wed, Jan 30, 2019 at 1:48 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit
https://github.com/scipy/scipy/search?q=curve_fit&type=Issues
As far as I know there was no complaint so far. Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice. That's just being pedantic about the API. Making the function api more annoying than it has to be is for me a wrong choice or even unpythonic if you are among that kind of crowd. A wrongness that matlab and alike software continuously annoy with their clunky UI. Users are not ours to educate. And if something is None then it means the code will make up some values not the correct ones; as clearly documented, some values. If you have better ones provide them.
Having said that, I am getting a lot of "horrible, utterly wrong, obstinate, disservice" etc. that makes me uncomfortable and for me it's past beyond the discussion. I am not good at interwebz so I'd better stop here. We are talking about a simple function argument being required or optional that is essentially a made-up array. Thus I would like to reserve these words for more important occasions It's just a python library I am contributing to so I don't want to be involved in this particular issue any more. Since I am clearly biased on one side, I leave it to other members to decide, I am fine with any outcome.
Best,
Except that I am not a user of curve_fit, I agree completely with Ilhan.
Actually, I think `ones` is one of the most reasonable default choices. Most cases are not in an arbitrary space of functions. The parameterization is often chosen to have nice positive values for interpretability. For example, I think that all (or almost all) parameters in scipy.stats distributions are positive or nonnegative (except for loc where any number is possible).
Based on a brief browsing of recent stackoverflow questions, it looks like there are many possible problems with curve_fit which is an inherent problem with nonlinear optimization in general. But I think that specifying starting values if the results don't look nice, should be an obvious solution to users (especially with improved docstring for curve_fit). Many other problems on stackoverflow seem to be that users don't use a good parameterization of their function. Starting values is just one possible source of problems, and a user needs to be willing to investigate those when the first try doesn't work. (*)
In the rest of scipy.optimize (and in related functions in statsmodels) there is no default starting values also because there is no information about the number of parameters (or length or the parameter) available. curve_fit using inspect and args was designed for making automatic starting values possible.
"If at first you don't succeed, try, try again."
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
This finally reminded me that I do have a similar example with default starting values, although with fmin and not leastsq. scipy had around 94 distributions when I started, and maybe around 65 (IIRC) continuous distributions with a fit method. This was too many to go through every distribution individually. Essentially the only information available in general is the number of parameters. The way I worked initially was to start with some generic defaults, and then work my way through the failing cases. Nice cases like normal and similar work out of the box (scale=1 is actually a good choice for default, mean=1 is arbitrary but no problem). Later we added fitstart for individual distributions with reasonable starting values that can be inferred from the data. Some distribution don't have a well behaved loglikelihood functions, and AFAIK, they still don't work "out of the box". Each stage of refinement requires more work for those cases that have failed all previous stages. However, the number of cases that are left is shrinking so the heavy work is mostly for nasty cases. (It's pretty much the same in statsmodels. We have a large number of models and very simple default starting parameters. In some cases or for some datasets this works fine. Other cases failed or still fail and I spent several months overall to improve starting values and numerical stability for those, not always with full success. But I don't "waste" that time on cases that work fine out of the box, i.e. with simple starting values.) Josef
Currently no user is prevented from specifying starting values. After the change everyone is forced to add this additional step, just because some users are surprised that nonlinear optimization doesn't always work (out of the box).
Josef
On Wed, Jan 30, 2019 at 5:05 PM Matt Newville <newville@cars.uchicago.edu> wrote:
Hi Ilhan,
On Tue, Jan 29, 2019 at 10:54 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
The problem I have with this is that there really is not an option to "try automatic first". There is "try `np.ones(n_variables)` first". This, or any other value, is really not a defensible choice for starting values. Starting values always depend on the function used and the data being fit.
Why not? 1.s are as good as any other choice.
Well, I agree that `np.ones(n_variables)` is as good as any other choice. All default choices are horrible and not defensible.
Mathematically, algorithmically, and conceptually, initial values ARE REQUIRED for non-linear least squares optimization. The codes underlying `curve_fit()` (including `leastsq` and the mess that is `least_square`) do not permit the user to not provide initial values. The algorithms used simply do not make sense without initial values.
Programmatically, an optional keyword argument to a function, say with a default value of `None`, implies that there is a sensible default for that value. Thus, default fitting tolerances might be 1.e-7 (or, perhaps square root of machine precision) or the default value for "method to calculate jacobian" might be `None` to mean "calculate by finite difference". For these optional inputs, the function (say, `curve_fit()`) has a sensible default value that will work independently from the other input.
That notion of "independent, sensible default value" is not ever possible for the initial values `p0` for `curve_fit`. Sensible initial values always depend on the data to be fit and the function modeling the data. Change the data values dramatically and `p0` must change. Change the definition of the function (or even the order of the arguments), and `p0` must change. There is not and cannot be a sensible default.
Telling the user that `p0` is optional and can be `None` (as the current documentation does clearly state) is utterly and profoundly wrong. It is mathematically indefensible. It is horrible API design. It harms the integretiy of `scipy.optimize` to tell the user this.
I don't know anything about the curve fit I will get in the end. So I
don't need to pretend that I know a good starting value.
It is not possible to do curve-fitting or non-linear least-squares minimization when you "don't know anything". The user MUST provide data to be modeled and MUST provide a function to model that data. It is "pretending" to think that this is sufficient. The user also must provide initial values.
Maybe for 3 parameter functions, fine I can come up with an argument but
you surely don't expect me to know the starting point if I am fitting a 7 parameter func involving esoteric structure. At that point I am completely ignorant about anything about this function. So not knowing where to start is not due to my noviceness about the tools but because by definition. My search might even turn out to be convex so initial value won't matter.
It is not possible to do curve-fitting or non-linear least-squares minimization when one is completely ignorant of the the function.
Currently `curve_fit` converts `p0=None` to `np.ones(n_variables)` without warning or explanation. Again, I do not use `curve_fit()` myself. I find several aspects of it unpleasant.
It is documented in the p0 argument docs. I am using this function quite often. That's why I don't like extra required arguments. It's annoying to enter some random array just to please the API where I know that I am just taking a shot in the dark.
It is not an extra keyword argument. It is required input for the problem. `curve_fit()` is converting your feigned (or perhaps obstinate) ignorance to a set of starting values for you. But starting values simply cannot be independent of the input model function or input data.
I am pretty confident that if we force this argument most of the people
you want to educate will enter np.zeros(n). Then they will get an even weirder error then they'll try np.ones(n) but misremember n then they get another error to remember the func parameter number which has already trippped up twice. This curve_fit function is one of those functions that you don't run just once and be done with it but over and over again until you give up or satisfied. Hence defaults matter a lot from a UX perspective. "If you have an initial value in mind fine enter it otherwise let me do my thing" is much better than "I don't care about your quick experiment give me some values or I will keep tripping up".
I refuse to speculate on what "most users" will do, and I also refuse to accept your speculation on this without evidence. There a great many applications of curve-fitting for which `np.ones(n_variables)` and `np.zeros(n_variables)` will completely fail -- the fit will never move from the starting point. For the kinds of fits done in the programs I support, either of these would mean that essentially all fits would never move from its starting point, as at least one parameter being 0 or 1 would essentially always mean the model function was 0 over the full data range.
But, again, I don't use `curve_fit` but other tools built on top of `scipy.optimize()`. Generally, the model function and data together imply sensible or at least guessable default values, but these cannot independent of model or data. In lmfit we do not permit the user to not supply default starting values -- default parameter values are `None` which will quickly raise a ValueError. I don't recall ever getting asked to change this. Because it should be obvious to all users that each parameter requires an initial value. Where appropriate and possible, we do provide methods for model functions to make initial guesses based on data. But again, the starting values always depend on model function and data.
Default arguments *do* matter from a UX perspective when defaults are sensible. `curve_fit` has three required positional arguments: a model function (that must be annoying to have to provide), "y" data to be fit (well, I guess I have that), and "x" data. Why are those all required? Why not allow `func=None` to be a function that calculates a sine wave? Why not allow `y=None` to mean `np.ones(1000)`? Why not allow `x=None` to mean `np.arange(len(y))`? Wouldn't that be friendlier to the user?
But this behavior strikes me as utterly wrong and a disservice to the scipy ecosystem. I do not think that a documentation change is sufficient.
Maybe a bit overzealous?
Nope, just trying to help `curve_fit()` work better.
--Matt
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes. This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API. What do you think? Best regards, Stéfan
On Wed, Jan 30, 2019 at 3:25 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes.
This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API.
What do you think?
I fully agree with that part. I think docstrings and tutorial are the places to "educate" users. I pointed out early in this thread, that the current documentation does not have examples with starting values and does not emphasize their importance. Also more explicit warnings on failure would be an obvious and not very intrusive ex-post reminder, IMO. I'm mainly arguing that forcing users to come up with random or meaningful starting values up-front is not in the (initial) "spirit" of curve_fit. Josef
Best regards, Stéfan
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Wed, Jan 30, 2019 at 2:26 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes.
This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API.
What do you think?
Well, I'm not sure that agreement here should be the sole driver for what scipy developers do. There will be disagreements in design philosophy, and someone needs to be willing and able to make decisions in such situations. I do not know who is making such decisions or reviewing changes in `scipy.optimize`, but it appears to me that this has suffered for awhile, leaving conceptual, interface, and organizational messes. I thought I would try to help by cleaning up one of the most egregious and simplest of these. The documentation for `curve_fit` does currrently state that `p0=None` is converted to `np.ones(n_variables)`. It appears that some view this as sufficient and that these folks view some automated assignment of initial values is useful, even while acknowledging that it cannot be correct in general. The argument for requiring initial values might be summarized as "initial values are actually required". The argument against might be summarized as "we don't want to change the current behavior". Anyway, I am perfectly willing to lose this argument (I do not use `curve_fit` myself, and do not feel compelled to support its use), and the decision is not mine to make. I do hope someone sensible is making these decisions. Cheers, --Matt
On Thu, Jan 31, 2019 at 9:30 AM Matt Newville <newville@cars.uchicago.edu> wrote:
On Wed, Jan 30, 2019 at 2:26 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes.
This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API.
What do you think?
Well, I'm not sure that agreement here should be the sole driver for what scipy developers do. There will be disagreements in design philosophy, and someone needs to be willing and able to make decisions in such situations. I do not know who is making such decisions or reviewing changes in `scipy.optimize`, but it appears to me that this has suffered for awhile, leaving conceptual, interface, and organizational messes. I thought I would try to help by cleaning up one of the most egregious and simplest of these.
The documentation for `curve_fit` does currrently state that `p0=None` is converted to `np.ones(n_variables)`. It appears that some view this as sufficient and that these folks view some automated assignment of initial values is useful, even while acknowledging that it cannot be correct in general.
The argument for requiring initial values might be summarized as "initial values are actually required". The argument against might be summarized as "we don't want to change the current behavior".
Anyway, I am perfectly willing to lose this argument (I do not use `curve_fit` myself, and do not feel compelled to support its use), and the decision is not mine to make. I do hope someone sensible is making these decisions.
I was only the stats reviewer for curve_fit, and never had any real stake in the API. Looking at the last heavily discussed change in curve_fit that I was involved in, I found this https://github.com/scipy/scipy/pull/3098#issuecomment-29837264 and a few followup comments. Josef
Cheers,
--Matt _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Thu, Jan 31, 2019 at 6:30 AM Matt Newville <newville@cars.uchicago.edu> wrote:
On Wed, Jan 30, 2019 at 2:26 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes.
This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API.
What do you think?
Well, I'm not sure that agreement here should be the sole driver for what scipy developers do. There will be disagreements in design philosophy, and someone needs to be willing and able to make decisions in such situations. I do not know who is making such decisions or reviewing changes in `scipy.optimize`, but it appears to me that this has suffered for awhile, leaving conceptual, interface, and organizational messes. I thought I would try to help by cleaning up one of the most egregious and simplest of these.
The documentation for `curve_fit` does currrently state that `p0=None` is converted to `np.ones(n_variables)`. It appears that some view this as sufficient and that these folks view some automated assignment of initial values is useful, even while acknowledging that it cannot be correct in general.
The argument for requiring initial values might be summarized as "initial values are actually required". The argument against might be summarized as "we don't want to change the current behavior".
Anyway, I am perfectly willing to lose this argument (I do not use `curve_fit` myself, and do not feel compelled to support its use), and the decision is not mine to make. I do hope someone sensible is making these decisions.
To answer this "who makes the decisions" question: we usually try to get consensus on any decision. Preferably by everyone who speaks up, otherwise by all the core developers. If that doesn't happen (not often), then in most cases people step out of the discussion and leave the decision to the one or two core devs who worked in that area most. This discussion is a rare example of there being two sides of the argument, both having merit, and it's not really converging. The final decision options are: 1. The SciPy BDFL (Pauli) makes the final decision. 2. We can't get to an agreement and the BDFL doesn't decide, so the status quo wins. For now we should go ahead with improving the docs (Stefan's suggestions), and then if Pauli wants to make a decision that would be great. Otherwise we'll stick with just the doc improvements. Cheers, Ralf
On Sat, Feb 2, 2019 at 8:55 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Thu, Jan 31, 2019 at 6:30 AM Matt Newville <newville@cars.uchicago.edu> wrote:
On Wed, Jan 30, 2019 at 2:26 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Wed, 30 Jan 2019 14:27:29 -0500, josef.pktd@gmail.com wrote:
(I'm strongly in favor of trying "defaults" first, and if that doesn't work, then dig into or debug likely candidates. in loose analogy of test driven development instead of up-front design.)
It seems unlikely that we will reach full agreement in this thread, given the differing experiences and philosophies at play. But, that's probably OK if we can all agree to modify the documentation to be clearer about the risks of the preset values for p0, how to select better values, and how to handle failure modes.
This won't 100% address Matt's concerns, but it will go a long way to keeping users out of trouble, without having to make breaking changes to the API.
What do you think?
Well, I'm not sure that agreement here should be the sole driver for what scipy developers do. There will be disagreements in design philosophy, and someone needs to be willing and able to make decisions in such situations. I do not know who is making such decisions or reviewing changes in `scipy.optimize`, but it appears to me that this has suffered for awhile, leaving conceptual, interface, and organizational messes. I thought I would try to help by cleaning up one of the most egregious and simplest of these.
The documentation for `curve_fit` does currrently state that `p0=None` is converted to `np.ones(n_variables)`. It appears that some view this as sufficient and that these folks view some automated assignment of initial values is useful, even while acknowledging that it cannot be correct in general.
The argument for requiring initial values might be summarized as "initial values are actually required". The argument against might be summarized as "we don't want to change the current behavior".
Anyway, I am perfectly willing to lose this argument (I do not use `curve_fit` myself, and do not feel compelled to support its use), and the decision is not mine to make. I do hope someone sensible is making these decisions.
To answer this "who makes the decisions" question: we usually try to get consensus on any decision. Preferably by everyone who speaks up, otherwise by all the core developers. If that doesn't happen (not often), then in most cases people step out of the discussion and leave the decision to the one or two core devs who worked in that area most. This discussion is a rare example of there being two sides of the argument, both having merit, and it's not really converging. The final decision options are: 1. The SciPy BDFL (Pauli) makes the final decision. 2. We can't get to an agreement and the BDFL doesn't decide, so the status quo wins.
For now we should go ahead with improving the docs (Stefan's suggestions), and then if Pauli wants to make a decision that would be great. Otherwise we'll stick with just the doc improvements.
Cheers, Ralf
Hi, I'd suggest the following action plan: 1. add a big prominent note to the docstring of curve_fit, to recommend setting initial values explicitly; 2. link from the curve_fit docstring to lmfit, and recommend that lmfit is used for anything more complicated than one-off quick fitting exercise; 3. add to the tutorial walk-through of using curve_fit and an eqiuvalent invocation of least_squares. This should discuss a simple problem (current examples in the least_squares docstring may be intimidating for new users); 4. add a worked example of lmfit usage to the scipy cookbook (I seem to remember that there are/were examples, but these looked somewhat clunky); 5. In curve_fit, check whether the popt equals p0, and raise a RuntimeWarning (or a subclass). The last part is a code change, and it assumes that the most common failure mode is `popt == p0` (not sure if that's the case for problems with bounds). 6. A possible enhancement could be to overload the p0 parameter to accept a dictionary with keys being the names of the variables. I.e., `curve_fit(lambda x, a, b: a*x**b, x, y, p0={'a': 1, 'b': 2})` Cheers, Evgeni
On Wed, Jan 30, 2019 at 10:48 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit
https://github.com/scipy/scipy/search?q=curve_fit&type=Issues
As far as I know there was no complaint so far.
StackOverflow is likely a better place to look for evidence: https://stackoverflow.com/questions/27097957/my-use-of-scipy-curve-fit-does-... https://stackoverflow.com/questions/23828226/scipy-curve-fit-does-not-seem-t... https://stackoverflow.com/questions/53509550/error-in-scipy-curve-fit-for-mo...
Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice.
That is certainly not the case. It is impossible to do so generally for every arbitrary nonconvex problem, which is why we can't automate it, but there are tons of problems where one has a reasonable idea from priors, examining the plotted data, or heuristic algorithms (e.g. finding the peak location and amplitude with a peak finder). -- Robert Kern
That is certainly not the case. It is impossible to do so generally for every arbitrary nonconvex problem, which is why we can't automate it, but
For SO questions, 1st one agreed, 2nd one look at the given values. 3rd one have a look at the first line and also the comments. That's actually a counter argument. there are tons of problems where one has a reasonable idea from priors, examining the plotted data, or heuristic algorithms (e.g. finding the peak location and amplitude with a peak finder). Yes exactly my point. If you have a better idea put it in. p0 is not going to reject your guess but insist on 1.s. But I can give you easily tons of problems you can not give a proper guess. That is simply impossible because that defeats the purpose of why we use this tool. On Wed, Jan 30, 2019 at 9:25 PM Robert Kern <robert.kern@gmail.com> wrote:
On Wed, Jan 30, 2019 at 10:48 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit
https://github.com/scipy/scipy/search?q=curve_fit&type=Issues
As far as I know there was no complaint so far.
StackOverflow is likely a better place to look for evidence:
https://stackoverflow.com/questions/27097957/my-use-of-scipy-curve-fit-does-...
https://stackoverflow.com/questions/23828226/scipy-curve-fit-does-not-seem-t...
https://stackoverflow.com/questions/53509550/error-in-scipy-curve-fit-for-mo...
Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice.
That is certainly not the case. It is impossible to do so generally for every arbitrary nonconvex problem, which is why we can't automate it, but there are tons of problems where one has a reasonable idea from priors, examining the plotted data, or heuristic algorithms (e.g. finding the peak location and amplitude with a peak finder).
-- Robert Kern _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
One reason why I think this has become a surprisingly controversial point is that on one hand curve_fit is a convenience function and it works in a lot of cases as-is (and breaking APIs is difficult), but on the other hand the guesswork involved in the default p0 choice goes against "in the face of ambiguity, refuse the temptation to guess" of native python. There are probably plenty of other cases where scipy makes pragmatic choices which might otherwise seem "unpythonic", and in case of such a high-level front-end I believe we're better off with the current API, given that the documentation is made clear enough when it comes to dangers of not passing a p0. But I can see why this arbitrary choice for an important input parameter can be seen as a problem. András On Wed, Jan 30, 2019 at 9:39 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
For SO questions, 1st one agreed, 2nd one look at the given values. 3rd one have a look at the first line and also the comments. That's actually a counter argument.
That is certainly not the case. It is impossible to do so generally for every arbitrary nonconvex problem, which is why we can't automate it, but there are tons of problems where one has a reasonable idea from priors, examining the plotted data, or heuristic algorithms (e.g. finding the peak location and amplitude with a peak finder).
Yes exactly my point. If you have a better idea put it in. p0 is not going to reject your guess but insist on 1.s. But I can give you easily tons of problems you can not give a proper guess. That is simply impossible because that defeats the purpose of why we use this tool.
On Wed, Jan 30, 2019 at 9:25 PM Robert Kern <robert.kern@gmail.com> wrote:
On Wed, Jan 30, 2019 at 10:48 AM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a frequent user of this function. I am also occasionally teaching control theory and I use this also professionally, I provided a proper use case as a "user". You already admitted that you don't use it. So excuse me when I say that I have more experience than you about this function API (please read this twice; function API not the underlying theory). How can I even provide evidence on this completely subjective matter? Here is all the issues related to curve_fit
https://github.com/scipy/scipy/search?q=curve_fit&type=Issues
As far as I know there was no complaint so far.
StackOverflow is likely a better place to look for evidence:
https://stackoverflow.com/questions/27097957/my-use-of-scipy-curve-fit-does-... https://stackoverflow.com/questions/23828226/scipy-curve-fit-does-not-seem-t... https://stackoverflow.com/questions/53509550/error-in-scipy-curve-fit-for-mo...
Hence that means it is really not that big of a deal that would grant such tone. I don't know what to add other than what I already provided. You cannot make educated guesses about initial points on nonconvex searches. As you mentioned, we are as blind as np.ones(n) choice.
That is certainly not the case. It is impossible to do so generally for every arbitrary nonconvex problem, which is why we can't automate it, but there are tons of problems where one has a reasonable idea from priors, examining the plotted data, or heuristic algorithms (e.g. finding the peak location and amplitude with a peak finder).
-- Robert Kern _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Dear Matt, Joseph, all, Here’s my 2 cents.
On 24. Jan 2019, at 17:26, josef.pktd@gmail.com wrote:
IMO, a warning and better documentation would be more appropriate. https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi...> does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
I think a better docstring for curve_fit would be good, mentioning the importance of specifying p0 and that the default is ones. Also +1 to add a mention of lmfit from the scipy.optimize docs somewhere, recommending it to users as a high-level package that directly builds on scipy.optimise providing extra functionality and conveniences. -1 on making the change to require p0 to be passed, and starting a deprecation process to eventually give an error on this in a few years. There are cases and existing scripts that work just fine with the current default. IMO they should continue to work (even without warning) in scipy 1.x just like they do in the currently scipy 1.1.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
-1 on adding complex logic and guessing and trying code for p0 to curve_fit. scipy.optimize is pretty low-level, adding this kind of “convenience” which is hard to explain and document and maintain seems a bit out of place to me. Also any change in how p0 is chosen likely will change results that come out for some use cases, so for the same reason mentioned above (stability within the scipy 1.x series) I’m -1 to change this. Christoph
As I commented on the GitHub issue, we are not supposed to educate users through keywords and default values in a passive manner. A clear "You need to choose p0 wisely to use this tool" message would be enough to be warned. On Fri, Jan 25, 2019 at 12:48 PM Christoph Deil < deil.christoph@googlemail.com> wrote:
Dear Matt, Joseph, all,
Here’s my 2 cents.
On 24. Jan 2019, at 17:26, josef.pktd@gmail.com wrote:
IMO, a warning and better documentation would be more appropriate.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fi... does not show an example with starting values. curve_fit could issue a warning if p0 is not specified, or warn if convergence fails and p0 was not specified.
I think a better docstring for curve_fit would be good, mentioning the importance of specifying p0 and that the default is ones. Also +1 to add a mention of lmfit from the scipy.optimize docs somewhere, recommending it to users as a high-level package that directly builds on scipy.optimise providing extra functionality and conveniences.
-1 on making the change to require p0 to be passed, and starting a deprecation process to eventually give an error on this in a few years. There are cases and existing scripts that work just fine with the current default. IMO they should continue to work (even without warning) in scipy 1.x just like they do in the currently scipy 1.1.
I think it should also be possible to improve the default starting values, e.g. if the function fails or if bounds are provided.
-1 on adding complex logic and guessing and trying code for p0 to curve_fit. scipy.optimize is pretty low-level, adding this kind of “convenience” which is hard to explain and document and maintain seems a bit out of place to me. Also any change in how p0 is chosen likely will change results that come out for some use cases, so for the same reason mentioned above (stability within the scipy 1.x series) I’m -1 to change this.
Christoph _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Matt, On Wed, 23 Jan 2019 19:24:57 -0600, Matt Newville wrote:
First, I apologize in advance if this sounds un-appreciative of the efforts made in scipy and scipy.optimize.
Nothing you said in your email sounds unappreciative to me, for what it's worth.
Unfortunately, faced with no initial parameter values, `curve_fit` silently chooses initial values. It doesn't try to make an informed decision, it simply chooses '1.0', which can easily be so far off as to prevent a solution from being found. When this happens, `curve_fit` gives no information to the user of what the problem is. Indeed it allows initial values to not be set, giving the impression that they are not important. This impression is wrong: initial values are always important. `curve_fit` is mistaken in having a default starting value.
I've made a Pull Request at https://github.com/scipy/scipy/pull/9701 to fix this misbehavior, so that `curve_fit()` requires starting values. It was suggested there that this topic should be discussed here. I'm happy to do so. It was suggested in the github Issue that forcing the user to give initial values was "annoying". It was also suggested that a deprecation cycle would be required. I should say that I don't actually use `curve_fit()` myself, I'm just trying to help make this commonly used routine be completely wrong less often.
Thank you for highlighting this issue. In general, magic is to be avoided, where possible. In this case, I would suggest: 1. For one or two releases, changing the function to warn when no default is specified. Then, turn that warning into an error, that also explains good rules of thumb for calculating initial values (and mentions the 'auto' method in #2 below). 2. Provide an easy mechanism to get sensible defaults. I.e., allow specific values to be specified, but also allow estimation methods, such as 'auto' (just an example, this would probably be more specific) or 'ones' (to restore old behavior, and to prevent existing users from having to modify their code drastically). Stéfan
On Thu, 24 Jan 2019 15:50:45 +1100, Andrew Nelson wrote:
In terms of proposed usage emitting warnings is relatively
straightforward. I guess p0 should become a new positional argument as part of this change (or some other name). How does one introduce a new positional argument whilst remaining back compatible? Using *args?
That's harder work; perhaps just keep it a kwd arg for now and make sure it gets specified? Soon, we'll have https://www.python.org/dev/peps/pep-3102/ ! Stéfan
Just one small remark: in my experience many users utilize curve_fit to fit linear models (models that are linear in the fit parameters, e.g. f(x) = a * exp(-x) + b * exp(-x^2)). In this case the linear least squares problem can be solved uniquely without providing initial values for a and b. Even though curve_fit employs a general nonlinear least squares solver it should be able to find the global minimum for virtually all starting values (although I have not comprehensively tested it). On the other hand when the model is truly nonlinear the initial values of curve_fit fail with very high probability. One may argue that using curve_fit for such cases is not the right choice. However, scipy does not provide a convenient interface such as curve_fit for linear models (as far as I know). And users who are not aware that a nonlinear least squares solver requires good initial values may also not be aware of the difference between linear and nonlinear models. (When I teach I regularly encounter the misconception that models that are nonlinear in the independent variable, e.g., x, require a nonlinear solver). Long story short: when deprecating the initial choice for p0 it may be worthwile to also consider providing a convenient interface for lsq_linear that is similar to curve_fit and does not require initial values. Cheers Nils
Five years ago I submitted a linear fitting routine for SciPy so that people could have an alternative to curve_fit when they wanted to fit a linear model. Here it is: https://github.com/djpine/linfit Some people felt it was redundant with linear regression routines in scipy.stats. However, the linear regression routine in scipy.stats does not offer the same capabilities as curve_fit for including error estimation (data weighting) so it may not meet the needs of users who typically use curve_fit. The above routine is meant to remedy that. pine@nyu.edu On Thu, Jan 24, 2019 at 7:04 PM Nils Geib <nilsc.becker@gmail.com> wrote:
Just one small remark: in my experience many users utilize curve_fit to fit linear models (models that are linear in the fit parameters, e.g. f(x) = a * exp(-x) + b * exp(-x^2)). In this case the linear least squares problem can be solved uniquely without providing initial values for a and b. Even though curve_fit employs a general nonlinear least squares solver it should be able to find the global minimum for virtually all starting values (although I have not comprehensively tested it). On the other hand when the model is truly nonlinear the initial values of curve_fit fail with very high probability.
One may argue that using curve_fit for such cases is not the right choice. However, scipy does not provide a convenient interface such as curve_fit for linear models (as far as I know). And users who are not aware that a nonlinear least squares solver requires good initial values may also not be aware of the difference between linear and nonlinear models. (When I teach I regularly encounter the misconception that models that are nonlinear in the independent variable, e.g., x, require a nonlinear solver).
Long story short: when deprecating the initial choice for p0 it may be worthwile to also consider providing a convenient interface for lsq_linear that is similar to curve_fit and does not require initial values.
Cheers Nils _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Fri, 25 Jan 2019 06:21:33 +0900, David J Pine wrote:
Five years ago I submitted a linear fitting routine for SciPy so that people could have an alternative to curve_fit when they wanted to fit a linear model. Here it is:
If we implement a higher-level `optimize`-like API, it may be worth switching out between implementations such as these. Interesting reading along these lines: "Data analysis recipes: Fitting a model to data" by Hogg et al. https://arxiv.org/abs/1008.4686 which also discusses rejection of data outliers. SciPy doesn't currently have any rejection methods included, as far as I am aware. Stéfan
participants (13)
-
Andras Deak -
Andrew Nelson -
Christoph Deil -
David J Pine -
Evgeni Burovski -
Ilhan Polat -
Johann Goetz -
josef.pktd＠gmail.com -
Matt Newville -
Nils Geib -
Ralf Gommers -
Robert Kern -
Stefan van der Walt