Re: [SciPy-User] fmin_bfgs failing on simple problem
I've recreated this problem on my machine. It appears to be relating to floating point precision, and only occurs when the x<0 return value is extremely large.
def f2(x,badval=1e20): ... if x<=0: ... return badval ... else: ... return x+1/x ... ans=1. badval=1e20 while abs(ans-1.)<1e-5: ... badval = 10*badval ... ans = fmin_bfgs(f2,10,args=(badval,))
The while loop ends when badval is 1.0e301, with ans as nan and the following error message: Warning: Desired error not necessarily achieved due to precision loss. Current function value: nan Iterations: 45 Function evaluations: 1989 Gradient evaluations: 654 But it works with badval=1.0e300. Would it hurt your problem to use 1e200 as your "nearly infinite" value instead of 1.79769313e+308? Or am I missing something here? On Tue, 2012-04-17 at 13:56 -0500, John Salvatier wrote:
Hmm, that's too bad. Looks like there was a big refactoring of linesearch.py ( https://github.com/scipy/scipy/blob/master/scipy/optimize/linesearch.py ) a couple of years ago ( https://github.com/scipy/scipy/commit/fefef2d73200d535b95ce0f21dcfe122301a96... )
Thanks for the help Nathaniel :)
On Tue, Apr 17, 2012 at 11:16 AM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Apr 17, 2012 at 7:14 PM, John Salvatier <jsalvati@u.washington.edu> wrote: > Well that's good news. I have scipy .9.0b1, what version do you have?
Less good news: I have 0.8.0 :-)
- N
> On Tue, Apr 17, 2012 at 11:13 AM, Nathaniel Smith <njs@pobox.com> wrote: >> >> On Tue, Apr 17, 2012 at 6:35 PM, John Salvatier >> <jsalvati@u.washington.edu> wrote: >> > Hi all! >> > >> > I am having a problem with the fmin_bfgs solver that's surprising to me. >> > Here's the toy problem I've set up: >> > >> > from scipy.optimize import fmin_bfgs, fmin_ncg >> > from numpy import * >> > import numpy as np >> > >> > def f(x ): >> > if x < 0: >> > return 1.79769313e+308 >> > else : >> > return x + 1./x >> > >> > >> > xs = fmin_bfgs(f, array( [10.]), retall = True) >> > >> > The solver returns [nan] as the solution. >> > >> > The problem is designed to be stiff: between 0 and 1, it slopes upward >> > to >> > infinity but between 1 and infinity, it slopes up at a slope of 1. Left >> > of 0 >> > the function has a "nearly infinite" value. If bfgs encounters a value >> > that's larger than the current value, it should try a different step >> > size, >> > no? Why does fmin_bfgs fail in this way? >> >> I can't reproduce this (on my computer it converges to 0.99999992), >> but have you tried making that < into a <=? The divide-by-zero at f(0) >> might be making it freak out. >> >> -- Nathaniel >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User@scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-user > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User@scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-user > _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org
Thanks Kathleen! Good idea. It probably wouldn't hurt in my case. The problem also comes up if you use "inf" too, though which I think would be more common. In any case, I've filed a bug report: http://projects.scipy.org/scipy/ticket/1644 John On Wed, Apr 18, 2012 at 9:39 AM, Kathleen M Tacina < Kathleen.M.Tacina@nasa.gov> wrote:
** I've recreated this problem on my machine. It appears to be relating to floating point precision, and only occurs when the x<0 return value is extremely large.
def f2(x,badval=1e20): ... if x<=0: ... return badval ... else: ... return x+1/x ... ans=1. badval=1e20 while abs(ans-1.)<1e-5: ... badval = 10*badval ... ans = fmin_bfgs(f2,10,args=(badval,))
The while loop ends when badval is 1.0e301, with ans as nan and the following error message: Warning: Desired error not necessarily achieved due to precision loss. Current function value: nan Iterations: 45 Function evaluations: 1989 Gradient evaluations: 654
But it works with badval=1.0e300. Would it hurt your problem to use 1e200 as your "nearly infinite" value instead of 1.79769313e+308?
Or am I missing something here?
On Tue, 2012-04-17 at 13:56 -0500, John Salvatier wrote:
Hmm, that's too bad. Looks like there was a big refactoring of linesearch.py ( https://github.com/scipy/scipy/blob/master/scipy/optimize/linesearch.py ) a couple of years ago ( https://github.com/scipy/scipy/commit/fefef2d73200d535b95ce0f21dcfe122301a96... )
Thanks for the help Nathaniel :)
On Tue, Apr 17, 2012 at 11:16 AM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Apr 17, 2012 at 7:14 PM, John Salvatier <jsalvati@u.washington.edu> wrote:
Well that's good news. I have scipy .9.0b1, what version do you have?
Less good news: I have 0.8.0 :-)
- N
On Tue, Apr 17, 2012 at 11:13 AM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Apr 17, 2012 at 6:35 PM, John Salvatier <jsalvati@u.washington.edu> wrote:
Hi all!
I am having a problem with the fmin_bfgs solver that's surprising to
me.
Here's the toy problem I've set up:
from scipy.optimize import fmin_bfgs, fmin_ncg from numpy import * import numpy as np
def f(x ): if x < 0: return 1.79769313e+308 else : return x + 1./x
xs = fmin_bfgs(f, array( [10.]), retall = True)
The solver returns [nan] as the solution.
The problem is designed to be stiff: between 0 and 1, it slopes upward to infinity but between 1 and infinity, it slopes up at a slope of 1. Left of 0 the function has a "nearly infinite" value. If bfgs encounters a value that's larger than the current value, it should try a different step size, no? Why does fmin_bfgs fail in this way?
I can't reproduce this (on my computer it converges to 0.99999992), but have you tried making that < into a <=? The divide-by-zero at f(0) might be making it freak out.
-- Nathaniel _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
participants (2)
-
John Salvatier -
Kathleen M Tacina