[SciPy-User] Catching warning and turning into errors for lmfit / scipy.optimize.leastsq

Fri Apr 5 13:34:39 EDT 2013

Thanks Matt.

It was solved it by putting a minimum bound to 'decay', which in this case,
"of course" always positive.

Thanks for pointing out how the warnings are generated and handled.

Best

Troels Emtekær Linnet

2013/4/5 Matt Newville <newville at cars.uchicago.edu>

> Hi Troels,
>
> On Fri, Apr 5, 2013 at 7:18 AM, Troels Emtekær Linnet <tlinnet at gmail.com>
> wrote:
> > Dear Scipy users.
> >
> > I haven't been dealing with Warning and Errors before, and need a little
> > help catching warnings to errors.
> >
> > I am making some tests on some dataset, which get worse and worse.
> >
> > I am fitting a decay model, to some intensities.
> >
> > When the data gets really bad, I start getting some Warnings.
> > Getting decay for 8
> > Getting decay for 6
> > Getting decay for 4
> > Getting decay for 2
> > TB.py:34: RuntimeWarning: overflow encountered in exp
> >   model = amp*exp(-decay*time)
> >
> > Function is:
> > def f_expdecay_lmfit(pars,time,data=None):
> >     amp = pars['amp'].value
> >     decay = pars['decay'].value
> >     model = amp*exp(-decay*time)
> >     if data is None:
> >         return model
> >     return (model-data)
> >
> > Call is:
> > try:
> >     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> > datY),method='leastsq')
> >     fitY = f_expdecay_lmfit(par,datX)
> > except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError) as
> e:
> >     print "Cannot fit expdecay for %s %s. Reason: %s"%(peak, peakname, e)
> >     save_the_data_as_null_or_flagged
> >
> > My problem is, that I don't know how to catch the warnings.
> > If I get a warning, I also want to make sure, that this fit is not saved.
> >
> > I have tried to use the warning package
> > import warnings
> > warnings.simplefilter('error')
> >
> > Then I get:
> > -------------------
> > Traceback (most recent call last):
> >   File "lmfit/minimizer.py", line 140, in __residual
> >     out = self.userfcn(self.params, *self.userargs, **self.userkws)
> >   File "/TB.py", line 33, in f_expdecay_lmfit
> >     model = amp*exp(-decay*time)
> > RuntimeWarning: overflow encountered in exp
> > Traceback (most recent call last):
> >   File "TB_fit.py", line 28, in <module>
> >     TB.getdecay(XXL,XXL['met'])
> >   File "/TB.py", line 268, in getdecay
> >     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> > datY),method='leastsq')
> >   File "lmfit/minimizer.py", line 498, in minimize
> >     fitter.leastsq()
> >   File "lmfit/minimizer.py", line 369, in leastsq
> >     lsout = scipy_leastsq(self.__residual, self.vars, **lskws)
> >   File "scipy/optimize/minpack.py", line 283, in leastsq
> >     gtol, maxfev, epsfcn, factor, diag)
> > minpack.error: Error occurred while calling the Python function named
> > __residual
> > --------------------------------------------
> > If I then rewrite the fitting function also.
> >
> > def f_expdecay_lmfit(pars,time,data=None):
> >     amp = pars['amp'].value
> >     decay = pars['decay'].value
> >     try:
> >         model = amp*exp(-decay*time)
> >     except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError)
> as
> > e:
> >         print "Cannot fit expdecay. Reason: %s"%(e)
> >     if data is None:
> >         return model
> >     return (model-data)
> >
> > I still get:
> > ------------------------
> > Cannot fit expdecay. Reason: overflow encountered in exp
> > Traceback (most recent call last):
> >   File "lmfit/minimizer.py", line 140, in __residual
> >     out = self.userfcn(self.params, *self.userargs, **self.userkws)
> >   File "/TB.py", line 39, in f_expdecay_lmfit
> >     return (model-data)
> > UnboundLocalError: local variable 'model' referenced before assignment
> > Traceback (most recent call last):
> >   File "TB_fit.py", line 28, in <module>
> >     TB.getdecay(XXL,XXL['met'])
> >   File "/TB.py", line 271, in getdecay
> >     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> > datY),method='leastsq')
> >   File "lmfit/minimizer.py", line 498, in minimize
> >     fitter.leastsq()
> >   File "lmfit/minimizer.py", line 369, in leastsq
> >     lsout = scipy_leastsq(self.__residual, self.vars, **lskws)
> >   File "scipy/optimize/minpack.py", line 283, in leastsq
> >     gtol, maxfev, epsfcn, factor, diag)
> > minpack.error: Error occurred while calling the Python function named
> > __residual
> > -----------------------
> >
> > I am not able to pass UnboundLocalError.
> >
> > I want to break out of the fitting rutine for warnings and errors, and
> save
> > that fit as null.
> >
> > Can someone help me to figure out how to fix this?
> >
> > Best
> > Troels
> >
> >
> >
> > Troels Emtekær Linnet
> > Ved kløvermarken 9, 1.th
> > 2300 København S
> > Mobil: +45 60210234
> >
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
>
>
> I believe the problem you're having is that the underlying fitting
> routine really needs to have a valid array returned to it.   When you
> get an RuntimeError exception because the exponentiation gives an
> overflow, the variable 'model' is not defined, and your return
> statements will cause another exception that the fitting routine can't
> handle.  You may find something like this to be helpful:
>
>      try:
>          model = amp*exp(-decay*time)
>      except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError)
> as e:
>          print "Cannot fit expdecay. Reason: %s"%(e)
>          return numpy.zeros_like(time)
>
> The other option is to figure why you're getting the overflows and
> prevent it.  It looks like you expect "time" and "decay" to be
> positive, and so "-decay*time" to be negative.  But numpy.exp()
> overflows when it's argument is > 700, which is probably a crazy value
> for you.   If "time" is positive-definite, and you expect "decay" to
> be positive, you could think about placing a lower bound on "decay" of
> "-max(time)/100.", which would allow "decay" to be slightly negative,
> but not so much so as to cause the overflow.   That would probably
> help avoid the problem in the first place.
>
> Hope that helps,
>
> --Matt
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20130405/5cee1029/attachment.html>