[SciPy-User] Optimization problem using fmin_slsqp
Qing Yu
yuqingpsy at gmail.com
Thu Apr 10 11:22:10 EDT 2014
Hi Shawn,
Thanks for your detailed explanation! I'm actually not sure how to check
whether residual() is spitting out nan; and also the fitting result (a~0.1,
mu~1, sigma~1.57, b~0.06) I got without bounds still fell in the bounds I
defined [(None,None),(0,pi),(0,None),(None,None)].
On the other hand I've tried the 'minimize' function. I mentioned it in my
reply to Matt's email and here it is:
> Hi Matt,
>
> Thanks so much! I've actually tried the 'minimize' function in
> scipy.optimize. There are several constrained optimization methods in
> 'minimize' and among them 'SLSQP' seems to be equivalent to the method
used
> in fmin_slsqp; I got similar bad fitting results using these two
functions.
> Meanwhile, using the 'L-BFGS-B' method in 'minimize', I can get similar
good
> fitting results as 'leastsq'. But I'm not sure how these two algorithms
> differ.
So yes I've tried the 'L-BFGS-B' method and it is working well. Also, using
the 'lmfit' package as Matt suggested, I can get same good fitting results
as 'L-BFGS-B'. The 'SLSQP' method, however, provides similar bad fitting
results as 'fmin_slsqp'. I think that is because they are using the same
algorithm (Sequential Least SQuares Programming).
Best,
Qing
'
""
On Wed, Apr 9, 2014 at 9:16 PM, Yuxiang Wang <yw5aj at virginia.edu> wrote:
> Hi Qing,
>
> I was thinking that your function residual() may give nan and this
> would make the fitting uncontrollable.
>
> I think that this may be the case because in your previous emails you
> mentioned the error:
> Iteration limit exceeded (Exit mode 9)
> Current function value: nan
> Iterations: 101
> Function evaluations: 1606
> Gradient evaluations: 101
> The current function value is a "nan".
>
> It might simply be that in the bounds you defined, the algorithm does
> not converge due to the nan; if you get rid of the bounds, it will go
> elsewhere for sure but that does not give what you want.
>
> Could you check that to see whether residual() function is spitting out
> nan?
>
>
> PS: I heard that minimize() should always be used and fmin_slsqp etc.
> should be deprecated... I personally do prefer minimize because then I
> can easily try some other algorithms like L-BFGS-B (this and SLSQP are
> my favorite two!).
>
> -Shawn
>
> On Wed, Apr 9, 2014 at 11:24 AM, Qing Yu <yuqingpsy at gmail.com> wrote:
> > Hi Shawn,
> >
> > Yes I was actually using a 'try...except...' command in my script. It's
> > simply because I want an output when the fitting procedure fails. Here's
> the
> > part of the script:
> >
> > try:
> > out, fx, its, imode, smode =
> >
> fmin_slsqp(residual,a0,args=(x,y),bounds=[(None,None),(0,pi),(0,None),(None,None)],
> > full_output=True)
> > except:
> > out = [nan, nan, nan, nan]
> >
> > But I suppose I won't get the nan output unless the fitting fails?
> >
> > Thanks!
> >
> >
> > Best,
> > Qing
> >
> > On Wed, Apr 9, 2014 at 11:00 AM, Yuxiang Wang <yw5aj at virginia.edu>
> wrote:
> >>
> >> Hi Qing,
> >>
> >> In your description I see a line:
> >>
> >> Current function value: nan
> >>
> >> Is your function outputting a np.nan for certain inputs? That might be
> >> bad for the algorithm to work. Could you check that?
> >>
> >> -Shawn
> >>
> >> On Mon, Apr 7, 2014 at 4:56 PM, Qing Yu <yuqingpsy at gmail.com> wrote:
> >> > Hi Matt,
> >> >
> >> > Thanks so much for your help! I've now changed the residual function
> as
> >> > below:
> >> >
> >> > def residual(a,x,y):
> >> > return sum((y-f(x,a))**2)
> >> >
> >> > the script seems to be working after the change, but I have another
> >> > problem
> >> > now:
> >> >
> >> > because I've set the upper and lower bounds of the parameters in
> >> > fmin_slsqp
> >> > (and that is why I use fmin_slsqp instead of leastsq):
> >> > bounds=[(None,None),(0,pi),(0,None),(None,None)], if I keep the
> bounds,
> >> > I
> >> > get the error:
> >> > Iteration limit exceeded (Exit mode 9)
> >> > Current function value: nan
> >> > Iterations: 101
> >> > Function evaluations: 1606
> >> > Gradient evaluations: 101
> >> > I tried changing iter to 10000, the error still existed.
> >> > If I delete the bound option, I can get the successful output.
> However,
> >> > the
> >> > fitted values are quite different from what I get using lsqcurvefit in
> >> > matlab. Moreover, if I plot the data, the model fit is indeed worse
> >> > using
> >> > fmin_slsqp. I'm not sure if the difference is caused by the missing
> >> > bounds.
> >> > But the outputs from fmin_slsqp did not go out of bounds anyway. Does
> >> > anyone
> >> > have any idea why I could not set the bounds in fmin_slsqp?
> >> >
> >> > Thanks so much for any help!
> >> >
> >> > Best,
> >> > Qing
> >> >
> >> >
> >> > On Mon, Apr 7, 2014 at 3:48 PM, Qing Yu <yuqingpsy at gmail.com> wrote:
> >> >>
> >> >> Dear SciPy experts,
> >> >>
> >> >> I'm using the fmin_slsqp function in scipy.optimize to fit my data, I
> >> >> first defined the function f and the residual of fitting, and then
> used
> >> >> actual data x and y to fit the curve. However, I encountered an error
> >> >> that I
> >> >> could not explain.
> >> >> Here's a short version of my script:
> >> >>
> >> >> def f(x,a):
> >> >> return a[0]*exp((a[2]**2)*cos(x-a[1])-1)+a[3]
> >> >> def residual(a,x,y):
> >> >> return y-f(x,a)
> >> >>
> >> >> x=[0,pi/6,pi/3,pi/2,2*pi/3,5*pi/6]
> >> >> y=[0.17,0.36,0.61,0.38,0.17,0.16]
> >> >> a0=[1,pi/3,pi/2,0]
> >> >> out, fx, its, imode, smode =
> >> >>
> >> >>
> fmin_slsqp(residual,a0,args=(x,y),bounds=[(None,None),(0,pi),(0,None),(None,None)],
> >> >> full_output=True)
> >> >>
> >> >> I ended up with the error: "failed in converting 8th argument `g' of
> >> >> _slsqp.slsqp to C/Fortran array." Does anyone have any idea of why
> this
> >> >> happened?
> >> >>
> >> >> Thanks so much in advance for any help!
> >> >>
> >> >> Best,
> >> >> Qing
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Graduate Student
> >> > Qing Yu
> >> > Department of Psychological and Brain Sciences
> >> > Dartmouth College
> >> > Hanover, New Hampshire 03755, US
> >> >
> >> > _______________________________________________
> >> > SciPy-User mailing list
> >> > SciPy-User at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >> >
> >>
> >>
> >>
> >> --
> >> Yuxiang "Shawn" Wang
> >> Gerling Research Lab
> >> University of Virginia
> >> yw5aj at virginia.edu
> >> +1 (434) 284-0836
> >> https://sites.google.com/a/virginia.edu/yw5aj/
> >> _______________________________________________
> >> SciPy-User mailing list
> >> SciPy-User at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-user
> >
> >
> >
> >
> > --
> > Graduate Student
> > Qing Yu
> > Department of Psychological and Brain Sciences
> > Dartmouth College
> > Hanover, New Hampshire 03755, US
> >
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
>
>
>
> --
> Yuxiang "Shawn" Wang
> Gerling Research Lab
> University of Virginia
> yw5aj at virginia.edu
> +1 (434) 284-0836
> https://sites.google.com/a/virginia.edu/yw5aj/
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
--
Graduate Student
Qing Yu
Department of Psychological and Brain Sciences
Dartmouth College
Hanover, New Hampshire 03755, US
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140410/fd556f87/attachment.html>
More information about the SciPy-User
mailing list