[SciPy-User] Least-squares fittings with bounds: why is scipy not up to the task?

[David Baddeley]

>From a pure performance perspective, you're probably going to be best setting your bounds by variable substitution (particularly if they're only single-ended - x**2 is cheap) - you really don't want to have the for loops, dictionary lookups and conditionals that lmfit introduces for it's bounds checking inside your objective function.

I think a high level wrapper that permitted bounds, an unadulterated goal function, and setting which parameters to fit, but also retained much of the raw speed of leastsq could be accomplished with some clever on the fly code generation (maybe also using Sympy to automatically derive the Jacobian). Would make an interesting project ...

David


________________________________
 From: Eric Emsellem <eemselle at eso.org>
To: Matthew Newville <matt.newville at gmail.com> 
Cc: scipy-user at scipy.org; scipy-user at googlegroups.com 
Sent: Friday, 9 March 2012 12:17 PM
Subject: Re: [SciPy-User] Least-squares fittings with bounds: why is scipy not up to the task?
 


> Yes, see https://github.com/newville/lmfit-py,  which does everything 
> you ask for, and a bit more, with the possible exception of "being 
> included in scipy".   For what its worth, I work with Mark Rivers 
> (who's no longer actively developing Python), and our group is full of 
> IDL users who are very familiar with Markwardt's implementation.
>
> The lmfit-py version uses scipy.optimize.leastsq(), which uses MINPACK 
> directly, so has the advantage of not being implemented in pure IDL or 
> Python. It is definitely faster than mpfit.py.
>
> With lmfit-py, one writes a python function-to-minimize that takes a 
> list of Parameters instead of the array of floating point variables 
> that scipy.optimize.leastsq() uses. Each Parameter can be freely 
> varied of fixed, have upper and/or lower bounds placed on them, or be 
> written as algebraic expressions of other Parameters.   Uncertainties 
> in varied Parameters and correlations between Parameters are estimated 
> using the same "scaled covariance" method as used in 
> scipy.optimize.curve_fit().   There is limited support for 
> optimization methods other than scipy.optimize.leastsq(), but I don't 
> find these methods to be very useful for the kind of fitting  problems 
> I normally see, so support for them may not be perfect.
>
> Whether this gets included into scipy is up to the scipy developers. 
> I'd be happy to support this module within scipy or outside scipy.
> I have no doubt that improvements could be made to lmfit.py.   If you 
> have suggestion, I'd be happy to hear them.

looks great! I'll have a go at this, as mentioned in my previous post. I 
believe that leastsq is probably the fastest anyway (according to the 
test Adam mentioned to me today) so this could be it. I'll make a test 
and compare it with mpfit (for the specific case I am thinking of, I am 
optimising over ~10^5-6 points with ~90 parameters...).

thanks again for this, and I'll try to report on this (if relevant) asap.

Eric
_______________________________________________
SciPy-User mailing list
SciPy-User at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20120308/d0db5c7c/attachment.html>