[SciPy-User] Least-squares fittings with bounds: why is scipy not up to the task?

[Jonathan Helmus]

I know I am jumping into this thread late and it has drifted into 
another topics but I have some code that others might be interested in.  
With all the discussion of bounded leastsq and variable substitution I 
recalled that I had a wrapped version of leastsq in a larger project 
that allows for min, max bound using the variable transformations that 
MINUIT uses.  I pulled out the necessary functions, refactored the code 
and made a github repo in case anyone is interested 
(https://github.com/jjhelmus/leastsqbound-scipy).  This might make a 
good jumping off point for a more complete bounded leastsq optimizer 
that David had in mind.

- Jonathan Helmus

David Baddeley wrote:
> From a pure performance perspective, you're probably going to be best 
> setting your bounds by variable substitution (particularly if they're 
> only single-ended - x**2 is cheap) - you really don't want to have the 
> for loops, dictionary lookups and conditionals that lmfit introduces 
> for it's bounds checking inside your objective function.
>
> I think a high level wrapper that permitted bounds, an unadulterated 
> goal function, and setting which parameters to fit, but also retained 
> much of the raw speed of leastsq could be accomplished with some 
> clever on the fly code generation (maybe also using Sympy to 
> automatically derive the Jacobian). Would make an interesting project ...
>
> David
>
> ------------------------------------------------------------------------
> *From:* Eric Emsellem <eemselle at eso.org>
> *To:* Matthew Newville <matt.newville at gmail.com>
> *Cc:* scipy-user at scipy.org; scipy-user at googlegroups.com
> *Sent:* Friday, 9 March 2012 12:17 PM
> *Subject:* Re: [SciPy-User] Least-squares fittings with bounds: why is 
> scipy not up to the task?
>
>
>
> > Yes, see https://github.com/newville/lmfit-py,  which does everything
> > you ask for, and a bit more, with the possible exception of "being
> > included in scipy".  For what its worth, I work with Mark Rivers
> > (who's no longer actively developing Python), and our group is full of
> > IDL users who are very familiar with Markwardt's implementation.
> >
> > The lmfit-py version uses scipy.optimize.leastsq(), which uses MINPACK
> > directly, so has the advantage of not being implemented in pure IDL or
> > Python. It is definitely faster than mpfit.py.
> >
> > With lmfit-py, one writes a python function-to-minimize that takes a
> > list of Parameters instead of the array of floating point variables
> > that scipy.optimize.leastsq() uses. Each Parameter can be freely
> > varied of fixed, have upper and/or lower bounds placed on them, or be
> > written as algebraic expressions of other Parameters.  Uncertainties
> > in varied Parameters and correlations between Parameters are estimated
> > using the same "scaled covariance" method as used in
> > scipy.optimize.curve_fit().  There is limited support for
> > optimization methods other than scipy.optimize.leastsq(), but I don't
> > find these methods to be very useful for the kind of fitting  problems
> > I normally see, so support for them may not be perfect.
> >
> > Whether this gets included into scipy is up to the scipy developers.
> > I'd be happy to support this module within scipy or outside scipy.
> > I have no doubt that improvements could be made to lmfit.py.  If you
> > have suggestion, I'd be happy to hear them.
>
> looks great! I'll have a go at this, as mentioned in my previous post. I
> believe that leastsq is probably the fastest anyway (according to the
> test Adam mentioned to me today) so this could be it. I'll make a test
> and compare it with mpfit (for the specific case I am thinking of, I am
> optimising over ~10^5-6 points with ~90 parameters...).
>
> thanks again for this, and I'll try to report on this (if relevant) asap.
>
> Eric
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org <mailto:SciPy-User at scipy.org>
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>