[SciPy-User] Least-squares fittings with bounds: why is scipy not up to the task?

[Matthew Newville]

David, 

On Thursday, March 8, 2012 9:14:23 PM UTC-6, David Baddeley wrote:
>
> From a pure performance perspective, you're probably going to be best 
> setting your bounds by variable substitution (particularly if they're only 
> single-ended - x**2 is cheap) - you really don't want to have the for 
> loops, dictionary lookups and conditionals that lmfit introduces for it's 
> bounds checking inside your objective function.
>

>From a performance perspective in which dictionary lookups and additions in 
the wrapper that lmfit puts on an objective function are considered high, I 
think you would probably not want the objective function written in python 
to begin with, but just use Fortran (or C).   Much of scipy presupposes 
that one must include development time (here, or writing and manipulating 
the objective function) into "performance".     

So, for some trivial cases, one can easily change the parameter from, say  
"x, with a minimum value of 0" to "x**2", but then one also has to change 
the objective function and re-map the estimated uncertainties in the 
parameters every time the bounds might be changed.   These would be changes 
that the end-user would have to do.....  Or they could use lmfit which does 
this automatically, if at a slight performance cost compared to having no 
bounds set.    Is that performance hit important?  I doubt it. 

The original question, and several follow-up messages, point to mpfit.py.  
As I'm sure you're aware, this implements the Levenberg-Marquardt algorithm 
**in python**.   And people use it because it provides a convenient way to 
set bounds.   So, like much of scipy and python, sometimes pure performance 
is not the main requirement.   Now, mpfit.py is slow (and is a translation 
of MINPACK from fortran to IDL to python-with-Numeric),  OTOH, lmfit uses 
scipy.optimize.leastsq(), which calls into the fortran version of MINPACK, 
and so does have improved performance compared to mpfit.py.

I think a high level wrapper that permitted bounds, an unadulterated goal 
> function, and setting which parameters to fit, but also retained much of 
> the raw speed of leastsq could be accomplished with some clever on the fly 
> code generation (maybe also using Sympy to automatically derive the 
> Jacobian). Would make an interesting project ..
>

 Sounds great..  Let me know when it's ready and I'll be happy to give it a 
try.

--Matt

>  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20120309/d61ff5dd/attachment.html>