[SciPy-User] scipy.optimize.minimize constraint optimization

[Matt Newville]

Kevin,


On Fri, Nov 29, 2013 at 3:29 AM, Kevin Kunzmann <kevinkunzmann at gmx.net> wrote:
> Hi,
>
> I have a question concerning constraint optimization with scipy. After
> setting up my problem using scipy.optimize.minimize and the SLSQP method
> the procedure failed to converge and the final iteration was a bit
> awkward. So I inspected the iteration steps closely and the problem
> seems to be, that the bounds / constraints are not respected in every
> step. This blows up some function evaluations, as they are only well
> defined for the feasible set. Is there a way of doing general purpose,
> nonlinear, constraint optimization, that respects the feasible set at
> ALL times in scipy (or at least in python?). The Matlab implementations
> of fmincon using sqp and trust-region-reflective do implement that
> feature
> http://www.mathworks.de/de/help/optim/ug/writing-constraints.html#br9p_ry but
> i would rather stick with python if possible...
> If that feature is available already in scipy an comment on that in the
> help would be superb :)
>
> thank you,
>
> Kevin
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


You might find the lmfit module (http://newville.github.io/lmfit-py/)
useful.  It takes a slightly different approach, wrapping many of the
scipy optimization routines (with .leastsq() and .fmin() being the
best tested) to be written as a function of Parameter objects instead
of floating point numbers.   By using a richer object, user-defined
bounds can be set for Parameter (each Parameter has a .min and .max
attribute, and the Minuit re-casting as David suggested, and as
implemented in Python by JJ Helmus, is used).  In addition, you can
fix (ie, not vary) a Parameter used by the objective function without
having to rewrite the objective function.  You can also set
constraints, including inequality constraints, by writing an
expression to be evaluated for a Parameters value.  While the
.leastsq() function provides an automatically generated estimate of
uncertainties and correlations between Parameters, the lmfit module
also provides convenient functions to do a brute force exploration of
confidence intervals.

Finally, the module also makes it easy (no recoding of objective
functions) to switch fitting algorithms.  Some people use the
Nelder-Mead algorithm (fmin) to find the best solution, the running
leastsq starting with that solution to get estimates of the
uncertainties.

Hope that helps,

--Matt Newville <newville at cars.uchicago.edu>