Nov. 21, 2013
4:59 p.m.
Rob Falck <robfalck <at> gmail.com> writes:
>
> I'm currently implementing the Sequential Least Squares Quadratic
Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy.The Fortran
code being wrapped with F2PY is here:
> http://www.netlib.org/toms/733 (its use within Scipy has been
cleared)SLSQP provides for bounds on the independent variables, as well as
equality and inequality constraint functions, which is a capability that
doesn't exist in
> scipy.optimize.Currently the code works, although the constraint normals
are being generated by approximation. I'm working on a way to pass these
in. I think the most elegant way will be a single function that returns the
matrix of constraint normals.
> For a demonstration of what the code can do, here is an optimization of
f(x,y) = 2xy + 2x - x**2 - 2y**2Example 14.2 in Chapra & Canale gives the
maximum as x=2.0, y=1.0.The unbounded optimization tests find this
solution. As expected, its faster when derivatives are provided rather than
approximated.
> Unbounded optimization. Derivatives approximated.Elapsed time:
1.45792961121 msResults [[1.9999999515712266, 0.99999996181577444],
-1.9999999999999984, 4, 0, 'Optimization terminated successfully.']
> Unbounded optimization. Derivatives provided.Elapsed time: 1.03211402893
msResults [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984,
4, 0, 'Optimization terminated successfully.']The following example uses an
equality constraint to find the optimal when x=y. Bound optimization.
Derivatives approximated.Elapsed time: 1.384973526 msResults
[[0.99999996845920858, 0.99999996845920858
> ], -0.99999999999999889, 4, 0, 'Optimization terminated
successfully.']I've tried to conform to the syntax used by the other
optimizers in scipy.optimize. The function definition and doc string are below.
> If anyone is interested in testing it out, let me know.def fmin_slsqp(
func, x0 , eqcons=[], f_eqcons=None, ieqcons=[],
f_ieqcons=None, bounds = [], fprime = None,
fprime_cons=None,args = (),
> iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0,
epsilon = _epsilon ): """ Minimize a function using
Sequential Least SQuares Programming
> Description: Python interface function for the SLSQP
Optimization subroutine originally implemented by Dieter Kraft.
Inputs: func - Objective function (in the form func(x,
*args))
> x0 - Initial guess for the independent variable(s).
eqcons - A list of functions of length n such that
eqcons[j](x0,*args) == 0.0 in a successfully
optimized problem
> f_eqcons - A function of the form f_eqcons(x, *args) that
returns an array in which each element must equal 0.0
in a successfully optimized problem. If f_eqcons is
> specified, eqcons is ignored.
ieqcons - A list of functions of length n such that
ieqcons[j](x0,*args) >= 0.0 in a successfully
optimized problem
> f_ieqcons - A function of the form f_ieqcons(x0, *args) that
returns an array in which each element must be greater
or equal to 0.0 in a successfully optimized problem.
If f_ieqcons is
> specified, ieqcons is
ignored. bounds
- A list of tuples specifying the lower and upper bound for each
independent variable [ (xl0, xu0), (xl1, xu1), ...]
> fprime - A function that evaluates the partial derivatives
of func fprime_cons - A function of the form f(x, *args) that
returns the m by n array of constraint normals. If
not provided,
> the normals will be approximated. Equality
constraint normals precede inequality constraint
normals. args - A sequence of additional arguments passed to
func and fprime
> iter - The maximum number of iterations (int)
acc - Requested accuracy (float) iprint - The
verbosity of fmin_slsqp. iprint <= 0 : Silent operation
> iprint == 1 : Print summary upon completion
(default) iprint >= 2 : Print status of each iterate
and summary full_output - If 0, return only the minimizer of func
(default). Otherwise,
> output final objective function and summary
information. epsilon - The step size for finite-difference
derivative estimates. Outputs: ( x, { fx, gnorm,
its, imode, smode })
> x - The final minimizer of func. fx -
The final value of the objective function. its - The number
of iterations. imode - The exit mode from the optimizer, as an
integer.
> smode - A string describing the exit mode from the
optimizer. Exit modes are defined as follows: -1 :
Gradient evaluation required (g & a) 0 : Optimization terminated
successfully.
> 1 : Function evaluation required (f & c) 2 : Number of
equality constraints larger than number of independent variables 3 :
More than 3*n iterations in LSQ subproblem 4 : Inequality
constraints incompatible
> 5 : Singular matrix E in LSQ subproblem 6 : Singular
matrix C in LSQ subproblem 7 : Rank-deficient equality constraint
subproblem HFTI 8 : Positive directional derivative for linesearch
> 9 : Iteration limit exceeded-- - Rob Falck
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>
I have a question regarding the SLSQP Code. If inexact linesearch is used,
which L1-penalty function is used?
I´m looking forward to your answer,
Monika