New subject: [SciPy-dev] SLSQP Constrained Optimizer Status

Dec. 12, 2007

Rob Falck <robfalck <at> gmail.com> writes:

> 
> I'm currently implementing the Sequential Least Squares Quadratic
Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy.The Fortran
code being wrapped with F2PY is here:  
> http://www.netlib.org/toms/733  (its use within Scipy has been
cleared)SLSQP provides for bounds on the independent variables, as well as
equality and inequality constraint functions, which is a capability that
doesn't exist in 
> scipy.optimize.Currently the code works, although the constraint normals
are being generated by approximation.  I'm working on a way to pass these
in.  I think the most elegant way will be a single function that returns the
matrix of constraint normals.
> For a demonstration of what the code can do, here is an optimization of
f(x,y) = 2xy + 2x - x**2 - 2y**2Example 14.2 in Chapra & Canale gives the
maximum as x=2.0, y=1.0.The unbounded optimization tests find this
solution.  As expected, its faster when derivatives are provided rather than
approximated.
> Unbounded optimization. Derivatives approximated.Elapsed time:
1.45792961121 msResults [[1.9999999515712266, 0.99999996181577444],
-1.9999999999999984, 4, 0, 'Optimization terminated successfully.']
> Unbounded optimization.  Derivatives provided.Elapsed time: 1.03211402893
msResults [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984,
4, 0, 'Optimization terminated successfully.']The following example uses an
equality constraint to find the optimal when x=y.  Bound optimization. 
Derivatives approximated.Elapsed time: 1.384973526 msResults
[[0.99999996845920858, 0.99999996845920858
> ], -0.99999999999999889, 4, 0, 'Optimization terminated
successfully.']I've tried to conform to the syntax used by the other
optimizers in scipy.optimize.  The function definition and doc string are below.
> If anyone is interested in testing it out, let me know.def fmin_slsqp(
func, x0 , eqcons=[], f_eqcons=None, ieqcons=[],
f_ieqcons=None,                bounds = [], fprime = None,
fprime_cons=None,args = (),
>                 iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0,
                epsilon = _epsilon ):    """    Minimize a function using
Sequential Least SQuares Programming    
>     Description:        Python interface function for the SLSQP
Optimization subroutine         originally implemented by Dieter Kraft.   
    Inputs:        func         - Objective function (in the form func(x,
*args))
>         x0           - Initial guess for the independent variable(s).
        eqcons       - A list of functions of length n such that
                       eqcons[j](x0,*args) == 0.0 in a successfully
optimized problem
>         f_eqcons     - A function of the form f_eqcons(x, *args) that
returns an                       array in which each element must equal 0.0
in a                        successfully optimized problem.  If f_eqcons is
>                        specified, eqcons is ignored.                   
        ieqcons      - A list of functions of length n such that
                       ieqcons[j](x0,*args) >= 0.0 in a successfully
optimized problem
>         f_ieqcons    - A function of the form f_ieqcons(x0, *args) that
returns an                       array in which each element must be greater
or equal to                       0.0 in a successfully optimized problem. 
If f_ieqcons is
>                        specified, ieqcons is
ignored.                                                       bounds      
- A list of tuples specifying the lower and upper bound for each
                       independent variable [ (xl0, xu0), (xl1, xu1), ...]
>         fprime       - A function that evaluates the partial derivatives
of func         fprime_cons  - A function of the form f(x, *args) that
returns the                       m by n array of constraint normals.  If
not provided,
>                        the normals will be approximated. Equality
constraint                       normals precede inequality constraint
normals.         args         - A sequence of additional arguments passed to
func and fprime
>         iter         - The maximum number of iterations (int)       
acc          - Requested accuracy (float)        iprint       - The
verbosity of fmin_slsqp.                       iprint <= 0 : Silent operation
>                        iprint == 1 : Print summary upon completion
(default)                       iprint >= 2 : Print status of each iterate
and summary        full_output  - If 0, return only the minimizer of func
(default).  Otherwise,
>                        output final objective function and summary
information.        epsilon      - The step size for finite-difference
derivative estimates.                         Outputs: ( x, { fx, gnorm,
its, imode, smode })
>         x            - The final minimizer of func.        fx           -
The final value of the objective function.        its          - The number
of iterations.        imode        - The exit mode from the optimizer, as an
integer.
>         smode        - A string describing the exit mode from the
optimizer.             Exit modes are defined as follows:        -1 :
Gradient evaluation required (g & a)         0 : Optimization terminated
successfully.
>          1 : Function evaluation required (f & c)         2 : Number of
equality constraints larger than number of independent variables         3 :
More than 3*n iterations in LSQ subproblem         4 : Inequality
constraints incompatible
>          5 : Singular matrix E in LSQ subproblem         6 : Singular
matrix C in LSQ subproblem         7 : Rank-deficient equality constraint
subproblem HFTI         8 : Positive directional derivative for linesearch
>          9 : Iteration limit exceeded-- - Rob Falck
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev <at> scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
> 

I have a question regarding the SLSQP Code. If inexact linesearch is used,
which L1-penalty function is used?

I´m looking forward to your answer,
Monika

SLSQP Constrained Optimizer Status

Rob Falck

dmitrey

Rob Falck

dmitrey

Rob Falck

Rob Falck

dmitrey

Rob Falck

Rob Falck

dmitrey

Rob Falck

Rob Falck

dmitrey

Rob Falck

dmitrey

Monika

Pauli Virtanen

tags

participants (4)