Mailman 3 January 2010 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

np.savetxt: apply patch in enhancement ticket 1079 to add headers?
by Skipper Seabold June 8, 2010

June 8, 2010

Hi all, I am assuming that this is ok to request via the list... Could we discuss or could someone apply the patch in enhancement ticket 1079? http://projects.scipy.org/numpy/ticket/1079 I needed this functionality recently, and this is a quick and easy fix that may have been overlooked. There is also another enhancement request about this here: http://projects.scipy.org/numpy/ticket/1236 The only thing that I can think of that might need to be added is a test to see that the header length is the same as the number of columns, but really that might just be up to the user to supply the right headers. It might also be nice to have a header = True, that uses the field names for a structured array, but I can live without that. Cheers, Skipper

3 11

namespace recommendation in scipy tutorial and docs for np.emath
by josef.pktd＠gmail.com May 15, 2010

May 15, 2010

I'm going through some scipy tickets and I found http://projects.scipy.org/scipy/ticket/882 I don't know much about the old usage of mixing (mixed up) namespaces. But I think it would be good if someone could check the namespace convention/recommendation in the basic scipy tutorial http://docs.scipy.org/doc/scipy/reference/tutorial/basic.html When I tried to figure out the difference between scipy.* and numpy.*, I also found that the functions in numpy.emath (numpy.scipy) have good docstrings, but they are not included in the docs (at least I don't find them), because http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.emath.rst/#n… is still a stub page without function links. http://docs.scipy.org/doc/numpy/reference/routines.emath.html Josef

2 2

compilation with fort77
by Thomas Tanner Feb. 4, 2010

Feb. 4, 2010

Hello, I'm trying to port scipy 0.7.1 to the Maemo platform (Linux/ARM). There is no Fortran compiler for this platform so I have to use fort77 (and f2c) to compile numpy and scipy. However, f2c does not seem to be 100% Fortran77 compatible (or may scipy is not?). I get (only :) two compilation errors. The first one is a trivial type conversion and fixed by my attached patch. the full build output can be found at https://garage.maemo.org/builder/fremantle/python-scipy_0.7.1-1maemo1/armel… I don't know how to fix the second error: compiling Fortran sources Fortran f77 compiler: /usr/bin/f77 -g -Wall -fno-second-underscore -fPIC -O2 -funroll-loops compile options: '-Ibuild/src.linux-armv5tel-2.5 -I/usr/lib/python2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c' f77:f77: scipy/stats/mvndst.f mvnun: Error on line 76: Declaration error for rho: adjustable dimension on non-argument Error on line 76: Declaration error for infin: adjustable dimension on non-argument Error on line 76: Declaration error for stdev: adjustable dimension on non-argument Error on line 76: Declaration error for nlower: adjustable dimension on non-argument Error on line 76: Declaration error for nupper: adjustable dimension on non-argument Error on line 24: wr_ardecls: nonconstant array size Error on line 24: wr_ardecls: nonconstant array size Error on line 24: wr_ardecls: nonconstant array size Error on line 24: wr_ardecls: nonconstant array size Error on line 24: wr_ardecls: nonconstant array size mvndst: mvndfn: entry mvndnt: mvnlms: covsrt: dkswap: rcswp: dkbvrc: dksmrc: mvnphi: phinvs: bvnmvn: bvu: mvnuni: the problem is that fort77 cannot deal with those variable sized input arguments: SUBROUTINE mvnun(d, n, lower, upper, means, covar, maxpts, & abseps, releps, value, inform) ... integer n, d, infin(d), maxpts, inform, tmpinf double precision lower(d), upper(d), releps, abseps, & error, value, stdev(d), rho(d*(d-1)/2), & covar(d,d), & nlower(d), nupper(d), means(d,n), tmpval integer i, j Could some Fortran expert please help me to make it fort77 compatible? Thanks for your help! best regards, -- Thomas Tanner ------ email: tanner(a)gmx.de GnuPG: 1024/5924D4DD --- python-scipy-0.7.1.orig/scipy/sparse/linalg/isolve/iterative/GMRESREVCOM.f.src +++ python-scipy-0.7.1/scipy/sparse/linalg/isolve/iterative/GMRESREVCOM.f.src @@ -106,6 +106,7 @@ $ NEED1, NEED2 <_t> <xdot=sdot,ddot,cdotc,zdotc> <_t> toz + <_t> CNORM <rt> BNRM2, RNORM, TOL, $ <rc=s,d,sc,dz>NRM2, $ <rc>APPROXRES @@ -266,8 +267,9 @@ * RNORM = <rc>NRM2( N, WORK( 1,V ), 1 ) toz = ONE/RNORM + CNORM = RNORM CALL <_c>SCAL( N, toz, WORK( 1,V ), 1 ) - CALL <_c>ELEMVEC( 1, N, RNORM, WORK( 1,S ) ) + CALL <_c>ELEMVEC( 1, N, CNORM, WORK( 1,S ) ) * * DO 50 I = 1, RESTRT i = 1

8 19

Ball Tree code updated (ticket 1048)
by Jake VanderPlas Feb. 1, 2010

Feb. 1, 2010

Hello, I have had comments from a few people over the last two months on the Ball Tree code that I submitted (ticket 1048). I cleaned up the code a bit and posted the changes on the tracker. Any other comments would be appreciated! -Jake

2 2

bug in signal.lsim2
by Ryan Krauss Jan. 28, 2010

Jan. 28, 2010

I believe I have discovered a bug in signal.lsim2. I believe the short attached script illustrates the problem. I was trying to predict the response of a transfer function with a pure integrator: g G = ------------- s(s+p) to a finite width pulse. lsim2 seems to handle the step response just fine, but says that the pulse response is exactly 0.0 for the entire time of the simulation. Obviously, this isn't the right answer. I am running scipy 0.7.0 and numpy 1.2.1 on Ubuntu 9.04, but I also have the same problem on Windows running 0.7.1 and 1.4.0. Thanks, Ryan

1 1

Difference between polynomial.trimcoef and trimseq
by David Goldsmith Jan. 24, 2010

Jan. 24, 2010

Is the only difference that which is in the Notes of trimseq: "Do [sic?] not lose the type info if the sequence contains unknown objects"? DG

3 10

Re: [SciPy-dev] SciPy Documentation
by Peter Demarest Jan. 22, 2010

Jan. 22, 2010

Gael, I am able to log in to the front page of the wiki using the using name: pdemarest Peter Peter Demarest demarest(a)alum.wpi.edu C: 281-954-4243 Message: 2 > Date: Thu, 21 Jan 2010 07:18:14 +0100 > From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> > Subject: Re: [SciPy-dev] SciPy Documentation > To: SciPy Developers List <scipy-dev(a)scipy.org> > Message-ID: <20100121061814.GA21316(a)phare.normalesup.org> > Content-Type: text/plain; charset=utf-8 > > On Wed, Jan 20, 2010 at 09:58:53PM -0600, Peter Demarest wrote: > > I would like to start contributing to the SciPy documentation. ?My > user > > name on the wiki is: pdemarest > > I thought I would start taking a look at the docstings in the optimize > > module. > > Hi, > > Thanks for your interest in the documentation. > > I can't find you in the list of users. How did you create your user? You > must go to http://docs.scipy.org/numpy/Front%20Page/ and click on the > 'log in' link on the top right. > > Cheers, > > Ga? > > >

3 2

Genetic Algorithms module in scikits.learn
by Fabian Pedregosa Jan. 22, 2010

Jan. 22, 2010

Hello. I'm trying to maintain and release scikits.learn[1], and for historical reasons there's a module for genetic algorithms in this package. As the module does not quite fit the package's thematic, has references to scipy.ga (which has moved to scipy.sandbox and is no longer maintained) and there are already some great genetic algorithm frameworks in Python, I'm rather inclined to remove this from scikits.learn. If there are no objections, I'll remove that module from scikits.learn, but just wanted to say that I've uploaded the code to github [2] in case someone finds it useful or is willing to maintain. Thanks, ~fabian [1] http://scikit-learn.sourceforge.net/ [2] http://github.com/fseoane/ga

2 3

SciPy Documentation
by Peter Demarest Jan. 21, 2010

Jan. 21, 2010

Hello, I would like to start contributing to the SciPy documentation. My user name on the wiki is: pdemarest I thought I would start taking a look at the docstings in the optimize module. Thanks Peter Demarest demarest(a)alum.wpi.edu C: 281-954-4243

2 1