Mailman 3 June 2011 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

prepending underscores to private module names
by Ralf Gommers May 16, 2012

May 16, 2012

Hi, A while ago we discussed making a clear distinction between public and private modules by using underscores consistently, see http://thread.gmane.org/gmane.comp.python.scientific.devel/14936. I've done this for a few modules now, see https://github.com/rgommers/scipy/tree/underscores. Most files could simply be underscored, some other would conflict with extension modules so I also had to append _py to the name. Also I added a file doc/source/api.rst that states which modules are public (up for discussion of course). With google code search I checked how common it is to import from modules that are supposed to be private, for example "from scipy.optimize.optimize import fmin". It's not very common but does happen, so deprecation warnings should be put in. Before going further I'd like to check that this looks okay to people. What do you think? Cheers, Ralf <https://github.com/rgommers/scipy/tree/underscores>

4 4

studentized range approximations
by Roger Lew March 28, 2012

March 28, 2012

Hi, I have implemented some approximations for studentized range quantiles and probabilities based on John R. Gleason's (1999) "An accurate, non-iterative approximation for studentized range quantiles." Computational Statistics & Data Analysis, (31), 147-158. Probability approximations rely on scipy.optimize.fminbound. The functions accept both scalars or array-like data thanks to numpy.vectorize. A fair amount of validation and testing has been conducted on the code. More details can be found here: http://code.google.com/p/qsturng-py/ I welcome any thoughts as to whether you all think this might be useful to add to SciPy or make into a scikit. Any general comments would be helpful as well. I should mention I'm a cognitive neuroscientist by trade, my use of statistical jargon probably isn't that good. Regards, Roger Roger Lew

3 5

Many errors/failures running scipy.test() using latest numpy and scipy
by Benjamin Root July 1, 2011

July 1, 2011

I just did a pull and rebuild of numpy and scipy. numpy tested fine, but scipy had many errors and failures. Attached is the log output. The first error has to do with the netcdf file (I thought that was fixed by now?). Then, some of the errors are related to the gmres module (I see that is already known). Then, there are many errors for "test_arpack.test_real_nonsymmetric_modes", "test_arpack.test_complex_nonsymmetric_modes", and "test_arpack.test_hermitian_modes". Lastly, there is a single failure for "test_morestats.TestAnderson". I hope this is useful. Ben Root

3 8

license typo
by Jason Grout June 30, 2011

June 30, 2011

Usually I'd just correct a small typo and submit a pull request, but this is a typo in the license, so it seems more sticky to correct. Condition (c) of the LICENSE.txt file starts: Neither the name of the Enthought nor the names of its contributors of course, "the Enthought" should probably just be "Enthought". Thanks, Jason -- Jason Grout

1 0

LATEST.txt file has svn instructions
by Jason Grout June 30, 2011

June 30, 2011

Just FYI, I just noticed that the LATEST.txt file has the old svn instructions still: https://github.com/scipy/scipy/blob/master/LATEST.txt These should probably be updated to github sometime before the next release... Thanks, Jason -- Jason Grout

2 1

Signal: Add type III/IV filter calculation to firwin2
by Andrey Smirnov June 24, 2011

June 24, 2011

Hello everybody, Since current version of firwin2 can only produce type I and type II filters as a result, and circumstances where type III/IV are needed are not unthinkable, I decided to implement said functionality. The results can be seen at: https://github.com/scipy/scipy/pull/39 I would really appreciate your comments on the code itself and on whether it is possible to include this functionality into scipy. Thanks in advance, Andrey Smirnov

1 0

Strange behaviour of gmres
by Nils Wagner June 20, 2011

June 20, 2011

Hi all, who can shed light on the following behaviour of gmres ? Nils import numpy as np import scipy.sparse.linalg for n in range(2,25): A = np.random.rand(n, n) + 1j*np.random.rand(n, n) b = np.random.rand(n) + 1j*np.random.rand(n) x, info = scipy.sparse.linalg.gmres(A, b) y = np.linalg.solve(A, b) print 'order of the matrix %i info: %i error %e ' %(n, info, abs(x - y).max()) order of the matrix 2 info: 0 error 2.482534e-16 order of the matrix 3 info: 0 error 3.723801e-16 order of the matrix 4 info: 0 error 1.680072e-15 order of the matrix 5 info: 0 error 4.965068e-15 order of the matrix 6 info: 0 error 2.909013e-16 order of the matrix 7 info: 0 error 4.820209e-15 order of the matrix 8 info: 0 error 1.012318e-15 order of the matrix 9 info: 0 error 1.404333e-15 order of the matrix 10 info: 0 error 1.807312e-15 order of the matrix 11 info: 0 error 4.428663e-14 order of the matrix 12 info: 0 error 7.304728e-15 order of the matrix 13 info: 0 error 2.100054e-14 order of the matrix 14 info: 0 error 3.038038e-14 order of the matrix 15 info: 0 error 1.333712e-15 order of the matrix 16 info: 0 error 9.711044e-14 order of the matrix 17 info: 0 error 1.405430e-15 order of the matrix 18 info: 0 error 1.519019e-14 order of the matrix 19 info: 0 error 7.759657e-15 order of the matrix 20 info: 0 error 6.402436e-14 order of the matrix 21 info: 0 error 1.273827e-04 order of the matrix 22 info: 220 error 4.131225e-02 order of the matrix 23 info: 230 error 1.475908e+00 order of the matrix 24 info: 240 error 1.899145e+00

2 1

Advertise on python3 support?
by Xavier Gnata June 20, 2011

June 20, 2011

Hi, Looking at http://www.scipy.org/ it is not obvious to find info on the numpy/scipy python3 support (welldon't even find this statement at all on scipy.org.). Is there a plan to advertise a bit more on that support? I think it is needed because it would clearly show to other packages maintainers that the trend to python3 has started. Xavier

2 1

ERROR: test_arpack.test_hermitian_modes
by Nils Wagner June 19, 2011

June 19, 2011

Hi Pauli, Thank you for very much for the extension to generalized eigenvalue problems. Your recent changes wrt arpack generate an error/failure. Nils ====================================================================== ERROR: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'F', 2, 'SM', None, None, <function aslinearoperator at 0x2640a28>) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 168, in eval_evec eval, evec = eigs_func(ac, k, bc, **kwargs) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 1425, in eigsh OPinv=OPinv) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 1251, in eigs params.iterate() File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 727, in iterate self.workd[yslice] = self.OP(self.workd[xslice]) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 641, in <lambda> self.OP = lambda x: Minv_matvec(matvec(x)) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/interface.py", line 123, in matvec y = self._matvec(x) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py", line 945, in _matvec % (self.ifunc.__name__, info)) ValueError: Error in inverting M: function gmres did not converge (info = 60). FAIL: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'D', 2, 'LM', None, None, <function aslinearoperator at 0x2640a28>) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", line 196, in eval_evec assert_allclose(LHS, RHS, rtol=_rtol[typ], atol=_atol[typ], err_msg=err) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 1167, in assert_allclose verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 635, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=2.22045e-13, atol=2.22045e-13 error for eigsh:general, typ=D, which=LM, sigma=None, mattype=aslinearoperator, OPpart=None, mode=normal (mismatch 100.0%) x: array([[ 0.40172292+3.97785014j, 1.26541377-2.2227806j ], [ 0.67562574+2.73428778j, -6.32196547-3.30680082j], [-0.75603444+1.17747523j, -0.02785408-1.70286801j],... y: array([[ 0.40172292+3.97785014j, 1.26541377-2.2227806j ], [ 0.67562574+2.73428778j, -6.32196547-3.30680082j], [-0.75603444+1.17747523j, -0.02785408-1.70286801j],...

2 4