Mailman 3 March 2012 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

prepending underscores to private module names
by Ralf Gommers May 16, 2012

May 16, 2012

Hi, A while ago we discussed making a clear distinction between public and private modules by using underscores consistently, see http://thread.gmane.org/gmane.comp.python.scientific.devel/14936. I've done this for a few modules now, see https://github.com/rgommers/scipy/tree/underscores. Most files could simply be underscored, some other would conflict with extension modules so I also had to append _py to the name. Also I added a file doc/source/api.rst that states which modules are public (up for discussion of course). With google code search I checked how common it is to import from modules that are supposed to be private, for example "from scipy.optimize.optimize import fmin". It's not very common but does happen, so deprecation warnings should be put in. Before going further I'd like to check that this looks okay to people. What do you think? Cheers, Ralf <https://github.com/rgommers/scipy/tree/underscores>

4 4

scipy.sparse and OpenMP
by Maximilian Nickel April 26, 2012

April 26, 2012

Hi everyone, I've been working with fairly large sparse matrices on a multiprocessor system lately and noticed that scipy.sparse is single-threaded. Since I needed faster computations, I've quickly added some OpenMP #pragma directives in scipy/sparse/sparsetools to the functions that I've been using in order to enable multithreading, what worked out nicely. I wondered if you would be interested in a more complete OpenMP-enabled version of scipy.sparse.setuptools. I've attached the patch of the quick-and-dirty changes that I made so far to this mail, to give you an idea. Best regards Max

5 4

cephes_smirnov never returns on mips/sparc/...
by Yaroslav Halchenko April 1, 2012

April 1, 2012

I have reported this issue some time ago on Debian http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653948 and then forgot about it until now that I ran into it again. With recent scipy (including git master 0fbfdbc) scipy.stats.ksone.fit seems to stall (never return) on big-endian boxes and return (1.0, nan, nan) on x86. It seems that it is working correctly with scipy 0.7.2 (as it is now in Debian stable). So -- is that anything known? snippet to replicate: import numpy as np import scipy.stats as ss d = np.array([-0.18879233, 0.15734249, 0.18695107, 0.27908787, -0.248649, -0.2171497 , 0.12233512, 0.15126419, 0.03119282, 0.4365294 , 0.08930393, -0.23509903, 0.28231224, -0.09974875, -0.25196048, 0.11102028, 0.1427649 , 0.10176452, 0.18754054, 0.25826724, 0.05988819, 0.0531668 , 0.21906056, 0.32106729, 0.2117662 , 0.10886442, 0.09375789, 0.24583286, -0.22968366, -0.07842391, -0.31195432, -0.21271196, 0.1114243 , -0.13293002, 0.01331725, -0.04330977, -0.09485776, -0.28434547, 0.22245721, -0.18518199, -0.10943985, -0.35243174, 0.06897665, -0.03553363, -0.0701746 , -0.06037974, 0.37670779, -0.21684405]) print "Fitting now" print ss.ksone.fit(d) -- =------------------------------------------------------------------= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic

3 21

Pull Request Review: compressed sparse graphs
by Jacob VanderPlas March 31, 2012

March 31, 2012

Hi, I think I'm finally finished with the new set of compressed sparse graph algorithms: https://github.com/scipy/scipy/pull/119 It has a fairly extensive set of efficient algorithms for sparse graph analysis, with full unit tests, documentation, and a short tutorial on using the tools to solve the word ladder problem. I've gotten plenty of good feedback along the way - I think the PR is ready for a final review. Thanks Jake

1 0

Re: [SciPy-Dev] Trac performance?
by Vincent Davis March 31, 2012

March 31, 2012

I read through the discussion from earlier this year concerning updated Trac, moving it to a new server, or to Github. I spent some time Sunday look at option to follow different subjects, bugs.... Starting with logging in most of what I did resulted in a generic database error or no results due to a database locked error. Not sure what the final consensus on the topic was. I think someone installed a github to Trac plugin, not sure what that does. Well I spent 30min on the SciPy Trac and had a bad experience. How can I help.? I would be willing to host a amazon ec2 machine if that is needed (I saw someone else offered the same) possibly converting the current machine to a virtual would be a sort term option. I have some experiance with this. Thanks vincent Davis -- Thanks Vincent Davis 720-301-3003

3 2

ODRPACK95
by Tim Cera March 31, 2012

March 31, 2012

Hello, I have a project where I could use the bounded parameter functionality in ODRPACK95. Is anyone working on implementing it? I started to look at it and have a rough idea of what to do, using f2py. Why does the current ODRPACK implementation use C? Is there something important where C is needed? Also, why isn't odr under scipy.optimization? ODRPACK95 can be called in such a way to calculate a least squares implementation. Would that be useful to someone to have bounded parameter least squares optimization? Kindest regards, Tim

2 2

[Feature Request] Generalized Schur decomposition
by Bruno André Rodrigues Coelho March 29, 2012

March 29, 2012

Hi, I'm translating some matlab code into Python/Scipy and noticed that Scipy lacks a generalized QZ decomposition that works like in Matlab. I found an equivalent by Sven Schreiber here: http://econ.schreiberlin.de/schreibersoftware.html which works exactly like Matlab's qz function. Could this be added to scipy.linalg? Cheers. -- Rodrigues Bruno ---------------------------- http://cbrunos.wordpress.com

2 7

Re: [SciPy-Dev] call signature for dgees f2py external user routine?
by Skipper Seabold March 29, 2012

March 29, 2012

On Wed, Mar 28, 2012 at 8:19 PM, Skipper Seabold <jsseabold(a)gmail.com> wrote: > Can someone explain the call signature for the select function used > from the gees routines? Or point me to a reference? I don't understand > the syntax. <_arg=...> > > https://github.com/scipy/scipy/blob/master/scipy/linalg/flapack_user.pyf.sr… Sorry meant to send this to the dev list. Please reply there. Skipper

1 0

studentized range approximations
by Roger Lew March 28, 2012

March 28, 2012

Hi, I have implemented some approximations for studentized range quantiles and probabilities based on John R. Gleason's (1999) "An accurate, non-iterative approximation for studentized range quantiles." Computational Statistics & Data Analysis, (31), 147-158. Probability approximations rely on scipy.optimize.fminbound. The functions accept both scalars or array-like data thanks to numpy.vectorize. A fair amount of validation and testing has been conducted on the code. More details can be found here: http://code.google.com/p/qsturng-py/ I welcome any thoughts as to whether you all think this might be useful to add to SciPy or make into a scikit. Any general comments would be helpful as well. I should mention I'm a cognitive neuroscientist by trade, my use of statistical jargon probably isn't that good. Regards, Roger Roger Lew

3 5