Mailman 3 October 2010 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Subversion scipy.stats irregular problem with source code example
by James Phillips Dec. 9, 2010

Dec. 9, 2010

Since I observed the following behavior in the SVN respository version of SciPy, it seemed to me proper to post to the dev mailing list. I'm using Ubuntu Lucid Lynx 32 bit and a fresh GIT of Numpy. I am not sure if a Trac bug report needs to be entered. Below is some example code for fitting two statistical distributions. Sometimes the numpy-generated data is fit, as I can see the estimated and fitted parameters are different. Sometimes I receive many messages repeated on the command line like: Warning: invalid value encountered in absolute Warning: invalid value encountered in subtract and the estimated parameters equal the fitted parameter values, indicating no fitting took place. Sometimes I receive on the command line: Traceback (most recent call last): File "/home/zunzun/local/lib/python2.6/site-packages/scipy/stats/distributions.py", line 1987, in func sk = 2*(b-a)*math.sqrt(a + b + 1) / (a + b + 2) / math.sqrt(a*b) ValueError: math domain error Traceback (most recent call last): File "example.py", line 10, in <module> fitStart_beta = scipy.stats.beta._fitstart(data) File "/home/zunzun/local/lib/python2.6/site-packages/scipy/stats/distributions.py", line 1992, in _fitstart a, b = optimize.fsolve(func, (1.0, 1.0)) File "/home/zunzun/local/lib/python2.6/site-packages/scipy/optimize/minpack.py", line 125, in fsolve maxfev, ml, mu, epsfcn, factor, diag) minpack.error: Error occured while calling the Python function named func and program flow is stopped. In summary, three behaviors: (1) Fits OK (2) Many exceptions with no fitting (3) minpack error. Running the program 10 times or so will reproduce these behaviors without fail from the "bleeding-edge" repository code. James Phillips ######################################################## import numpy, scipy, scipy.stats # test uniform distribution fitting data = numpy.random.uniform(2.0, 3.0, size=100) fitStart_uniform = scipy.stats.uniform._fitstart(data) fittedParameters_uniform = scipy.stats.uniform.fit(data) # test beta distribution fitting data = numpy.random.beta(2.0, 3.0, size=100) fitStart_beta = scipy.stats.beta._fitstart(data) fittedParameters_beta = scipy.stats.beta.fit(data) print print 'uniform._fitstart returns', fitStart_uniform print 'fitted parameters for uniform =', fittedParameters_uniform print print 'beta._fitstart returns', fitStart_beta print 'fitted parameters for beta =', fittedParameters_beta print

7 42

matlab reading / writing changes - review anyone?
by Matthew Brett Nov. 12, 2010

Nov. 12, 2010

Hi, In the spirit of the development workflow discussions on the numpy list - I'd be very grateful for a review of changes I've made to the matlab reader. The changes are to: a) Allow the reader to read some malformed files that matlab is able to read b) Prevent the writer needlessly up-casting some numpy array types (uint8, uint32) I'm sorry - I probably have probably made the comparison a little too large to be taken in in an instant, but I'd be very grateful for feedback before merge. http://github.com/matthew-brett/scipy-work/compare/master...mio-read-fix Thanks, Matthew

2 3

Google custom search for ask.scipy.org?
by Pauli Virtanen Nov. 11, 2010

Nov. 11, 2010

Hi, A current (killer) problem with ask.scipy.org is that it does not have search. As a band-aid solution, it should be possible to add a Google custom search box, like on this page: http://scikit-learn.sourceforge.net/ Google searches "site:ask.scipy.org/en/topic KEYWORDS" work well, so that a custom search should also work OK. -- Pauli Virtanen

2 2

scipy.stats.distributions: note on initial parameters for fitting the beta distribution
by James Phillips Nov. 8, 2010

Nov. 8, 2010

Here is a link on estimating initial parameters for fitting the beta distribution. I'm posting as it seems a good bit different from the current calculation in scipy.stats.distribution, and it calculates only the sample mean and biased sample variance. I am not competent to judge which of the two methods is better for scipy, and hope someone on the mailing list can do so. http://www.xycoon.com/beta_parameterestimation.htm James

2 10

Partial Least Squares Regression (PLSR) implementation
by Aman Thakral Oct. 26, 2010

Oct. 26, 2010

Hi, I've recently implemented PLSR in Python. Is there currently an implementation in scipy? If not, i'd be happy to share it (and get it reviewed :)) I'm working on including SPE, VIP, and Hotelling's T squared as part of the analysis. Cheers, Aman

4 7

[ANN]: FEMTEC 2011: track on Open-source projects and Python in scientific computing.
by Gael Varoquaux Oct. 24, 2010

Oct. 24, 2010

The 3rd International Conference on Finite Element Methods in Engineering and Science (FEMTEC 2011, http://hpfem.org/events/femtec-2011/) will have a track on Open-source projects and Python in scientific computing. FEMTEC 2011 is co-organized by the University of Nevada (Reno), Desert Reseach Institute (Reno), Idaho National Laboratory (Idaho Falls, Idaho), and U.S. Army Engineer Research and Development Center (Vicksburg, Mississippi). The objective of the meeting is to strengthen the interaction between researchers who develop new computational methods, and scientists and engineers from various fields who employ numerical methods in their research. Specific focus areas of FEMTEC 2011 include, but are not limited to, the following: * Computational methods in hydrology, atmospheric modeling, and other earth sciences. * Computational methods in nuclear, mechanical, civil, electrical, and other engineering fields. * Mesh generation and scientific visualization. * Open-source projects and Python in scientific computing. Part of the conference will be a software afternoon featuring open source projects of participants. Proceedings Proceedings of FEMTEC 2011 will appear as a special issue of Journal of Computational and Applied Mathematics (2008 SCI impact factor 1.292), and additional high-impact international journals as needed. ____________ On a personnal note, I was at ESCO 2010 this year, the European version organized by the same community, and it was really a refreshing and friendly conference during which I learned a lot.

1 0

Added "Checkin Checklist" to the developer's wiki
by Warren Weckesser Oct. 24, 2010

Oct. 24, 2010

Back in May I sent a "Checkin Checklist" to this list. I've finally added it to the developer's wiki; see the first link under "Guidelines" here: http://projects.scipy.org/scipy/wiki Warren

1 0

stats.distributions ticket 1291
by josef.pktd＠gmail.com Oct. 23, 2010

Oct. 23, 2010

http://projects.scipy.org/scipy/ticket/1291 Warren, if you want, there are a few lines in stats.distributions to change (my scipy is out of date, and I don't have time to update.) As explained in the ticket, the bug cannot be really tested in the test suite, since it shows up only on the first call to the distribution. Thanks, Josef

1 0

scipy svn + python3.1.2?
by Neal Becker Oct. 19, 2010

Oct. 19, 2010

Is this supposed to work? I got as far as: env ATLAS=/usr/lib64 FFTW=/usr/lib64 BLAS=/usr/lib64 LAPACK=/usr/lib64 python3 setup.py config_fc --fcompiler=gnu95 build Converting to Python3 via 2to3... ... RefactoringTool: /home/nbecker/scipy/build/py3k/scipy/sparse/linalg/dsolve/umfpack/setup.py RefactoringTool: /home/nbecker/scipy/build/py3k/scipy/sparse/linalg/isolve/setup.py /home/nbecker/scipy/build/py3k/scipy/linalg/basic.py /home/nbecker/scipy/build/py3k/scipy/linalg/decomp.py ... /home/nbecker/scipy/build/py3k/scipy/sparse/linalg/isolve/iterative.py Traceback (most recent call last): File "setup.py", line 179, in <module> setup_package() File "setup.py", line 144, in setup_package shutil.copy(site_cfg, src_path) NameError: global name 'shutil' is not defined

2 2