Mailman 3 May 2010 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

np.savetxt: apply patch in enhancement ticket 1079 to add headers?
by Skipper Seabold June 8, 2010

June 8, 2010

Hi all, I am assuming that this is ok to request via the list... Could we discuss or could someone apply the patch in enhancement ticket 1079? http://projects.scipy.org/numpy/ticket/1079 I needed this functionality recently, and this is a quick and easy fix that may have been overlooked. There is also another enhancement request about this here: http://projects.scipy.org/numpy/ticket/1236 The only thing that I can think of that might need to be added is a test to see that the header length is the same as the number of columns, but really that might just be up to the user to supply the right headers. It might also be nice to have a header = True, that uses the field names for a structured array, but I can live without that. Cheers, Skipper

3 11

Summer Marathon Skypecon tomorrow
by David Goldsmith June 3, 2010

June 3, 2010

So far, no one has RSVP-ed (positive or negative). Is this due to: A) Lack of time; B) Bad scheduling (i.e., you have time, just not at the time we've chosen); C) Lack of interest; D) Lack of issues to discuss (i.e., you have interest, but not in a meeting without a specific aganda); E) Bad choice of conference media (i.e., can't/won't do Skype); F) Just forgot to RSVP; G) Some of the above; H) Other/None of the above? If I haven't heard from anyone, in the positive, by midnight tonight, EDT, this week's Skypecon is canceled. DG

3 3

Mea culpa: deprecation and API changes
by Warren Weckesser June 3, 2010

June 3, 2010

Over the last several months, I've changed the API of few function in ways that are not compatible with the previous behavior. These are: constants.find It now returns a list instead of printing it. (http://projects.scipy.org/scipy/ticket/996) linalg.solveh_banded It now returns just the solution, instead of the tuple containing the solution and the cholesky factor. (http://projects.scipy.org/scipy/ticket/676) signal.chirp Removed one keyword argument and added a new keyword argument. Also, it no longer handles a polynomial (or polynomial-like) argument. (http://projects.scipy.org/scipy/ticket/1105, but the API changes are not actually related to the problem reported there.) The recent discussion about deprecation and ppimport reminded me that these changes should be deprecated for one release. What I would like to do is leave trunk as it is, and after 0.8 is branched, make the appropriate changes in the branch to follow the deprecation policy. Is that a reasonable approach? Warren

6 19

scipy.stats
by josef.pktd＠gmail.com June 2, 2010

June 2, 2010

Since Travis seems to want to take back control of scipy.stats, I am considering my role as inofficial maintainer as ended. I would have appreciated his help almost 3 years ago, when I started to learn numpy, scipy, and started to submit patches for scipy.stats.distributions. But by now, I have pretty strong opinions about statistics in python, after almost three years, I'm a bit tired of cleaning up the mess of others (and want to clean up my own mess), and there are obviously big philosophical differences for the development process between me and Travis (no discussion, no review, no tests). http://projects.scipy.org/scipy/log/trunk/scipy/stats/tests Watching the scipy changelog and checking any function that Travis quietly commits is no fun (see mailing list for the introduction of curve_fit or ask Stefan). I said early on that I would like to trust the results that scipy.stats produces (although I don't find the mailing list thread any more). I considered scipy to go into a stable direction like Python is, kitchen sink for scientific programming, which might be slow-moving but with high standards, and not a sandbox. Details are at http://mail.scipy.org/pipermail/scipy-dev/2010-April/014058.html After my initial scipy.stats.distributions cleanup, test coverage was at 91%, I have no idea where it is after this weekend. This is more about the process then the content, distributions was Travis's baby (although unfinished), and most of his changes are very good, but I don't want to look for the 5-10% (?) typos anymore. Cheers, Josef

12 47

Scipy archive on PyPI
by Charles Vejnar June 2, 2010

June 2, 2010

Hi, I was trying to install Scipy with easy_install and it seems that downloading from Sourceforge is no longer possible (Sourceforge no longer gives a direct link to the .tar.gz file) which makes the install fail. Would it be possible to always upload the latest Scipy tarball to PyPI ? Thanks Charles

3 4

Development process (was: scipy.stats)
by Matthew Brett June 1, 2010

June 1, 2010

Hi, Sorry - I moved this thread over because it seemed to be of general importance - both for scipy.stats but also for our working process. I really think it's a discussion we need to have in public - either here on the list - or in a public meeting in scipy. On Mon, May 31, 2010 at 7:16 AM, <josef.pktd(a)gmail.com> wrote: > Since Travis seems to want to take back control of scipy.stats, I am > considering my role as inofficial maintainer as ended. > > I would have appreciated his help almost 3 years ago, when I started > to learn numpy, scipy, and started to submit patches for > scipy.stats.distributions. > > But by now, I have pretty strong opinions about statistics in python, > after almost three years, I'm a bit tired of cleaning up the mess of > others (and want to clean up my own mess), and there are obviously big > philosophical differences for the development process between me and > Travis (no discussion, no review, no tests). > http://projects.scipy.org/scipy/log/trunk/scipy/stats/tests Returning to Josef's frustration. It seems to me there is a general thirst for a more formal understanding of code ownership, and the process for discussion and review of changes. I think Josef's frustration is very important; my impression is that Josef's work on scipy.stats has been a very large step forward for a module that previously was seriously broken and not properly tested. It's obvious I think that Josef is the de-facto maintainer of scipy.stats, and it's obvious that, as maintainer, he should first - be asked about changes, and second - should have the right of review before acceptance. Any maintainer would expect this, and any maintainer is going to get annoyed if those accepted rules of behavior are not observed. Travis - hoping you are reading this thread - does that also seem reasonable to you? What do you think would be the best way of proceeding? Best, Matthew

4 5

updated 0.8.0 release schedule
by Ralf Gommers May 31, 2010

May 31, 2010

Hi all, Beta 1 for 0.8.0 was supposed to be available today, and I built some binaries before realizing that scipy.stats is currently not importable on python 2.4/2.5. On top of that I will be traveling, probably without internet access, from 8-14 June. So here's a revised release schedule: 03/06: 0.8.0 beta 1 17/06: 0.8.0 release candidate 1 24/06: 0.8.0 release I hope that doesn't inconvenience anyone too much. Cheers, Ralf

1 0

Invalid syntax on Python < 2.6 in scipy.stats
by Pauli Virtanen May 31, 2010

May 31, 2010

There's some invalid syntax in Scipy trunk and 0.8.x branches that would need to be fixed: http://projects.scipy.org/scipy/ticket/1186

1 0

lambertw test failure
by Ralf Gommers May 31, 2010

May 31, 2010

This test has been failing for a long time. Does anyone know what's going on, and can it be fixed in the next ten days or so? ====================================================================== FAIL: test_lambertw.test_values ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-0.11.1-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/Users/rgommers/Code/scipy/scipy/special/tests/test_lambertw.py", line 80, in test_values FuncData(w, data, (0,1), 2, rtol=1e-10, atol=1e-13).check() File "/Users/rgommers/Code/scipy/scipy/special/tests/testutils.py", line 187, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 2.5797 Max |rdiff|: 3.81511 Bad results for the following points (in output 0): (-0.44800000000000001+0.40000000000000002j) 0j => (-1.2370928928166736-1.6588828572971359j) != (-0.11855133765652383+0.66570534313583418j) (rdiff 3.8151122286225245) There's also a ticket regarding the lambertw docstring: http://projects.scipy.org/scipy/ticket/1148 Thanks, Ralf

3 2