Mailman 3 January 2009 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Patching fmin_l_bfgs_b in scipy.optimize
by Gilles Rochefort April 15, 2009

April 15, 2009

Hi, I had some troubles getting a stable version of fmin_l_bfgs_b. I got a segmentation fault as soon as I turned a different value than -1 to iprint argument. As far as I understood, this iprint parameter is directly related to some fortran printing/writing logs. It prints iteration, norm of gradient on screen and also in a file called iterate.dat .. Even recompiling the whole stuff from lapack, atlas etc.. (using gfortran only) up to scipy doesn't solve the problem. Because I definitely gave up the idea to fix the bug by recompiling, I decided to patch the way python interacts with lbfgsb fortran procedure. So, I disabled the fortran iprint and add some iprint functionnality in pure python code directly in lbfgsb.py (scipy/optimize). Doing so, I get rid of the useless iterate.dat file too. While patching, I followed by adding some few things : - an optional stopping rule based on maximum number of iterations. I think this is more usefull than the number of function evaluations because the fortran code performs a linesearch procedure inside the lbfgsb ones. - a callback procedure which allows to inspect solution, criterion, and gradient at current iterate. Maybe my patch should be usefull for someone, so I decided to share it with the scipy community. Gilles. PS: I provide two files : lbfgsb.py -- the full python script to put in scipy/optimize scipy-optimize-lbfgsb-12162008.patch -- a patch file

3 4

Warning about remaining issues in stats.distributions ?
by josef.pktd＠gmail.com March 16, 2009

March 16, 2009

There are still some known failures or methods, that don't work very well for some distributions: * skew. kurtosis: a few remaining incorrect results, no exact test available * entropy: returns nan in some cases, correctness of values not tested * fit: does not converge, or does not estimate correctly for some distributions, this is expected since maximum likelihood does not always work * distributions that have problems for some range of parameters * incorrect random number generator for one discrete distribution Is it useful to put a warning about these issues in stats.distributions in the release? A friend of mine is doing value-at-risk calculations for operational risk analysis, but I am still reluctant to recommend scipy.stats, when it is not clear whether the results are correct or not, although scripting in Python would be much easier than to struggle with some commercial package with a GUI user interface. The problem is that in this application they are using many distributions automatically, and not just a few that are well tested by frequent use. Per Brodtkorb has an improved version of distributions.py with an additional estimation procedure, which we can discuss as an enhancement after 0.7 is out of the way, if there is an interest in this. Josef

4 15

FAIL: test_yn_zeros, test_ynp_zeros, Negative-order Bessels
by Nils Wagner Feb. 13, 2009

Feb. 13, 2009

Hi all, I do get three failures wrt. to special functions and python2.6 ====================================================================== FAIL: test_yn_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1598, in test_yn_zeros 488.98055964441374646], rtol=1e-19) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-19, atol=0 (mismatch 100.0%) x: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) y: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) ====================================================================== FAIL: test_ynp_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1604, in test_ynp_zeros assert_tol_equal(yvp(443, ao), 0, atol=1e-15) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=1e-15 (mismatch 100.0%) x: array([ 1.239e-10, -8.119e-16, 3.608e-16, 5.898e-16, 1.226e-15]) y: array(0) ====================================================================== FAIL: Negative-order Bessels ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1665, in test_ticket_853 assert_tol_equal(iv(-0.5, 1 ), 1.231200214592967) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=0 (mismatch 100.0%) x: array(0.0) y: array(1.231200214592967) Nils

12 50

DCT naming conventions ?
by David Cournapeau Feb. 10, 2009

Feb. 10, 2009

Hi, I needed DCT transforms, so I finished implementing them in scipy (both single and double prec): http://projects.scipy.org/scipy/scipy/browser/trunk/scipy/fftpack/realtrans… I followed the definitions of wikipedia/fftw, using the fftpack implementations. I am not sure about naming conventions: - dct2 is the type II DCT, not the 2dimension DCT (multidimensional dcts are not implemented - I don't need them myself) - dct3 and dct2 are inverse from each other (compared to dct/idct notation of matlab; dct2(x, norm=ortho) is the same as matlab dct(x), and dct3(x, norm='ortho') is the same as matlab idct. Unless someone has a better idea/suggestion, I will put those definitions in scipy.fftpack namespace in a couple of days, cheers, David

7 25

scipy.org is down again
by Nils Wagner Feb. 5, 2009

Feb. 5, 2009

Hi all, I cannot access http://www.scipy.org Nils

3 2

removing wx-related weave code (was test errors blocking 0.7.x branch)
by Jarrod Millman Jan. 29, 2009

Jan. 29, 2009

I changed the subject line for this because I wanted to make sure everyone reads it. The only remaining issue that needs to be resolved before I tag the 0.7.0rc1 is what to do about the wx-related weave tests. Depending on your configuration and how you run the tests, the wx-related weave code will generate a number of test errors. I think that given the huge amount effort that has gone into making a very polished 0.7.0 release, we need to deal with these errors quickly. After looking at the code, I don't believe we are going to be able to fix these errors so there are only two options that I see: 1. Mark all the wx-related weave tests as known failures 2. Remove the wx-related weave code and tests Fernando's email below advocates the 2nd option, which is the one I am slightly leaning to at this point. I would rather not add a number of known failures to our test system at this point, especially for code that doesn't seem to me to be very likely to work on many systems given the number of hard-coded paths in wx_spec.py (stefan fixed this somewhat in r3266). I would like to know if anyone is using this functionality, so if you are or know someone who is please speak up immediately. On Wed, Jan 7, 2009 at 3:29 PM, Fernando Perez <fperez.net(a)gmail.com> wrote: > On Tue, Jan 6, 2009 at 5:50 PM, Jarrod Millman <millman(a)berkeley.edu> wrote: > >> ====================================================================== >> ERROR: no_test_no_check_return (test_wx_spec.TestWxConverter) >> ---------------------------------------------------------------------- > > As a note, an easy way of seeing all the wx problems is to run the > test suite inside a screen session, which doesn't set $DISPLAY. In > that case, you start seeing a lot of: > > ====================================================================== > ERROR: no_test_no_check_return (test_wx_spec.TestWxConverter) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/fperez/usr/opt/lib64/python2.5/site-packages/scipy/weave/tests/test_wx_spec.py", > line 30, in setUp > self.app = wx.App() > File "/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode/wx/_core.py", > line 7880, in __init__ > raise SystemExit(msg) > SystemExit: Unable to access the X Display, is $DISPLAY set properly? > > > IMHO, the WX support in weave might as well be ripped out altogether. > I'd be very surprised if it really works at all, and it seems to be a > remnant from the early days when scipy had lots of plotting and GUI > support. Now that scipy's focus is pretty much algorithms, and > scientific computing (which includes data formats, I/O and a few > non-numerical things, I know), it seems to me that keeping WX code > around is just creating unnecessary noise and maintenance headaches. > > Does anyone know if the WX code in weave actually works correctly > under any circumstances? I'm honestly curious, especially because > seeing code like: > > searched_locations = ['c:\third\wxpython*', > '/usr/lib/wx*'] > > makes me a bit suspicious :) > > Just an opinion from the peanut gallery... > > Cheers, > > f

9 23

A buildbot farm with shell access - for free ?
by David Cournapeau Jan. 29, 2009

Jan. 29, 2009

Hi, Just saw that on one ML: http://www.snakebite.org/ http://mail.python.org/pipermail/python-committers/2009-January/000331.html Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available. It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ? cheers, David

3 2

Matrix sign function is missing in the docs
by Nils Wagner Jan. 29, 2009

Jan. 29, 2009

Hi all, The matrix sign function (linalg.signm) is missing in the docs scipy/doc/build/html/tutorial/linalg.html#matrix-functions Nils

2 1

make latex in scipy/doc failed
by Nils Wagner Jan. 27, 2009

Jan. 27, 2009

Hi all, a make latex in scipy/doc failed Intersphinx hit: PyCObject_FromVoidPtrAndDesc http://docs.python.org/dev/c-api/cobject.html Intersphinx hit: PyCObject_FromVoidPtr http://docs.python.org/dev/c-api/cobject.html WARNING: /home/nwagner/svn/scipy/doc/source/index.rst:: (WARNING/2) undefined label: numpy-reference.routines.io -- if you don't give a link caption the label must precede a section header. Intersphinx hit: numpy.poly1d http://docs.scipy.org/doc/numpy/reference/generated/numpy.poly1d.html writing... Exception occurred: File "/home/nwagner/local/lib64/python2.6/site-packages/Sphinx-0.5.1-py2.6.egg/sphinx/latexwriter.py", line 619, in visit_entry raise NotImplementedError('Column or row spanning cells are ' NotImplementedError: Column or row spanning cells are not implemented. The full traceback has been saved in /tmp/sphinx-err-GOKGyC.log, if you want to report the issue to the author. Please also report this if it was a user error, so that a better error message can be provided next time. Send reports to sphinx-dev(a)googlegroups.com. Thanks! make: *** [latex] Fehler 1 log file attached. Nils

1 0