Mailman 3 June 2008 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Question about Kaiser Implementation in firwin
by Buehler, Eric (AGRE) Oct. 4, 2008

Oct. 4, 2008

Hello, I have been using the signal.firwin FIR design tool and I have a question about the implementation. Scipy-0.4.8 Numpy-0.9.6 I am generating a kaiser window with the following parameters: N = 128 Cutoff = ~4e7 width = .3 All of the special kaiser generation parameters in the function follow Oppenheim and Schafer well. Even the sinc function for linear phase is follows exactly. However, the last line of the firwin function is causing me some heartburn. filter_design.py 1538 win = get_window(window,N,fftbins=1) 1539 alpha = N//2 1540 m = numpy.arange(0,N) 1541 h = win*special.sinc(cutoff*(m-alpha)) 1542 return h / sum(h) Line 1542 of filter_design.py, "return h / sum(h)", normalizes the function where it doesn't seem necessary, at least in the kaiser window case. Without the normalization, the kaiser window already returns a value of 1 at the zero frequency point. This normalization scales all of the data, making the window difficult to use in the frequency domain. Can someone point me to the rationale for this line? Looking at the code, this seems to be a pretty recent change (within the last year/year and a half). Thanks, Eric Buehler eric <dot> buehler <at> smiths-aerospace <dot> com ****************************************** The information contained in, or attached to, this e-mail, may contain confidential information and is intended solely for the use of the individual or entity to whom they are addressed and may be subject to legal privilege. If you have received this e-mail in error you should notify the sender immediately by reply e-mail, delete the message from your system and notify your system manager. Please do not copy it for any purpose, or disclose its contents to any other person. The views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the company. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused, directly or indirectly, by any virus transmitted in this email. ******************************************

3 2

Install error on RHEL5
by Michael Hearne June 30, 2008

June 30, 2008

I just finished an install of numpy/scipy/matplotlib on a 64-bit RHEL5 system, and am now trying to replicate that on a 32-bit system. So far, there are two differences: 1) fftw3 is not available through yum on the 32-bit system, so I built it from source. 2) Attempting to build scipy results in the error below. Attached is a PDF with the notes (most of which came from Phil Austin) I compiled after installing on the 64-bit system. I'm currently recompiling ATLAS (which takes an amazingly long time), and then will try to rebuild LAPACK, BLAS, etc. Before I get there, does anyone have any idea what the root of the below error might be? usr/local/bin/python setup.py build Traceback (most recent call last): File "setup.py", line 53, in <module> setup_package() File "setup.py", line 28, in setup_package from numpy.distutils.core import setup File "/usr/local/lib/python2.5/site-packages/numpy/__init__.py", line 103, in <module> import linalg File "/usr/local/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in <module> from linalg import * File "/usr/local/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 29, in <module> from numpy.linalg import lapack_lite ImportError: /usr/local/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: cblas_zgemm Thanks, Mike -- ------------------------------------------------------ Michael Hearne mhearne(a)usgs.gov (303) 273-8620 USGS National Earthquake Information Center 1711 Illinois St. Golden CO 80401 Senior Software Engineer Synergetics, Inc. ------------------------------------------------------

2 1

Python/Sage minisymposium at SIAM Annual meeting July 9/10 2008
by Fernando Perez June 30, 2008

June 30, 2008

Hi all, this is just a reminder, in case you'll be attending the SIAM 2008 annual meeting next week in San Diego, that there will be a 3-part minisymposium focusing on the uses of Python and Sage in scientific computing: http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=7369 http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=7370 http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=7447 We hope to see many of you there! Regards, Fernando and Randy

1 0

Possibly one more easy bug
by Yosef Meller June 30, 2008

June 30, 2008

http://www.scipy.org/scipy/scipy/ticket/598 in optimize/anneal.py. I posted two patches, I'm not sure which one is more appropriate. -- http://yosefm.imagekind.com/Eclectic

2 1

Removing globals from minpack wrapper, v2
by Yosef Meller June 28, 2008

June 28, 2008

My continued worl on removing globals from the minpack wrapper, for your testing pleasure. Changes: * Applied the same strategy to minpack_hybrd (mostly, but not fully, a copy-paste job from minpack_hybrj) * polished the previous code a bit. * Now with multithreading! in minpack_hybrj and jac_multipack_calling_function the GIL is released and reacquired as necessary. Works on my machine, and you can test it with the attached python program. I still haven't had the chance to test it on a dual core system, though, and it might be a while before I do. -- http://yosefm.imagekind.com/Eclectic

1 0

Building numpy/scipy on RHEL5 OR CentOS5
by Michael Hearne June 27, 2008

June 27, 2008

I've been wrestling with installing scipy (in any form - binary, source, egg, rpm) on some RHEL5 machines that I am compelled to work with, and have failed miserably. I noticed a thread a few months back about building scipy on CentOS 5, and noticed that at least one of the respondents indicated he'd had success. As I understand it, CentOS strives for binary compatibility with RHEL, so I'm hopeful that CentOS solutions will work for me. Does anyone have (in order of preference): 1) A CentOS/RHEL5 yum repository for scipy/numpy/(optionally matplotlib) 2) CentOS/RHEL5 RPM's for scipy/numpy/(optionally matplotlib) 3) A set of instructions for building scipy from source on either of these platforms? I'm getting a little desperate here - this is likely one of the most difficult installs I've ever had to deal with. Thanks, Mike -- ------------------------------------------------------ Michael Hearne mhearne(a)usgs.gov (303) 273-8620 USGS National Earthquake Information Center 1711 Illinois St. Golden CO 80401 Senior Software Engineer Synergetics, Inc. ------------------------------------------------------

6 16

SciPy Conference Updates
by Travis Vaught June 27, 2008

June 27, 2008

Greetings, The SciPy Conference is not too far away. I thought I'd summarize some recent news about the conference in case some of you missed it: - Accommodations (news!): We've negotiated a group rate with a nearby Marriott hotel, for those that would like to take advantage of it. The hotel has set up a web site for our event here: http://cwp.marriott.com/laxot/scipyworkshop/ - Student Sponsorship: As you may have seen, the Python Software Foundation has agreed to partner with Enthought to sponsor 10 students' travel, registration, and accommodation for the tutorials, conference and (most importantly) sprints. If you're in college or a graduate program, please check out the details here: http://conference.scipy.org/sponsoring - Abstracts Submission Deadline Extended: The review committee is extending the deadline to Monday, June 30th. Please see the Call for Papers for instructions on abstract submission here: http://conference.scipy.org/call_for_papers Please drop me an email if you have any questions or comments. Best, Travis

1 0

0.7 release schedule
by Nathan Bell June 27, 2008

June 27, 2008

Is there a plan to get SciPy 0.7 out the door? Numerous improvements have been made since the release of 0.6, which was 9 months ago. Whatever the faults of the current SVN (e.g. unittest that have been broken for over a month), are they not now preferable to those in 0.6? A release now would be helpful to those who want to use SciPy in the classroom this fall. -- Nathan Bell wnbell(a)gmail.com http://graphics.cs.uiuc.edu/~wnbell/

9 31

Removing globals from minpack wrapper
by Yosef Meller June 26, 2008

June 26, 2008

I was having my fun with optimize.fsolve() at work, when I realized that both the Python side and the C/FORTRAN side allocate work spaces, and then copy between them, which seemed wasteful. I started playing around with the code to try and create a version that allows the same buffer to move around between Python and minpack. I couldn't do that (and I doubt now that it's possible), but on the way I found a method to remove the global variables that are used to store the data needed by the multipack functions but not passed in by the FORTRAN code which doesn't even know about it. The trick was to allocate extra space on the work buffers and use it as a pointer to a structure that contains the information. This may seem just as ugly, but it has all the advantages of not using globals, including the prospects of someday making it thread-safe. The attached patch implements the trick in minpack_hybrj, and the Python code tests that it works. Do you think it's worth doing to the rest of the minpack functions? -- http://yosefm.imagekind.com/Eclectic

2 2