Mailman 3 January 2008 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Question about Kaiser Implementation in firwin
by Buehler, Eric (AGRE) Oct. 4, 2008

Oct. 4, 2008

Hello, I have been using the signal.firwin FIR design tool and I have a question about the implementation. Scipy-0.4.8 Numpy-0.9.6 I am generating a kaiser window with the following parameters: N = 128 Cutoff = ~4e7 width = .3 All of the special kaiser generation parameters in the function follow Oppenheim and Schafer well. Even the sinc function for linear phase is follows exactly. However, the last line of the firwin function is causing me some heartburn. filter_design.py 1538 win = get_window(window,N,fftbins=1) 1539 alpha = N//2 1540 m = numpy.arange(0,N) 1541 h = win*special.sinc(cutoff*(m-alpha)) 1542 return h / sum(h) Line 1542 of filter_design.py, "return h / sum(h)", normalizes the function where it doesn't seem necessary, at least in the kaiser window case. Without the normalization, the kaiser window already returns a value of 1 at the zero frequency point. This normalization scales all of the data, making the window difficult to use in the frequency domain. Can someone point me to the rationale for this line? Looking at the code, this seems to be a pretty recent change (within the last year/year and a half). Thanks, Eric Buehler eric <dot> buehler <at> smiths-aerospace <dot> com ****************************************** The information contained in, or attached to, this e-mail, may contain confidential information and is intended solely for the use of the individual or entity to whom they are addressed and may be subject to legal privilege. If you have received this e-mail in error you should notify the sender immediately by reply e-mail, delete the message from your system and notify your system manager. Please do not copy it for any purpose, or disclose its contents to any other person. The views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the company. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused, directly or indirectly, by any virus transmitted in this email. ******************************************

3 2

Moving numexpr to its own project
by David M. Cooke April 15, 2008

April 15, 2008

Hi, With the pending closing of the scipy.sandbox (anybody want to cough up more details on that, btw? When? How?), I want to move numexpr to its own project, likely hosted on Google Code. It's not something I think is "sciencey" enough for a scikit (plus, I don't like sharing a bug database with other, separate, projects), and it's more like weave (but simpler). Tim H., Francesc, Ivan? You're the main contributors; does this work for you? -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm(a)physics.mcmaster.ca

6 12

Status of sandbox.spline and my other code.
by John Travers April 7, 2008

April 7, 2008

Hi all, This email is intended to clarify the status of various bits of code I've committed/not committed so I don't leave too much of a mess after me. I'm about to start writing my PhD thesis, so I'll be too busy to work on scipy until September. 1. sandbox.spline stuff The module here was supposed to be a tidy up of scipy.interpolate. All of the dierkx functionality has been moved to f2py wrappers, which I think makes maintenance easier. However, it does not add any new functionality, and in retrospect appears to have been a waste of my time. So I'll leave to you to decide if you want to integrate it into scipy.interpolate or not; though it was originaly planned to be a seperate module to clear up the ambiguity between interpolation and smoothing spline functionality. However I have added 20 unit tests to the code (most of which simply check the wrapper against the pure fortran output, but still useful I think) which could easily be moved over to the current scipy.interpolate module. 2. Radial basis function module in sandbox (rbf) This code has recently had attention from Robert Hetland and is quite improved. I'm not sure if I will ever go into scipy though?? If not, I will put it into a scikit at some point. Related to this, the wiki page has been updated, however, it is at http://www.scipy.org/RadialBasisFunctions but should probably be in the Cookbook. I don't know how to move it over, so some pointers would be helpful. In addition there are four unused attachments I uploaded that could be deleted. 3. Boundary value ODE solver (BVP_SOLVER) I have fairly functional code interfacing to some very good BVP solver code (http://cs.smu.ca/~muir/BVP_SOLVER_Webpage.shtml). The license is good for scipy (as confirmed by the authors), but the code if fortran 95 - therefore cannot apparently go into scipy. I'll make a scikit when I get time (probably September). 4. An interface to the pikaia genetic algorithm routine I have a python interface to pikaia fortran code for genetic algorithm optimisation of real valued functions (http://www.hao.ucar.edu/Public/models/pikaia/pikaia.html). The fortran code itself is robust and highly optimised for numerical work, though it is nowhere near as general as the sandbox.ga module. I think it is useful to go in scipy.optimization alongside the annealing module. If people disagree I'll make it into a scikit. I hope this clears a few things up. Best regards, John Travers

4 4

Re: [SciPy-dev] Scipy, numpy and Abaqus
by Gilbert, Linda M CTR USAF AFRL/RXOC Feb. 27, 2008

Feb. 27, 2008

> _____________________________________________ > From: Gilbert, Linda M CTR USAF AFRL/MLOC > Sent: Thursday, January 03, 2008 11:01 AM > To: 'scipy-dev(a)scipy.org'; 'pearu(a)cens.ioc.ee' > Subject: Scipy, numpy and Abaqus > > Hello, > > Do you know how to get numpy and scipy to work from within abaqus? > > Thank you in advance, > > Linda Gilbert > AFRL/RXOC > RCF Information Systems > 937-904-7632 > >

3 3

ZVODE, a Double Precision Complex ODE Solver
by Nils Wagner Feb. 19, 2008

Feb. 19, 2008

This might be of interest. Nils From: Alan Hindmarsh <alanh(a)llnl.gov> Date: Wed, 6 Dec 2006 18:20:32 -0500 Subject: ZVODE, a Double Precision Complex ODE Solver Of the many solvers freely available for ODE initial value problems, there seem to be very few, if any, for the case of double precision complex dependent variables, even though there have been occasional user calls for this functionality. (Of course a complex system can be treated as a real system of twice the size, but direct treatment as a complex system is preferable.) I have provided such a solver: ZVODE. ZVODE is a straightforward modification of the Fortran ODE solver DVODE, for double precision real systems, making it part of a trio, SVODE/DVODE/ZVODE. ZVODE is available in two locations: (1) Netlib, in /ode (see file zvode.f), and (2) the LLNL CASC software site, www.llnl.gov/CASC/software.html (see VODE). The latter site also includes a demonstration program, zvdemo. Alan Hindmarsh Center for Applied Scientific Computing Lawrence Livermore National Laboratory

4 5

scipy.splinalg or bust
by Nathan Bell Feb. 1, 2008

Feb. 1, 2008

I've started migrating code to scipy.splinalg. Completed tasks: (1) linalg.iterative -> splinalg.isolve http://projects.scipy.org/scipy/scipy/changeset/3861 For backwards compatibility scipy.linalg imports all the iterative solvers from splinalg.isolve . This is needed to avoid breaking things like 'from scipy.linalg import cg' What's the best way to inform the user that these functions have moved? Is there a nice way to decorate these functions to issue a warning? Remaining tasks: (1) sandbox.lobpcg -> splinalg.eigen.lobpcg (2) sandbox.arpack -> splinalg.eigen.arpack (3) create splinalg.eigen.speigs (4) scipy.linsolve -> splinalg.dsolve (5) scipy.linsolve.umfpack -> scikit (6) create LinearOperator wrapper class and document it (7) document splinalg.dsolve (8) document splinalg.isolve (9) document splinalg.eigen I will start working on (4) now. Should I remove umfpack in the process? linalg Robert C, can you give us a timeline on (1) and (5)? Can (1) be done without symeig (perhaps using scipy.linalg.eig as a fallback)? Any volunteers for (2) and (3)? For reference: http://projects.scipy.org/scipy/scipy/wiki/ArpackWrapper Here are the remaining ARPACK tickets: http://projects.scipy.org/scipy/scipy/ticket/231 http://projects.scipy.org/scipy/scipy/ticket/366 http://projects.scipy.org/scipy/scipy/ticket/554 We discussed a plan for (6) in this thread: http://thread.gmane.org/gmane.comp.python.scientific.devel/7070/focus=7285 Any comments/suggestions are most welcome. -- Nathan Bell wnbell(a)gmail.com http://graphics.cs.uiuc.edu/~wnbell/

6 11

umfpack scikit
by Robert Cimrman Jan. 29, 2008

Jan. 29, 2008

Hi I am learning about scikits from http://scipy.org/scipy/scikits/wiki/ScikitsForDevelopers. I have tried to check-out the scikits trunk with the following result: svn co http://projects.scipy.org/scipy/scikits/trunk scikits svn: PROPFIND request failed on '/scipy/scikits' svn: PROPFIND of '/scipy/scikits': 200 Ok (http://projects.scipy.org) do I need some special account to do that (I have normal svn access (username rc))? I admit I did not follow all the discussion on scikits very closely, so sorry if I ask something obvious. After I get the sources, I should be able to copy shamelessly the structure of another scikit project to get started :) r.

1 1

documentation - looking to hire
by Joe Harrington Jan. 29, 2008

Jan. 29, 2008

I'd like to use Python to teach my astronomical data analysis class next semester. I tried using it last semester, and it was a big problem because the students could not find docs for the routines. They (and I, it was my first real use of Python) spent many hours at a stretch looking for docs on basic routines that had little or no docstrings. Lots of other things were missing, too, like searchable lists of routines, routines categorized by topic areas, getting-started and user guides, etc. Travis's book was too high-level for this purpose; it's written for a different audience. To use Python next semester, I'd need at least adequate docstrings for all of numpy and certain parts of scipy, and for some of the above-mentioned lists to be made, ideally automatically. I'm looking for options to hire one or more people to do the job, under some guidance from me and in concert with the community efforts. This could be as a direct hire or on a subcontract to Enthought or another institution. I have not made any detailed plans for this, but I'm guessing that one full time or two half-time people (a coder and a writer) could do a good-enough first cut at the job, provided they were already competent at numpy/scipy and were good writers. If you are interested in taking on this work, from roughly now until the end of the summer, please email me directly and let me know a bit about yourself and your availability. If you have ideas on how this might be done to best integrate with the community effort, respond to the list. Your collective response will shape my thinking on how to do this best, such as the level at which to hire. If you look at http://scipy.org/Developer_Zone , you'll see that I've written some basic description of the first two needs (which needs editing, but it is late), but that I haven't identified a doc lead nor a doc process. It would be very helpful if someone could describe for me what this process is. Is it as simple as writing a docstring according to http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/doc/HOWTO_DOCUMEN… and http://svn.scipy.org/svn/numpy/trunk/numpy/doc/example.py and entering it as a Trac ticket? Is there any review? How about keywords for indexing? What tools exist or are contemplated for searching? Is there any tool for extracting all the docstrings and putting them out either as files or in one big file, for searching or reading while not running python (or even being on the web)? Thanks, --jh-- Prof. Joseph Harrington Department of Physics MAP 414 4000 Central Florida Blvd. University of Central Florida Orlando, FL 32816-2385 (407) 823-3416 voice (407) 823-5112 fax (407) 823-2325 physics office jh(a)physics.ucf.edu

3 2

svn.scipy.org still down
by dmitrey Jan. 27, 2008

Jan. 27, 2008

AFAIK it was up for several seconds but it's down again for now, at least for several hours. ping svn.scipy.org PING new.scipy.org (216.62.213.231) 56(84) bytes of data. ^C --- new.scipy.org ping statistics --- 362 packets transmitted, 0 received, 100% packet loss, time 361196ms D.

5 4