Mailman 3 August 2009 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Continuous Wavelet Transform for SciPy
by Sean Arms Nov. 12, 2009

Nov. 12, 2009

Greetings! My name is Sean Arms and I'm a graduate student at the University of Oklahoma in the School of Meteorology. As part of my PhD research, I'm studying coherent structures in atmospheric boundary layer turbulence, primarily using in-situ observations and, secondarily, Direct Numerical Simulation (DNS) output. One common approach for detecting coherent structures in observational datasets relies on the use of the global wavelet power spectrum as estimated from a continuous wavelet transform (CWT). I know SciPy has a DWT impementation, and I've already been in contact with Filip. He recommeded that I post my code in hopes that it would add some momentum to the python-cwt development and create some feedback (I'm currently looking for a good place to post my code). I've implemented the CWT using pure python (that is, I did not write any C extension code myself - nothing to build), along with one mother wavelet (second derivitive of a Gaussian, or the Mexican Hat) - I'll be adding more Mother wavelets as I go along. I've made it a point to (try to) design my MotherWavelet class to be easily extendable. I'm working on documentation and a few tests at the moment, but so far my code compares well with other wavelet routines. The point of this email is to introduce myself and let the SciPy dev community know that I am willing to help develop CWT support for SciPy - I'll already be doing the work for my research, so I might as well put in the extra effort to make is usable by the larger community! Cheers! Sean Arms Graduate Research Assistant School of Meteorology University of Oklahoma

4 7

SciPy Milestones doc editor link broken
by Ralf Gommers Aug. 30, 2009

Aug. 30, 2009

Hi, The link to the SciPy Milestones (at the top of every page) in the doc wiki is pointing to the NumPy version again. Can one of the wiki admins please change the link back to http://docs.scipy.org/scipy/Milestones/ ? Thanks, Ralf

1 0

EPD 5.0 released
by Ilan Schnell Aug. 29, 2009

Aug. 29, 2009

Hello, I am pleased to announce that EPD (Enthought Python Distribution) version 5.0.0 has been released. You may find more information about EPD, as well as download a 30 day free trial, here: http://www.enthought.com/products/epd.php This release contains updates to a large number of packages. You can find the release notes here: https://svn.enthought.com/epd/wiki/Py25/5.0.0/RelNotes About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python Programming Language, including over 80 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, database adapters, and a lot of other tools right out of the box. http://www.enthought.com/products/epdlibraries.php It is currently available as a single-click installer for Windows XP (x86), Mac OS X (a universal binary for OS X 10.4 and above), RedHat 3, 4 and 5, as well as Solaris 10 (x86 and x86_64/amd64). EPD is free for academic use. An annual subscription including installation support is available for individual and commercial use. Additional support options, including customization, bug fixes and training classes are also available: http://www.enthought.com/products/support_level_table.php - Ilan

1 0

Callback arguments in Sphinx Documentation
by John Salvatier Aug. 28, 2009

Aug. 28, 2009

Hello all, I am writing the Documentation for scikits.bvp_solver using Sphinx. bvp_solver uses a lot of relatively complex callback arguments, so I am looking for a good way to document their format. Nesting ".. method::" directives does not seem to work. Does anyone have any ideas about how to document the signatures of callback arguments elegantly; has anyone dealt with this before? Best Regards, John Salvatier

2 1

help with plot directive in scipy.stats.tutorial
by josef.pktd＠gmail.com Aug. 27, 2009

Aug. 27, 2009

Since the stats.tutorial has now been merged into svn, I would like to clean up the plot directives. When I wrote this initially, I didn't realize that the plot commands do not have access to previously defined variables. I would like to replace the current code with plot directives using script files instead of the inline plot directive. Because, I'm currently not able or setup to build the docs, I would like to know if this is ok or whether someone can test this. The attached files are my original scripts, I haven't really reviewed it just checked that they work. Thanks, Josef .. plot:: examples/normdiscr_plot1.py :align: center :include-source: 0 .. plot:: examples/normdiscr_plot2.py :align: center :include-source: 0

3 4

Please help!
by Niall Moran Aug. 26, 2009

Aug. 26, 2009

Hi, I am trying to find the lowest eigenvalues and eigenvectors of a sparse complex matrix using scipy. I could not find any suitable looking routines in the stable version 0.6.0 so I have compiled the and installed the development version 0.8.0.dev5798 and have been trying to use the scipy.sparse.linalg.eigen routine which calls the relevant fortran arpack routines. This is working great for real matrices but is not working for complex matrices. Even when all the matrix elements are real and the matrix is just of complex type. The error I get is "RuntimeError: Error info=-9999 in arpack". on line 224 of .../python2.6/dist-packages/scipy/sparse/linalg/eigen/arpack/arpack.py. The matrix I am using is of type complex128 so the arpack function called is znaupd (http://www.caam.rice.edu/software/ARPACK/UG/node138.html). Looking at the documentation for a return value of info=-9999 "= -9999: Could not build an Arnoldi factorization. c User input error highly likely. Please c check actual array dimensions and layout. c IPARAM(5) returns the size of the current Arnoldi c factorization." suggests a user input error is likely. IPARAM(5) has a value of 4. The page for znaupd (linked above) gives details on the expected size and type of each of the arguments. I have went through each of these in the arpack.py file and checked to make sure the python code is creating the correct size arrays and everything looks fine so I am at a complete loss as to what is going wrong as the real version is working fine. The only thing I have been thinking is that maybe there is something different about the way python and fortran store complex arrays and maybe there is some problem in the crossover. Can anyone suggest how I may debug this further? Please see the attached source (test_complex.py) which creates a simple 16x16 matrix with all real elements and tries to find the eigenvalues when this is stored as a complex sparse matrix and when it is stored as a real sparse matrix. Niall.

6 11

botched PDF
by David Warde-Farley Aug. 25, 2009

Aug. 25, 2009

Someone brought it to my attention on IRC that the reference guide PDF is currently borked: http://docs.scipy.org/doc/scipy/scipy-ref.pdf Can someone with requisite permissions rebuild it at your earliest convenience? David

3 2

draft: NumPy User Guide introduction
by Andrew Hawryluk Aug. 25, 2009

Aug. 25, 2009

I have posted a first draft of a NumPy User Guide introduction. Hopefully it is both useful and factually correct, but please jump in where you see room for improvement. I am assuming familiarity with Python programming, but no prior knowledge of NumPy. http://docs.scipy.org/numpy/docs/numpy-docs/user/introduction.rst/ Andrew

11 22

[timeseries][patch] Accept Date(datetime) as Date(datetime=datetime)
by Matteo Bertini Aug. 25, 2009

Aug. 25, 2009

Sometime I get lost in a nasty bug because I forget to use the named argument Date(datetime=datetime). I propose a small patch to make 'datetime' take the same way 'string' is already following. http://scipy.org/scipy/scikits/ticket/102 [---8<---] if (value %% (PyDateTime_Check(value) || PyDate_Check(value))) { if (!datetime) { datetime = value; } value = NULL; } // datetime = (datetime||value), value = NULL [---8<---] Ciao, Matteo Bertini

1 0