Mailman 3 May 2009 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Continuous Wavelet Transform for SciPy
by Sean Arms Nov. 12, 2009

Nov. 12, 2009

Greetings! My name is Sean Arms and I'm a graduate student at the University of Oklahoma in the School of Meteorology. As part of my PhD research, I'm studying coherent structures in atmospheric boundary layer turbulence, primarily using in-situ observations and, secondarily, Direct Numerical Simulation (DNS) output. One common approach for detecting coherent structures in observational datasets relies on the use of the global wavelet power spectrum as estimated from a continuous wavelet transform (CWT). I know SciPy has a DWT impementation, and I've already been in contact with Filip. He recommeded that I post my code in hopes that it would add some momentum to the python-cwt development and create some feedback (I'm currently looking for a good place to post my code). I've implemented the CWT using pure python (that is, I did not write any C extension code myself - nothing to build), along with one mother wavelet (second derivitive of a Gaussian, or the Mexican Hat) - I'll be adding more Mother wavelets as I go along. I've made it a point to (try to) design my MotherWavelet class to be easily extendable. I'm working on documentation and a few tests at the moment, but so far my code compares well with other wavelet routines. The point of this email is to introduce myself and let the SciPy dev community know that I am willing to help develop CWT support for SciPy - I'll already be doing the work for my research, so I might as well put in the extra effort to make is usable by the larger community! Cheers! Sean Arms Graduate Research Assistant School of Meteorology University of Oklahoma

4 7

Porting SciPy to Py3k GSOC project
by ross smith July 19, 2009

July 19, 2009

Hello everyone, I am interested in porting SciPy/NumPy to Py3k. I've been working this past school year to port an existing code base to py3k for a research group on campus. A lot of the code relies on SciPy and NumPy but the scope of my project didn't let me work on porting either project, to my dismay. I'd love the opportunity to port a project I use heavily in my own code and gain a better understanding of how it works. We are supposed to contact the group we would be working with, to flesh out the details of our application. I've looked at the application and the only thing I know I'll need significant help with is the Milestones portion. Of course, Any and all suggestions are welcome! thank you, Ross Smith (Gaurdro on Freenode)

2 3

rgb_to_hsv in scipy.misc ? (was: [Numpy-discussion] Optimizing speed for large-array inter-element algorithms (specifically, color space conversion))
by Nicolas Pinto July 7, 2009

July 7, 2009

Hello, Would it be possible to include the following rgb to hsv conversion code in scipy (probably in misc along with misc.imread, etc.) ? What do you think? Thanks in advance. Best regards, -- Nicolas Pinto Ph.D. Candidate, Brain & Computer Sciences Massachusetts Institute of Technology, USA http://web.mit.edu/pinto # ------------------------------------------------------------------------------ import numpy as np def rgb_to_hsv_arr(arr): """ fast rgb_to_hsv using numpy array """ # adapted from Arnar Flatberg # http://www.mail-archive.com/numpy-discussion@scipy.org/msg06147.html # it now handles NaN properly and mimics colorsys.rgb_to_hsv output arr = arr/255. out = np.empty_like(arr) arr_max = arr.max(-1) delta = arr.ptp(-1) s = delta / arr_max s[delta==0] = 0 # red is max idx = (arr[:,:,0] == arr_max) out[idx, 0] = (arr[idx, 1] - arr[idx, 2]) / delta[idx] # green is max idx = (arr[:,:,1] == arr_max) out[idx, 0] = 2. + (arr[idx, 2] - arr[idx, 0] ) / delta[idx] # blue is max idx = (arr[:,:,2] == arr_max) out[idx, 0] = 4. + (arr[idx, 0] - arr[idx, 1] ) / delta[idx] out[:,:,0] = (out[:,:,0]/6.0) % 1.0 out[:,:,1] = s out[:,:,2] = arr_max # rescale back to [0, 255] out *= 255. # remove NaN out[np.isnan(out)] = 0 return out

4 5

Setup doesn't pickup Intel Fortran compiler
by Matthieu Brucher June 3, 2009

June 3, 2009

Hi, I have trouble getting Scipy (and numpy) using the correct Fortran compiler : Found executable /appli/intel/10.1.018/intel64/cce/bin/icc customize GnuFCompiler Found executable /appli/intel/10.1.018/intel64/fce/bin/ifort gnu: no Fortran 90 compiler found Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found customize IntelFCompiler customize LaheyFCompiler Could not locate executable lf95 lahey: no Fortran 90 compiler found lahey: no Fortran 90 compiler found customize PGroupFCompiler Could not locate executable pgf90 pg: no Fortran 90 compiler found pg: no Fortran 90 compiler found customize AbsoftFCompiler Could not locate executable f90 absoft: no Fortran 90 compiler found absoft: no Fortran 90 compiler found absoft: no Fortran 90 compiler found absoft: no Fortran 90 compiler found absoft: no Fortran 90 compiler found absoft: no Fortran 90 compiler found customize NAGFCompiler Found executable /usr/bin/f95 customize VastFCompiler customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize CompaqFCompiler Could not locate executable fort compaq: no Fortran 90 compiler found compaq: no Fortran 90 compiler found customize IntelItaniumFCompiler customize IntelEM64TFCompiler customize Gnu95FCompiler Found executable /usr/bin/gfortran customize Gnu95FCompiler customize UnixCCompiler customize UnixCCompiler using scons Found executable /appli/intel/10.1.018/intel64/cce/bin/icpc Why is it trying to search for another Fortran compiler when it actually found the Intel one? I used python setup.py install or python setup.py install --fcompiler=intelem, but the output was the same :| Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher

2 9

0.7.1: new schedule
by David Cournapeau May 29, 2009

May 29, 2009

Hi, I have changed the schedule for scipy 0.7.1. I set up the release date for the first June. Ideally, I would like that all backports go into the 0.7.x branch within the end of next WE. Is that ok with everyone involved working on 0.7.x branch ? Some matlab IO problems have been fixed by Matthew, Pauli and Josef have backported some fixes in stats and special, and I have fixed everything I wanted to see fixed for 0.7.x. cheers, David

4 10

Fixing correlate: handling api breakage ?
by David Cournapeau May 28, 2009

May 28, 2009

Hi, I have taken a look at the correlate function in scipy.signal. There are several problems with it. First, it is wrong on several accounts: - It assumes that the correlation of complex numbers corresponds to complex multiplication, but this is not the definition followed by most textbooks, at least as far as signal processing is concerned. - More significantly, it is wrong with respect to the ordering: it assumes that correlate(a, b) == correlate(b, a), which is not true in general. It may be argued that the first problem is a matter of definition, but I don't think the 2nd one is. I would expect some people depending on the functionality, though. What should we do ? Adding a new function which implements the usual definition, replacing the current one or something else ? cheers, David

2 6

Updating blitz++ within weave?
by Norbert Nemec May 27, 2009

May 27, 2009

Hi there, it seems that the blitz++ code within weave is branched off some ancient version and only minimally patched. Wouldn't it make sense to pull the latest version from the blitz CVS repo? Greetings, Norbert -- Neu: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate + Telefonanschluss für nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02

1 0

RFR: #902 firwin upgrades
by Tom K. May 25, 2009

May 25, 2009

(I sent this to scipy-dev(a)scipy.org, thinking it would appear here, I am horribly confused - sorry for the duplication :-) SIMPLE REVIEW NEEDED - SINGLE FUNCTION Link to Trac ticket: http://projects.scipy.org/scipy/ticket/902 This is adding a highpass, bandpass, bandstop, and multiband capability to signal.firwin. I submitted a patch attached to the ticket to filter_design.py. I also added a test class in tests/test_filter_design.py. Let me know if there are any problems with the patch as this is my first patch to scipy and I am new to SVN. To do: - consider accepting a list of tuples of band edges for cutoff, since that seems more natural for describing multiband filters. - figure out what to do about even length filters which are currently not linear-phase (that is symmetric). Also, I seem to be having trouble with the trac system, it's saying that the data base is locked. It won't let me add notes or change the status to "needs review"... Cheers, Tom K.

2 4

RFR: Add io to ndimage
by Stéfan van der Walt May 25, 2009

May 25, 2009

Hi all, It's bothered me for a long time that our image reading routines are hidden in scipy.misc. Here is a patch that implements image reading via the PIL array interface. Please let me know whether you like this or not. Regards Stéfan

2 2