Mailman 3 January 2012 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

prepending underscores to private module names
by Ralf Gommers May 16, 2012

May 16, 2012

Hi, A while ago we discussed making a clear distinction between public and private modules by using underscores consistently, see http://thread.gmane.org/gmane.comp.python.scientific.devel/14936. I've done this for a few modules now, see https://github.com/rgommers/scipy/tree/underscores. Most files could simply be underscored, some other would conflict with extension modules so I also had to append _py to the name. Also I added a file doc/source/api.rst that states which modules are public (up for discussion of course). With google code search I checked how common it is to import from modules that are supposed to be private, for example "from scipy.optimize.optimize import fmin". It's not very common but does happen, so deprecation warnings should be put in. Before going further I'd like to check that this looks okay to people. What do you think? Cheers, Ralf <https://github.com/rgommers/scipy/tree/underscores>

4 4

studentized range approximations
by Roger Lew March 28, 2012

March 28, 2012

Hi, I have implemented some approximations for studentized range quantiles and probabilities based on John R. Gleason's (1999) "An accurate, non-iterative approximation for studentized range quantiles." Computational Statistics & Data Analysis, (31), 147-158. Probability approximations rely on scipy.optimize.fminbound. The functions accept both scalars or array-like data thanks to numpy.vectorize. A fair amount of validation and testing has been conducted on the code. More details can be found here: http://code.google.com/p/qsturng-py/ I welcome any thoughts as to whether you all think this might be useful to add to SciPy or make into a scikit. Any general comments would be helpful as well. I should mention I'm a cognitive neuroscientist by trade, my use of statistical jargon probably isn't that good. Regards, Roger Roger Lew

3 5

Update on pythonequations unit tests
by James Phillips March 14, 2012

March 14, 2012

From: Alan G Isaac <alan.isaac <at> gmail.com> Subject: Re: Subversion scipy.stats irregular problem with source code example<http://news.gmane.org/find-root.php?message_id=%3c4CA22FA3.5080402%40gmail.…> Newsgroups: gmane.comp.python.scientific.devel<http://news.gmane.org/gmane.comp.python.scientific.devel> Date: 2010-09-28 18:10:43 GMT (1 year, 2 weeks, 1 day, 15 hours and 46 minutes ago) As long as you can provide unit tests, I don't see a problem. But you and Skipper shd work out the details. Cheers, Alan Isaac ------------------------------------------------------------------------------------------------------------------ I have 76 unit tests so far, see below. I'm working on more, and expect to be done sometime around the end of this year. James Phillips http://zunzun.com testCalculateCoefficientAndFitStatisticsUsingDE_2D (Test_CalculateCoefficientAndFitStatistics.TestCalculateCoefficientAndFitStatistics) ... ok testCalculateCoefficientAndFitStatisticsUsingSpline_2D (Test_CalculateCoefficientAndFitStatistics.TestCalculateCoefficientAndFitStatistics) ... ok testDataCache_2D (Test_DataCache.TestDataCache) ... ok testDataCache_3D (Test_DataCache.TestDataCache) ... ok testReducedDataSize_2D (Test_DataCache.TestDataCache) ... ok testConversionOfColumns_ASCII_2D_NoWeights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_2D_NoWeights_ExampleData (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_2D_Weights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_3D_NoWeights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_3D_Weights (Test_DataConverterService.TestConversions) ... ok testPolynomialLinear_2D (Test_EquationPolynomials.TestEquationPolynomials) ... ok testPolynomialLinear_3D (Test_EquationPolynomials.TestEquationPolynomials) ... ok testExtendedVersion_Exponential_WithExponentialDecayAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialDecay_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialGrowthAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialGrowth_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearDecayAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearDecay_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearGrowthAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearGrowth_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Inverse_Exponential_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Inverse_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Reciprocal_Exponential_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Reciprocal_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testArcTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testCosine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testExponential_VariableTimesNegativeOne_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testExponential_VariableUnchanged_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicCosine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicSine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testOffset_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeOne_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeOne_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeTwo_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeTwo_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeZeroPointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_OnePointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_Two_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_Two_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_ZeroPointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testSine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testVariableUnchanged_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testSplineSolve_2D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testSplineSolve_3D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testUserDefinedFunctionSolve_2D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testUserDefinedFunctionSolve_3D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testConversionFromCppToCSHARP (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToJAVA (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToMATLAB (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToPYTHON (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToSCILAB (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToVBA (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testGenerationOfCPP (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfCSHARP (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfJAVA (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfMATLAB (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfPYTHON (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfSCILAB (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfVBA (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testSolveUsingDE_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingDE_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingLevenbergMarquardt_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingLevenbergMarquardt_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingLinear_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingLinear_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingODR_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingODR_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_SSQABS_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_SSQREL_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSpline_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSpline_3D (Test_SolverService.TestSolverService) ... ok ---------------------------------------------------------------------- Ran 76 tests in 94.615s OK

1 2

scipy.io.wavfile
by Jean-Louis Durrieu Feb. 2, 2012

Feb. 2, 2012

Hi everyone, I have been using the scipy.io.wavfile for some time now. I am quite thankful for the person(s) who contributed that, as it makes it easy for me to research, develop and have other people use my programs. And it's easier to install than audiolab (sorry david), but way less powerful (is that better? :) ). I just found a strange behaviour, and wanted to know what could be done: I have a few wav files for which I got, with scipy.io.wavfile.read, the right sampling rate, but a bad data chunk (actually strings, instead of int). As it were, these files (from the MIREX tempo tracking challenge http://www.music-ir.org/mirex/wiki/2011:Audio_Beat_Tracking) do have strange chunks, appended to the data chunk, which contain stuff like annotations or labels. While audacity or any other program does not bother with these, scipy.io.wavfile.read still reads them and, worse of all, replaces the correct data chunk with these labels (and we get a warning saying it does not understand the data type, and reads rubbish instead). Well, I was wondering if such a behaviour was desired, or if there should not just be something like: * If there is a data chunk, read it. * If there are many such data chunks, keep the first, send a warning. * If there is a data chunk, and other "funny ones", keep the data chunk, send warning about the others (like providing their chunk_ids?) * If there is no data chunk, send an error (with list of found chunks?). That would at least make it not break at the first difficulty, right? Of course, I might be wrong assuming the above cases are the only ones that could occur, but one has to start somewhere, eh? Additionally, I was wondering why numpy does not recognize 24 bits integers. It would seem quite some people work with 24 bit audio, so maybe some conversion should also be allowed there, although using numpy.fromfile may not work anymore (except if we add 24 bit integers to the allowed data types...). For this matter, I m more curious than pushy, so no need to stress about it :-) Best regards ! Jean-Louis

3 5

Scikit Signal or similar
by Stuart Mumford Feb. 1, 2012

Feb. 1, 2012

Hello, I am interested in contributing to the scikit-signal project, I have been working on a wavelet package recently which I believe would be useful. https://github.com/Cadair/scikit-signal I have been working on this with one of my friends in my office, to analyse one dataset (which is in the git) and it reproduces the same result as his existing IDL code. There are a few other things we are still working on implementing to this, (significance contouring, more flexibility in parameters). On a side note, we also have use for the pyHHT project, to analyse the same data. Also this is my first attempt at contributing to something like this, so sorry if I go wrong! Stuart

5 4

Cannot build scipy from trunk
by Nils Wagner Jan. 31, 2012

Jan. 31, 2012

scipy/special/_cephesmodule.c: In function scipy_special_raise_warning: scipy/special/_cephesmodule.c:1055:5: error: expected =, ,, ;, asm or __attribute__ before char scipy/special/_cephesmodule.c:1062:5: error: __save__ undeclared (first use in this function) scipy/special/_cephesmodule.c:1062:5: note: each undeclared identifier is reported only once for each function it appears in scipy/special/_cephesmodule.c:1063:5: error: expected ; before PyErr_WarnEx scipy/special/_cephesmodule.c:1065:1: error: expected ; before } token scipy/special/_cephesmodule.c:1076:18: error: invalid storage class for function errprint_func scipy/special/_cephesmodule.c:1091:3: error: initializer element is not constant scipy/special/_cephesmodule.c:1091:3: error: (near initialization for methods[0].ml_meth) scipy/special/_cephesmodule.c:1177:1: error: expected declaration or statement at end of input In file included from /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1972:0, from /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:17, from /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/arrayobject.h:14, from scipy/special/_cephesmodule.c:12: /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API" scipy/special/_cephesmodule.c: In function scipy_special_raise_warning: scipy/special/_cephesmodule.c:1055:5: error: expected =, ,, ;, asm or __attribute__ before char scipy/special/_cephesmodule.c:1062:5: error: __save__ undeclared (first use in this function) scipy/special/_cephesmodule.c:1062:5: note: each undeclared identifier is reported only once for each function it appears in scipy/special/_cephesmodule.c:1063:5: error: expected ; before PyErr_WarnEx scipy/special/_cephesmodule.c:1065:1: error: expected ; before } token scipy/special/_cephesmodule.c:1076:18: error: invalid storage class for function errprint_func scipy/special/_cephesmodule.c:1091:3: error: initializer element is not constant scipy/special/_cephesmodule.c:1091:3: error: (near initialization for methods[0].ml_meth) scipy/special/_cephesmodule.c:1177:1: error: expected declaration or statement at end of input error: Command "/usr/bin/gcc -fno-strict-aliasing -DNDEBUG -fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -fPIC -I/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include -I/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 -c scipy/special/_cephesmodule.c -o build/temp.linux-x86_64-2.6/scipy/special/_cephesmodule.o" failed with exit status 1

2 1

scikit webpage error
by alex arsenovic Jan. 31, 2012

Jan. 31, 2012

to whom it may concern, this page on the scikits site, gives me an error http://scikits.appspot.com/scikits

2 2

scikit odes - release 1.0.0 via pypi and github
by Benny Malengier Jan. 27, 2012

Jan. 27, 2012

Scipy devels, The new odes scikit code is available from http://pypi.python.org/pypi/scikits.odes/1.0.0 and https://github.com/bmcage/odes which is tested with python 2.7 and 3.2. I don't think I'm allowed to remove the old scikit code in the svn repo at http://projects.scipy.org/scikits/browser/trunk/odes Can somebody delete those so as to avoid confusion? For those interested in the DAE and ODE api, see short examples: https://github.com/bmcage/odes/blob/master/docs/src/examples/ode/simpleosci… https://github.com/bmcage/odes/blob/master/docs/src/examples/simpleoscillat… Benny

1 0

least squares solvers
by Sturla Molden Jan. 27, 2012

Jan. 27, 2012

Why is the non-linear LS solver in sp.optimize called leastsq, whereas the linear solver in sp.linalg is called lstsq? Wouldn't a consistent name be better? sp.linalg.lstsq uses SVD, by lapack driver *gelss. If we don't need singular values, solving by QR is faster (lapack driver *gels). (I actually use Fortran just to get DGELS instead of DGELSS.) Sturla

3 4