Mailman 3 October 2011 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 25, 2013

Nov. 25, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

prepending underscores to private module names
by Ralf Gommers May 16, 2012

May 16, 2012

Hi, A while ago we discussed making a clear distinction between public and private modules by using underscores consistently, see http://thread.gmane.org/gmane.comp.python.scientific.devel/14936. I've done this for a few modules now, see https://github.com/rgommers/scipy/tree/underscores. Most files could simply be underscored, some other would conflict with extension modules so I also had to append _py to the name. Also I added a file doc/source/api.rst that states which modules are public (up for discussion of course). With google code search I checked how common it is to import from modules that are supposed to be private, for example "from scipy.optimize.optimize import fmin". It's not very common but does happen, so deprecation warnings should be put in. Before going further I'd like to check that this looks okay to people. What do you think? Cheers, Ralf <https://github.com/rgommers/scipy/tree/underscores>

4 4

studentized range approximations
by Roger Lew March 29, 2012

March 29, 2012

Hi, I have implemented some approximations for studentized range quantiles and probabilities based on John R. Gleason's (1999) "An accurate, non-iterative approximation for studentized range quantiles." Computational Statistics & Data Analysis, (31), 147-158. Probability approximations rely on scipy.optimize.fminbound. The functions accept both scalars or array-like data thanks to numpy.vectorize. A fair amount of validation and testing has been conducted on the code. More details can be found here: http://code.google.com/p/qsturng-py/ I welcome any thoughts as to whether you all think this might be useful to add to SciPy or make into a scikit. Any general comments would be helpful as well. I should mention I'm a cognitive neuroscientist by trade, my use of statistical jargon probably isn't that good. Regards, Roger Roger Lew

3 5

Update on pythonequations unit tests
by James Phillips March 14, 2012

March 14, 2012

From: Alan G Isaac <alan.isaac <at> gmail.com> Subject: Re: Subversion scipy.stats irregular problem with source code example<http://news.gmane.org/find-root.php?message_id=%3c4CA22FA3.5080402%40gmail.…> Newsgroups: gmane.comp.python.scientific.devel<http://news.gmane.org/gmane.comp.python.scientific.devel> Date: 2010-09-28 18:10:43 GMT (1 year, 2 weeks, 1 day, 15 hours and 46 minutes ago) As long as you can provide unit tests, I don't see a problem. But you and Skipper shd work out the details. Cheers, Alan Isaac ------------------------------------------------------------------------------------------------------------------ I have 76 unit tests so far, see below. I'm working on more, and expect to be done sometime around the end of this year. James Phillips http://zunzun.com testCalculateCoefficientAndFitStatisticsUsingDE_2D (Test_CalculateCoefficientAndFitStatistics.TestCalculateCoefficientAndFitStatistics) ... ok testCalculateCoefficientAndFitStatisticsUsingSpline_2D (Test_CalculateCoefficientAndFitStatistics.TestCalculateCoefficientAndFitStatistics) ... ok testDataCache_2D (Test_DataCache.TestDataCache) ... ok testDataCache_3D (Test_DataCache.TestDataCache) ... ok testReducedDataSize_2D (Test_DataCache.TestDataCache) ... ok testConversionOfColumns_ASCII_2D_NoWeights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_2D_NoWeights_ExampleData (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_2D_Weights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_3D_NoWeights (Test_DataConverterService.TestConversions) ... ok testConversionOfColumns_ASCII_3D_Weights (Test_DataConverterService.TestConversions) ... ok testPolynomialLinear_2D (Test_EquationPolynomials.TestEquationPolynomials) ... ok testPolynomialLinear_3D (Test_EquationPolynomials.TestEquationPolynomials) ... ok testExtendedVersion_Exponential_WithExponentialDecayAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialDecay_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialGrowthAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithExponentialGrowth_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearDecayAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearDecay_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearGrowthAndOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithLinearGrowth_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Inverse_Exponential_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Inverse_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Reciprocal_Exponential_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testExtendedVersion_Reciprocal_Exponential_WithOffset_2D (Test_ExtendedVersionHandlers.TestExtendedVersionHandlers) ... ok testArcTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testCosine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testExponential_VariableTimesNegativeOne_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testExponential_VariableUnchanged_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicCosine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicSine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testHyperbolicTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testOffset_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeOne_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeOne_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeTwo_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeTwo_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_NegativeZeroPointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_OnePointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_Two_OfLog_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_Two_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testPower_ZeroPointFive_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testSine_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testTangent_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testVariableUnchanged_Term (Test_IndividualPolyFunctions.TestPolyFunctions) ... ok testSplineSolve_2D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testSplineSolve_3D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testUserDefinedFunctionSolve_2D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testUserDefinedFunctionSolve_3D (Test_ModelSolveMethods.TestModelSolveMethods) ... ok testConversionFromCppToCSHARP (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToJAVA (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToMATLAB (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToPYTHON (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToSCILAB (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testConversionFromCppToVBA (Test_OutputSourceCodeService.TestConversionsFromCPP) ... ok testGenerationOfCPP (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfCSHARP (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfJAVA (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfMATLAB (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfPYTHON (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfSCILAB (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testGenerationOfVBA (Test_OutputSourceCodeService.TestGenerationOfOutputSourceCode) ... ok testSolveUsingDE_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingDE_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingLevenbergMarquardt_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingLevenbergMarquardt_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingLinear_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingLinear_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingODR_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingODR_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_3D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_SSQABS_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSimplex_SSQREL_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSpline_2D (Test_SolverService.TestSolverService) ... ok testSolveUsingSpline_3D (Test_SolverService.TestSolverService) ... ok ---------------------------------------------------------------------- Ran 76 tests in 94.615s OK

1 2

installation of scipy on Mac OS X 10.7
by Ben Willmore Jan. 27, 2012

Jan. 27, 2012

I have spent a day attempting to get a version of scipy that passes unit tests on Mac OS X 10.7. I hope the results of my investigations (below) are useful to others trying to do the same thing. Ben == Introduction I have attempted to obtain a version of scipy on Mac OS X 10.7 that passes unit tests, both by downloading it, or by compiling against the system's python, using the three C compilers available, and various compile flags. == Conclusions 1. Both Enthought Python 7.1 and the scipy superpack crash during scipy unit tests. 2. Compilation with the default C compiler (llvm-gcc-4.2) also leads to a crash during unit tests. 3. Compilation using either of the other C compilers (gcc-4.2 or clang) allows scipy to complete unit tests, only if the fortran flag --ff2c is used. In both cases, scipy passes almost all tests -- producing errors on one, and failing two more. With gcc-4.2, numpy passes all tests. With clang, numpy fails one test. 4. The following recipe was most successful for me: * Remove everything in /usr/local and /Library/Python/2.7/site-packages (BEWARE! These directories may well contain stuff you care about) * Install gfortran from here: http://r.research.att.com/gfortran-lion-5666-3.pkg mkdir ~/tmp cd ~/tmp git clone git://github.com/numpy/numpy.git cd numpy export CC=gcc-4.2 export CXX=g++-4.2 export FFLAGS=-ff2c python setupegg.py build --fcompiler=gfortran sudo python setupegg.py install cd ~/tmp git clone git://github.com/scipy/scipy.git cd scipy export CC=gcc-4.2 export CXX=g++-4.2 export FFLAGS=-ff2c python setupegg.py build --fcompiler=gfortran sudo python setupegg.py install (repeat for matplotlib, ipython ...) == Details of test results = Enthought python 7.1 numpy.test(): OK scipy.test(): Crash (Segmentation fault) = Scipy superpack numpy.test(): 3 failures: FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 1, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -1, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 0.0, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -0.0, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, inf, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, nan, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, 1, 0.0, inf) scipy.test(): Crash (Abort trap) = Compilation from scratch (see above for recipe) = llvm-gcc-4.2 = (no compiler flags) numpy.test(): OK scipy.test(): Crash (Segmentation fault: 11) = gcc-4.2 = CC=gcc-4.2 CXX=g++-4.2 numpy.test(): OK scipy.test(): Hang (during ARPACK tests) = clang = CC=clang CXX=clang++ numpy.test(): 1 failure: FAIL: Test basic arithmetic function errors AssertionError: Type <type 'numpy.complex64'> did not raise fpe error ''. scipy.test(): Hang (during ARPACK tests) = llvm-gcc-4.2 and -ff2c = FFLAGS=-ff2c numpy.test(): OK scipy.test(): Crash (Segmentation fault: 11) = gcc-4.2 and -ff2c = CC=gcc-4.2 CXX=g++-4.2 FFLAGS=-ff2c numpy.test(): OK scipy.test(): 1 error, 2 failures: ERROR: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'F', 2, 'SA', None, None, <function aslinearoperator at 0x1098cd500>) FAIL: test_iterative.test_convergence(<function bicg at 0x1098cdd70>, <nonsymposdef>) FAIL: test_expon (test_morestats.TestAnderson) = clang and -ff2c = CC=clang CXX=clang++ FFLAGS=-ff2c numpy.test(): 1 failure: FAIL: Test basic arithmetic function errors AssertionError: Type <type 'numpy.complex64'> did not raise fpe error ''. scipy.test(): 1 error, 2 failures: ERROR: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'F', 2, 'SA', None, None, <function aslinearoperator at 0x1098cd500>) FAIL: test_iterative.test_convergence(<function bicg at 0x1098cdd70>, <nonsymposdef>) FAIL: test_expon (test_morestats.TestAnderson)

6 6

scipy 0.10.0 release schedule update
by Ralf Gommers Nov. 2, 2011

Nov. 2, 2011

Hi all, For those of you who are wondering what the status of the 0.10.0 release is, we should have RC1 ready very soon. The blocker was/is that single precision Arpack routines on OS X are completely broken. For now the plan is to simply switch to double precision. See https://github.com/scipy/scipy/pull/98 There was one other unresolved issue that came up for the beta release, which is that on Windows there's a Cython bug which prevents compilation with any released Cython version. We can't switch to the latest released Cython + bugfix, because the io.matlab Cython files can't be compiled with them: http://projects.scipy.org/scipy/ticket/1524. My assessment of this issue is that it's not a blocker for the release. Please speak up if you disagree. Cheers, Ralf

3 9

tplquad alternative
by Vladímir Rodríguez Oct. 31, 2011

Oct. 31, 2011

I have a working alternative to tplquad which I builded from http://dl.acm.org/citation.cfm?id=155785&dl=ACM&coll=DL&CFID=66021551&CFTOK… via F2PY. For testing aim you can use Scilab's int3d function (also from this code), I found it faster and smarter. How can we include it in standart Scipy library? Vladímir

3 3

oops, don't trust a test even if it's in the test suite
by josef.pktd＠gmail.com Oct. 31, 2011

Oct. 31, 2011

I made a mistake when I wrote some tests for scipy.stats.distributions, typo and thinko, and I didn't use a pre-made tests. stats() for some distributions returns an incorrect variance, I thought it's only skew and kurtosis that are still wrong. http://projects.scipy.org/scipy/ticket/1546 Josef

1 0

[ANN] SciPy India 2011 Abstracts due November 2nd
by Jarrod Millman Oct. 29, 2011

Oct. 29, 2011

========================== SciPy 2011 Call for Papers ========================== The third `SciPy India Conference <http://scipy.in>`_ will be held from December 4th through the 7th at the `Indian Institute of Technology, Bombay (IITB) <http://www.iitb.ac.in/>`_ in Mumbai, Maharashtra India. At this conference, novel applications and breakthroughs made in the pursuit of science using Python are presented. Attended by leading figures from both academia and industry, it is an excellent opportunity to experience the cutting edge of scientific software development. The conference is followed by two days of tutorials and a code sprint, during which community experts provide training on several scientific Python packages. We invite you to take part by submitting a talk abstract on the conference website at: http://scipy.in Talk/Paper Submission ========================== We solicit talks and accompanying papers (either formal academic or magazine-style articles) that discuss topics regarding scientific computing using Python, including applications, teaching, development and research. We welcome contributions from academia as well as industry. Important Dates ========================== November 2, 2011, Wednesday: Abstracts Due November 7, 2011, Monday: Schedule announced November 28, 2011, Monday: Proceedings paper submission due December 4-5, 2011, Sunday-Monday: Conference December 6-7 2011, Tuesday-Wednesday: Tutorials/Sprints Organizers ========================== * Jarrod Millman, Neuroscience Institute, UC Berkeley, USA (Conference Co-Chair) * Prabhu Ramachandran, Department of Aerospace Engineering, IIT Bombay, India (Conference Co-Chair) * FOSSEE Team

1 0

numpy.test hangs
by akshar bhosale Oct. 22, 2011

Oct. 22, 2011

Hi, i have intel xeon 64 bit machine running rhel 5.2 x86_64. i have intel cluster toolkit installed (11/069) and mkl 10.3. I have installed numpy-1.6.0. I am testing it by using nose. numpy.test just hangs like below and nothing comes. What could be the issue? ###################### python Python 2.6 (r26:66714, Sep 22 2011, 12:28:47) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.test(); Running unit tests for numpy NumPy version 1.6.0 NumPy is installed in /home/akshar/Python-2.6/lib/python2.6/site-packages/numpy Python version 2.6 (r26:66714, Sep 22 2011, 15:10:47) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] nose version 1.0.0 ..................................................................................................................... ########################### -akshar

3 10