Mailman 3 July 2011 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck 24 Nov '13

24 Nov '13

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

prepending underscores to private module names
by Ralf Gommers 16 May '12

16 May '12

Hi, A while ago we discussed making a clear distinction between public and private modules by using underscores consistently, see http://thread.gmane.org/gmane.comp.python.scientific.devel/14936. I've done this for a few modules now, see https://github.com/rgommers/scipy/tree/underscores. Most files could simply be underscored, some other would conflict with extension modules so I also had to append _py to the name. Also I added a file doc/source/api.rst that states which modules are public (up for discussion of course). With google code search I checked how common it is to import from modules that are supposed to be private, for example "from scipy.optimize.optimize import fmin". It's not very common but does happen, so deprecation warnings should be put in. Before going further I'd like to check that this looks okay to people. What do you think? Cheers, Ralf <https://github.com/rgommers/scipy/tree/underscores>

4 4

studentized range approximations
by Roger Lew 28 Mar '12

28 Mar '12

Hi, I have implemented some approximations for studentized range quantiles and probabilities based on John R. Gleason's (1999) "An accurate, non-iterative approximation for studentized range quantiles." Computational Statistics & Data Analysis, (31), 147-158. Probability approximations rely on scipy.optimize.fminbound. The functions accept both scalars or array-like data thanks to numpy.vectorize. A fair amount of validation and testing has been conducted on the code. More details can be found here: http://code.google.com/p/qsturng-py/ I welcome any thoughts as to whether you all think this might be useful to add to SciPy or make into a scikit. Any general comments would be helpful as well. I should mention I'm a cognitive neuroscientist by trade, my use of statistical jargon probably isn't that good. Regards, Roger Roger Lew

3 5

scipy.interpolate.Rbf default epsilon
by J.J. Green 17 Aug '11

17 Aug '11

Hi all, I believe that there is a problem with the default value of the "epsilon" parameter in the scipy.interpolate.Rbf initialisation function. In the original documentation for the library from which this package is derived (Matlab code by Alex Chirokov) the author states that epsilon should be "the average distance between nodes", but means "the average distance between consecutive nodes" (slide 10 of the ppt included with the matlab package, available here http://www.mathworks.com/matlabcentral/fileexchange/10056-scattered-data-in… ). The distinction is not clear as the author deals with the one dimensional case in his example. The version in scipy.interpolate.Rbf takes this advice literally and calculates the mean distance between nodes, so uses a value of epsilon which is *much* too large in general. I noticed this when getting bad results interpolating from 300 data-points on a region 4,500 square, the mean distance between *nearest* data-points is around 330, but the default epsilon used by the package was 2,500. To fix this one could use the method from the original package (matlab) (prod(max(x')-min(x'))/nXCount)^(1/nXDim); ie, take the volume of the bounding cube, divide by the number of points and take the n-th root (n = number of dimensions). This approach would give 260 for my example above. Cheers Jim -- J.J. Green <j.j.green(a)gmx.fr>

2 1

[ANN] IPython 0.11 is officially out
by Fernando Perez 03 Aug '11

03 Aug '11

Hi all, on behalf of the IPython development team, I'm thrilled to announce, after more than two years of development work, the official release of IPython 0.11. This release brings a long list of improvements and new features (along with hopefully few new bugs). We have completely refactored IPython, making it a much more friendly project to participate in by having better separated and organized internals. We hope you will not only use the new tools and libraries, but also join us with new ideas and development. After this very long development effort, we hope to make a few stabilization releases at a quicker pace, where we iron out the kinks in the new APIs and complete some remaining internal cleanup work. We will then make a (long awaited) IPython 1.0 release with these stable APIs. *Downloads* Download links and instructions are at: http://ipython.org/download.html And IPython is also on PyPI: http://pypi.python.org/pypi/ipython Those contain a built version of the HTML docs; if you want pure source downloads with no docs, those are available on github: Tarball: https://github.com/ipython/ipython/tarball/rel-0.11 Zipball: https://github.com/ipython/ipython/zipball/rel-0.11 * Features * Here is a quick listing of the major new features: - Standalone Qt console - High-level parallel computing with ZeroMQ - New model for GUI/plotting support in the terminal - A two-process architecture - Fully refactored internal project structure - Vim integration - Integration into Microsoft Visual Studio - Improved unicode support - Python 3 support - New profile model - SQLite storage for history - New configuration system - Pasting of code with prompts And many more... We closed over 500 tickets, merged over 200 pull requests, and more than 60 people contributed over 2200 commits for the final release. Please see our release notes for the full details on everything about this release: https://github.com/ipython/ipython/zipball/rel-0.11 As usual, if you find any problem, please file a ticket --or even better, a pull request fixing it-- on our github issues site (https://github.com/ipython/ipython/issues/). Many thanks to all who contributed! Fernando, on behalf of the IPython development team. http://ipython.org

1 1

scipy.linalg asarray_chkfinite
by Christopher Jordan-Squire 27 Jul '11

27 Jul '11

Hi, I've been trying to speed up a linear least squares solve in scipy and running into some problems. I was hoping someone could help. I've been using scipy.linalg.lstsq. But it's slower than I want. Profiling showed A LOT of time in lstsq was spent just checking that the entries of my input matrices were finite using asarray_chkfinite. (Typically around 1/3 of the lstsq function call seems to be checking that.) But I know that the matrices I'm passing in have finite values. Also, lstsq uses an svd-based solution method instead of the faster qr solution method. I had hoped I could write my own qr-based least squares solver to circumvent both problems, but I found the asarray_chkfinite calls seem to be in most scipy.linalg functions. Is there a way around these time consuming checks when calling scipy.linalg routines? Thanks, Chris Jordan-Squire

3 3

supported suitesparse versions?
by Jack Howarth 26 Jul '11

26 Jul '11

We are in the process of updating the fink 10.7 suitesparse package and are running into a build issue with scipy-0.90 when built against suitesparse-3.6.1... building 'scipy.sparse.linalg.dsolve.umfpack.__umfpack' extension compiling C sources C compiler: gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/include creating build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve creating build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack compile options: '-DNO_ATLAS_INFO=3 -I/sw/lib/python2.6/site-packages/numpy/core/include -I/sw/include/python2.6 -c' extra options: '-msse3 -I/System/Library/Frameworks/vecLib.framework/Headers' gcc: build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:2351:23: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] res = SWIG_AddCast(res); ~~~ ^ ~~~ build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:2354:23: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] res = SWIG_AddCast(res); ~~~ ^ ~~~ build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.c:6941:14: warning: explicitly assigning a variable of type 'void *' to itself [-Wself-assign] clientdata = clientdata; ~~~~~~~~~~ ^ ~~~~~~~~~~ 3 warnings generated. gcc -L/sw/lib -bundle -L/sw/lib/python2.6/config -lpython2.6 -I/sw/include build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o -L/opt/local/lib -Lbuild/temp.macosx-10.7-x86_64-2.6 -lumfpack -lamd -o build/lib.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so -Wl,-framework -Wl,Accelerate Undefined symbols for architecture x86_64: "_cholmod_start", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_transpose", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_analyze", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_free_sparse", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_free_factor", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_print_common", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_finish", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_l_start", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_transpose", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_analyze", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_free_sparse", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_free_factor", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_print_common", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_finish", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) Undefined symbols for architecture x86_64: "_cholmod_start", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_transpose", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_analyze", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_free_sparse", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_free_factor", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_print_common", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_finish", referenced from: _umf_i_cholmod in libumfpack.a(umf_i_cholmod.o) "_cholmod_l_start", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_transpose", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_analyze", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_free_sparse", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_free_factor", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_print_common", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) "_cholmod_l_finish", referenced from: _umf_l_cholmod in libumfpack.a(umf_l_cholmod.o) ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) error: Command "gcc -L/sw/lib -bundle -L/sw/lib/python2.6/config -lpython2.6 -I/sw/include build/temp.macosx-10.7-x86_64-2.6/build/src.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/_umfpack_wrap.o -L/opt/local/lib -Lbuild/temp.macosx-10.7-x86_64-2.6 -lumfpack -lamd -o build/lib.macosx-10.7-x86_64-2.6/scipy/sparse/linalg/dsolve/umfpack/__umfpack.so -Wl,-framework -Wl,Accelerate" failed with exit status 1 ### execution of /var/tmp/tmp.1.p2OWF9 failed, exit code 1 FYI, the build patch we are using for suitesparse is... diff -uNr SuiteSparse/KLU/Include/klu_internal.h SuiteSparse.patched/KLU/Include/klu_internal.h --- SuiteSparse/KLU/Include/klu_internal.h 2009-03-24 14:55:04.000000000 -0400 +++ SuiteSparse.patched/KLU/Include/klu_internal.h 2011-07-24 09:50:47.000000000 -0400 @@ -9,7 +9,6 @@ #include "klu.h" #include "btf.h" -#include "klu_version.h" /* ========================================================================== */ /* make sure debugging and printing is turned off */ @@ -36,6 +35,8 @@ #include <limits.h> #include <stdlib.h> #include <math.h> +/* moved klu_version.h after math.h to circumvent compile error on ppc */ +#include "klu_version.h" #undef ASSERT #ifndef NDEBUG diff -uNr SuiteSparse/UFconfig/UFconfig.mk SuiteSparse.patched/UFconfig/UFconfig.mk --- SuiteSparse/UFconfig/UFconfig.mk 2011-05-10 16:47:15.000000000 -0400 +++ SuiteSparse.patched/UFconfig/UFconfig.mk 2011-07-24 09:53:37.000000000 -0400 @@ -54,7 +54,7 @@ F77LIB = # C and Fortran libraries -LIB = -lm +# LIB = -lm # For compiling MATLAB mexFunctions (MATLAB 7.5 or later) MEX = mex -O -largeArrayDims -lmwlapack -lmwblas @@ -91,8 +91,8 @@ # naming the BLAS and LAPACK library (*.a or *.so) files. # This is probably slow ... it might connect to the Standard Reference BLAS: -BLAS = -lblas -lgfortran -LAPACK = -llapack +# BLAS = -lblas -lgfortran +# LAPACK = -llapack # NOTE: this next option for the "Goto BLAS" has nothing to do with a "goto" # statement. Rather, the Goto BLAS is written by Dr. Kazushige Goto. @@ -132,13 +132,13 @@ # The path is relative to where it is used, in CHOLMOD/Lib, CHOLMOD/MATLAB, etc. # You may wish to use an absolute path. METIS is optional. Compile # CHOLMOD with -DNPARTITION if you do not wish to use METIS. -METIS_PATH = ../../metis-4.0 -METIS = ../../metis-4.0/libmetis.a +# METIS_PATH = ../../metis-4.0 +# METIS = ../../metis-4.0/libmetis.a # If you use CHOLMOD_CONFIG = -DNPARTITION then you must use the following # options: -# METIS_PATH = -# METIS = +METIS_PATH = +METIS = #------------------------------------------------------------------------------ # UMFPACK configuration: @@ -194,7 +194,7 @@ # -DNSUNPERF for Solaris only. If defined, do not use the Sun # Performance Library -CHOLMOD_CONFIG = +CHOLMOD_CONFIG = -DNPARTITION #------------------------------------------------------------------------------ # SuiteSparseQR configuration: @@ -208,7 +208,7 @@ # -DHAVE_TBB enable the use of Intel's Threading Building Blocks (TBB) # default, without timing, without TBB: -SPQR_CONFIG = +SPQR_CONFIG = -DNPARTITION # with timing and TBB: # SPQR_CONFIG = -DTIMING -DHAVE_TBB # with timing @@ -328,11 +328,11 @@ # Macintosh #------------------------------------------------------------------------------ -# CC = gcc -# CFLAGS = -O3 -fno-common -no-cpp-precomp -fexceptions -# LIB = -lstdc++ -# BLAS = -framework Accelerate -# LAPACK = -framework Accelerate +CC = gcc +CFLAGS = -O3 -fno-common -no-cpp-precomp -fexceptions +LIB = -lstdc++ +BLAS = -framework Accelerate +LAPACK = -framework Accelerate #------------------------------------------------------------------------------ # IBM RS 6000 What is the known versions of suitesparse that are compatible with scipy-0.90? There seems to be a range of different suitesparse releases in use for Linux these days but no distribution appears to be using the 3.6.x releases. Jack ps On fink, scipy-0.90 builds fine against suitesparse-3.1.0. MacPorts is building against suitesparse-3.4.0 which is the latest version that I see any vendor shipping.

1 0

f2py
by collinstocks＠gmail.com 25 Jul '11

25 Jul '11

Lists (scipy-dev and scipy-user), If anybody here is at least somewhat familiar with f2py, could you please review and comment on that section of my pull request: https://github.com/ scipy/scipy/pull/44 Specifically, please take a look at https://github.com/scipy/scipy/pull/44/files#diff-1 Thanks in advance. Collin

1 0

Re: [SciPy-Dev] SciPy Central: a file and link sharing site - now active
by Kevin Dunn 22 Jul '11

22 Jul '11

> Hi, > > On Thu, 21 Jul 2011 01:17:28 -0400, Kevin Dunn wrote: >> The SciPy Central website is now active and waiting for high-quality >> code snippets and links to scientific resources of interest to the SciPy >> community. > > This looks pretty good! Thanks for taking the plunge and getting the site > up to working order, I'm sure this will be an useful resource. Thanks Pauli. The Django code that yourself and Jason Grout had up on GitHub was a good start for me. > It looks like this could already be a better replacement for the Cookbook > section of the scipy.org site. When it's stabilized a bit, I believe it > would be useful to add links to it from docs.scipy.org and scipy.org. That's the plan. The docs.scipy.org links seem stable, so people should feel free to start adding them to SciPy Central so other users searching for terms can be directed to the official docs, in addition to whatever other results they find. Maybe someone would like to create a "SciPy Docs" username, so that no one in particular gets credit for them? If someone takes ownership for this, I will set the search engine settings to bump those links to the top of the search results. In my mind the site is stable enough to start making these additions. The database will be backed up automatically to an off-site location every 6 hours (from tonight onwards). Also, what is the general consensus of transferring the Cookbook items to SciPy Central. That will also be a lot of work. I started with one entry (http://scpyce.org/5/), but don't have time right now to look at the others. Kevin

1 0

Pull Request Review: scipy.optimize doc additions
by Christopher Jordan-Squire 22 Jul '11

22 Jul '11

Hi--I just submitted a pull request adding notes and references for several scipy.optimize methods. This is my first scipy pull request, so I'd appreciate any feedback or additions/modification suggestions. https://github.com/scipy/scipy/pull/45 -Chris Jordan-Squire

1 0