Mailman 3 July 2012 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

SciPy.org Website Move and Redesign
by Kyle Mandli Aug. 26, 2012

Aug. 26, 2012

At the scipy conference this year there were a number of people who expressed interest in moving the scipy.org website off of the current server and migrate away from using MoinMoinWiki. It turns out there have been a number of attempts at this over the past couple of years but nothing has ever stuck (as can be seen in the github org page). After discussing a number of approaches it sounds like the best strategy is one similar to what IPython and many other packages are using, namely using github pages and sphinx. The plan would be: 1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative). 2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years). 3) Trac Projects - Numpy is in the process (or so we thought) of moving all the trac issues to github issues and as far as we are aware represents really the last reason these trac projects are still being used. Once this is done we can dispense with these as well. The scipy trac pages have not been moved at all and would also have to be done. These represent some effort but the website could make the transition before this is complete anyway. 4) Anything else? I am hoping that we can move the content quickly and set up the system first to get it off the ground and update the pages as we can. Hopefully we can get something going quickly and avoid the fates of previous attempts at this. Kyle

9 11

ANN: SciPy 0.11.0 release candidate 1
by Ralf Gommers Aug. 10, 2012

Aug. 10, 2012

Hi, I am pleased to announce the availability of the first release candidate of SciPy 0.11.0. For this release many new features have been added, and over 120 tickets and pull requests have been closed. Also noteworthy is that the number of contributors for this release has risen to over 50. Some of the highlights are: - A new module, sparse.csgraph, has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. Sources and binaries can be found at https://sourceforge.net/projects/scipy/files/scipy/0.11.0rc1/, release notes are copied below. Please try this release candidate and report any problems on the scipy mailing lists. Cheers, Ralf ========================== SciPy 0.11.0 Release Notes ========================== .. note:: Scipy 0.11.0 is not released yet! .. contents:: SciPy 0.11.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. Highlights of this release are: - A new module has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Our development attention will now shift to bug-fix releases on the 0.11.x branch, and on adding new features on the master branch. This release requires Python 2.4-2.7 or 3.1-3.2 and NumPy 1.5.1 or greater. New features ============ Sparse Graph Submodule ---------------------- The new submodule :mod:`scipy.sparse.csgraph` implements a number of efficient graph algorithms for graphs stored as sparse adjacency matrices. Available routines are: - :func:`connected_components` - determine connected components of a graph - :func:`laplacian` - compute the laplacian of a graph - :func:`shortest_path` - compute the shortest path between points on a positive graph - :func:`dijkstra` - use Dijkstra's algorithm for shortest path - :func:`floyd_warshall` - use the Floyd-Warshall algorithm for shortest path - :func:`breadth_first_order` - compute a breadth-first order of nodes - :func:`depth_first_order` - compute a depth-first order of nodes - :func:`breadth_first_tree` - construct the breadth-first tree from a given node - :func:`depth_first_tree` - construct a depth-first tree from a given node - :func:`minimum_spanning_tree` - construct the minimum spanning tree of a graph ``scipy.optimize`` improvements ------------------------------- The optimize module has received a lot of attention this release. In addition to added tests, documentation improvements, bug fixes and code clean-up, the following improvements were made: - A unified interface to minimizers of univariate and multivariate functions has been added. - A unified interface to root finding algorithms for multivariate functions has been added. - The L-BFGS-B algorithm has been updated to version 3.0. Unified interfaces to minimizers ```````````````````````````````` Two new functions ``scipy.optimize.minimize`` and ``scipy.optimize.minimize_scalar`` were added to provide a common interface to minimizers of multivariate and univariate functions respectively. For multivariate functions, ``scipy.optimize.minimize`` provides an interface to methods for unconstrained optimization (`fmin`, `fmin_powell`, `fmin_cg`, `fmin_ncg`, `fmin_bfgs` and `anneal`) or constrained optimization (`fmin_l_bfgs_b`, `fmin_tnc`, `fmin_cobyla` and `fmin_slsqp`). For univariate functions, ``scipy.optimize.minimize_scalar`` provides an interface to methods for unconstrained and bounded optimization (`brent`, `golden`, `fminbound`). This allows for easier comparing and switching between solvers. Unified interface to root finding algorithms ```````````````````````````````````````````` The new function ``scipy.optimize.root`` provides a common interface to root finding algorithms for multivariate functions, embeding `fsolve`, `leastsq` and `nonlin` solvers. ``scipy.linalg`` improvements ----------------------------- New matrix equation solvers ``````````````````````````` Solvers for the Sylvester equation (``scipy.linalg.solve_sylvester``, discrete and continuous Lyapunov equations (``scipy.linalg.solve_lyapunov``, ``scipy.linalg.solve_discrete_lyapunov``) and discrete and continuous algebraic Riccati equations (``scipy.linalg.solve_continuous_are``, ``scipy.linalg.solve_discrete_are``) have been added to ``scipy.linalg``. These solvers are often used in the field of linear control theory. QZ and QR Decomposition ```````````````````````` It is now possible to calculate the QZ, or Generalized Schur, decomposition using ``scipy.linalg.qz``. This function wraps the LAPACK routines sgges, dgges, cgges, and zgges. The function ``scipy.linalg.qr_multiply``, which allows efficient computation of the matrix product of Q (from a QR decompostion) and a vector, has been added. Pascal matrices ``````````````` A function for creating Pascal matrices, ``scipy.linalg.pascal``, was added. Sparse matrix construction and operations ----------------------------------------- Two new functions, ``scipy.sparse.diags`` and ``scipy.sparse.block_diag``, were added to easily construct diagonal and block-diagonal sparse matrices respectively. ``scipy.sparse.csc_matrix`` and ``csr_matrix`` now support the operations ``sin``, ``tan``, ``arcsin``, ``arctan``, ``sinh``, ``tanh``, ``arcsinh``, ``arctanh``, ``rint``, ``sign``, ``expm1``, ``log1p``, ``deg2rad``, ``rad2deg``, ``floor``, ``ceil`` and ``trunc``. Previously, these operations had to be performed by operating on the matrices' ``data`` attribute. LSMR iterative solver --------------------- LSMR, an iterative method for solving (sparse) linear and linear least-squares systems, was added as ``scipy.sparse.linalg.lsmr``. Discrete Sine Transform ----------------------- Bindings for the discrete sine transform functions have been added to ``scipy.fftpack``. ``scipy.interpolate`` improvements ---------------------------------- For interpolation in spherical coordinates, the three classes ``scipy.interpolate.SmoothSphereBivariateSpline``, ``scipy.interpolate.LSQSphereBivariateSpline``, and ``scipy.interpolate.RectSphereBivariateSpline`` have been added. Binned statistics (``scipy.stats``) ----------------------------------- The stats module has gained functions to do binned statistics, which are a generalization of histograms, in 1-D, 2-D and multiple dimensions: ``scipy.stats.binned_statistic``, ``scipy.stats.binned_statistic_2d`` and ``scipy.stats.binned_statistic_dd``. Deprecated features =================== ``scipy.sparse.cs_graph_components`` has been made a part of the sparse graph submodule, and renamed to ``scipy.sparse.csgraph.connected_components``. Calling the former routine will result in a deprecation warning. ``scipy.misc.radon`` has been deprecated. A more full-featured radon transform can be found in scikits-image. ``scipy.io.save_as_module`` has been deprecated. A better way to save multiple Numpy arrays is the ``numpy.savez`` function. The `xa` and `xb` parameters for all distributions in ``scipy.stats.distributions`` already weren't used; they have now been deprecated. Backwards incompatible changes ============================== Removal of ``scipy.maxentropy`` ------------------------------- The ``scipy.maxentropy`` module, which was deprecated in the 0.10.0 release, has been removed. Logistic regression in scikits.learn is a good and modern alternative for this functionality. Minor change in behavior of ``splev`` ------------------------------------- The spline evaluation function now behaves similarly to ``interp1d`` for size-1 arrays. Previous behavior:: >>> from scipy.interpolate import splev, splrep, interp1d >>> x = [1,2,3,4,5] >>> y = [4,5,6,7,8] >>> tck = splrep(x, y) >>> splev([1], tck) 4. >>> splev(1, tck) 4. Corrected behavior:: >>> splev([1], tck) array([ 4.]) >>> splev(1, tck) array(4.) This affects also the ``UnivariateSpline`` classes. Behavior of ``scipy.integrate.complex_ode`` ------------------------------------------- The behavior of the ``y`` attribute of ``complex_ode`` is changed. Previously, it expressed the complex-valued solution in the form:: z = ode.y[::2] + 1j * ode.y[1::2] Now, it is directly the complex-valued solution:: z = ode.y Minor change in behavior of T-tests ----------------------------------- The T-tests ``scipy.stats.ttest_ind``, ``scipy.stats.ttest_rel`` and ``scipy.stats.ttest_1samp`` have been changed so that 0 / 0 now returns NaN instead of 1. Other changes ============= The SuperLU sources in ``scipy.sparse.linalg`` have been updated to version 4.3 from upstream. The function ``scipy.signal.bode``, which calculates magnitude and phase data for a continuous-time system, has been added. The two-sample T-test ``scipy.stats.ttest_ind`` gained an option to compare samples with unequal variances, i.e. Welch's T-test. ``scipy.misc.logsumexp`` now takes an optional ``axis`` keyword argument. Authors ======= This release contains work by the following people (contributed at least one patch to this release, names in alphabetical order): * Jeff Armstrong * Chad Baker * Brandon Beacher + * behrisch + * borishim + * Matthew Brett * Lars Buitinck * Luis Pedro Coelho + * Johann Cohen-Tanugi * David Cournapeau * dougal + * Ali Ebrahim + * endolith + * Bjørn Forsman + * Robert Gantner + * Sebastian Gassner + * Christoph Gohlke * Ralf Gommers * Yaroslav Halchenko * Charles Harris * Jonathan Helmus + * Andreas Hilboll + * Marc Honnorat + * Jonathan Hunt + * Maxim Ivanov + * Thouis (Ray) Jones * Christopher Kuster + * Josh Lawrence + * Denis Laxalde + * Travis Oliphant * Joonas Paalasmaa + * Fabian Pedregosa * Josef Perktold * Gavin Price + * Jim Radford + * Andrew Schein + * Skipper Seabold * Jacob Silterra + * Scott Sinclair * Alexis Tabary + * Martin Teichmann * Matt Terry + * Nicky van Foreest + * Jacob Vanderplas * Patrick Varilly + * Pauli Virtanen * Nils Wagner + * Darryl Wally + * Stefan van der Walt * Liming Wang + * David Warde-Farley + * Warren Weckesser * Sebastian Werk + * Mike Wimmer + * Tony S Yu + A total of 55 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

9 14

Re: [SciPy-Dev] [SciPy-User] ANN: SciPy 0.11.0 release candidate 1
by Ralf Gommers Aug. 7, 2012

Aug. 7, 2012

On Sat, Jul 21, 2012 at 7:21 PM, John Hassler <hasslerjc(a)comcast.net> wrote: > > On 7/20/2012 5:00 PM, Pauli Virtanen wrote: > > 20.07.2012 22:24, John Hassler kirjoitti: > >> The offending line is: > >> AA,BB,Q,Z,sdim = qz(A,B,sort=sort) > >> > >> I couldn't find 'sort' defined anywhere, so I arbitrarily changed it to > >> sort='lhp'. Then it runs, although the test fails. > >> > >> Is there something else I can try? > > Seems to be a problem with the callable sort function then. > > > > That it works with sort='lhp' is strange, and probably means that there > > is not a problem with all callback functions, but something goes wrong > > in the algorithm. (Which would be expected, if *gees callbacks work > > without problems.) > > > > If you can/know how to recompile, try recompiling with > > > > set OPT=-g -DDEBUGCFUNCS > > python setup.py ......... > > > > This should make the f2py wrappers spit out extra information on what is > > going on. > > > I'm not set up to recompile. Is there anyone who can reproduce the issue and investigate this further? Alternatively I can build some binaries with the above debug flags. I can't reproduce the issue though, so that may be a painful way to debug. Ralf > I set up a little test program to > reproduce the problem (below). It leads to the Fortran call to gges in > _decomp_qz.py. This crashes if sort_t = 1 (which calls the lambda), but > not for sort_t = 0. I didn't play with it any further. > john > > # Test qz > import numpy as np > from scipy.linalg import qz > > A = np.array([[3.9, 12.5, -34.5, -0.5], > [ 4.3, 21.5, -47.5, 7.5], > [ 4.3, 21.5, -43.5, 3.5], > [ 4.4, 26.0, -46.0, 6.0 ]]) > > B = np.array([[ 1.0, 2.0, -3.0, 1.0], > [1.0, 3.0, -5.0, 4.0], > [1.0, 3.0, -4.0, 3.0], > [1.0, 3.0, -4.0, 4.0]]) > > > #sort = lambda ar,ai,beta : ai == 0 ## crashes > sort = None ## runs > > print(qz(A,B,sort=sort)) > > _______________________________________________ > SciPy-User mailing list > SciPy-User(a)scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-user >

3 3

Contributing to SciPy?
by Thomas Haslwanter Aug. 4, 2012

Aug. 4, 2012

Hi, I found an error on the SciPy Cookbook (described below), and wanted to fix it. So I wrote a corrected version and a unittest procedure. But now I don't know how I can submit/contribute it to SciPy. (BTW, the wiki did not let me to change it, even after I registered.) Is the only way to pull the whole sourcecode, install Fortran and C++ compilers, etc? Or can I just submit Python code? Please let me know. The problem: The Cookbook entry http://www.scipy.org/Cookbook/SavitzkyGolay has a number of problems: 1) As it is, it does in general not run. The line "import numpy as np" should be inserted before the first "try" in line 49 2) I don't completely understand the workings of the code. However, when you put in a sine-wave and calculate the first derivative, you get out a negative(!) cosine - not a cosine. The correct result appears if you replace "m," in line 68 with "m[::-1]," Since the Savitzky-Golay filter is a very commonly used filter, I regard this mistake as significant. thomas

2 1

SciPy bugtracker on GitHub (a la NumPy)?
by Sturla Molden July 30, 2012

July 30, 2012

(This might have been discussed before, forgive me if I've missed it.) NumPy is using "Issues" on GitHub as bugtracker in addition to trac it seems. It has several advantages. Everything is kept in one place on GitHub. The interface is more tidy. And pull requests can be attached to "issues" (cf. tickets with attached .diff files on trac). In my experience the patches contributed on trac often get "forgotten". Someone has to take time to get them into git/svn. (Which quickly discouraged me from contributing anything.) I think this is the same reason Ralph Gommers complained when I mailed cKDTree code here instead of using git myself: "I know you may not have the time or interest to learn about git right now, but it may make both our lives easier if you try the below steps. It will allow you to put your commits on top of mine without any manual copying." I guess that applies to anything attached to a ticket on trac as well? When SciPy is on GitHub I think we should consider using GitHub's issue tracker. (And NumPy is already using it.) Sturla

4 8

scipy.org is very slow
by Sturla Molden July 30, 2012

July 30, 2012

scipy.org takes forever to do anything. I think the server might need a restart. Sturla

1 0

Problems building SciPy-0.11.0b1 on CentOS
by Doll, Margaret Ann July 27, 2012

July 27, 2012

I have CentOS release 5.6 (Final) I have defined the variables ATLAS=/usr/local/lib/libsatlas.so:/usr/local/lib/libtatlas.so BLAS=/usr/lib64/libblas.so LAPACK= FLAPACK=/usr/lib/liblapack.so.3 Executed the line to build and install scipy-0.11.0b1]# python setup.py install This ends with File "/usr/lib64/python2.4/site-packages/numpy/distutils/system_info.py", line 321, in get_info return cl().get_info(notfound_action) File "/usr/lib64/python2.4/site-packages/numpy/distutils/system_info.py", line 472, in get_info raise self.notfounderror(self.notfounderror.__doc__) numpy.distutils.system_info.LapackNotFoundError: Lapack (http://www.netlib.org/lapack/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [lapack]) or by setting the LAPACK environment variable. I have also tried LAPACK=/usr/lib/liblapack.so.3 How do I fix this problem? Atlas was built with ./configure --shared --prefix=/share/apps --with-netlib-lapack-tarfile=/share/apps/lapack-3.4.1.tgz

1 0

DST I precision (was ANN: SciPy 0.11.0 release candidate 1)
by Ralf Gommers July 25, 2012

July 25, 2012

On Sat, Jul 21, 2012 at 6:06 PM, John Hassler <hasslerjc(a)comcast.net> wrote: > > On 7/21/2012 10:43 AM, Ralf Gommers wrote: > > > On Fri, Jul 20, 2012 at 9:57 PM, John Hassler <hasslerjc(a)comcast.net>wrote: > > ====================================================================== >> FAIL: test_definition (test_real_transforms.TestDSTIDouble) >> >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "C:\Python32\lib\site-packages\scipy\fftpack\tests\test_real_transforms.py", >> line 213, in test_definition >> err_msg="Size %d failed" % i) >> File "C:\Python32\lib\site-packages\numpy\testing\utils.py", line 800, >> in assert_array_almost_equal >> header=('Arrays are not almost equal to %d decimals' % decimal)) >> File "C:\Python32\lib\site-packages\numpy\testing\utils.py", line 636, >> in assert_array_compare >> raise AssertionError(msg) >> AssertionError: >> Arrays are not almost equal to 15 decimals >> Size 256 failed >> (mismatch 3.515625%) >> x: array([ 1.00000000e+00, -5.03902743e-01, 3.33300126e-01, >> -2.51913719e-01, 1.99940224e-01, -1.67900640e-01, >> 1.42771743e-01, -1.25881543e-01, 1.11000399e-01,... >> y: array([ 1.00000000e+00, -5.03902743e-01, 3.33300126e-01, >> -2.51913719e-01, 1.99940224e-01, -1.67900640e-01, >> 1.42771743e-01, -1.25881543e-01, 1.11000399e-01,... >> > > This looks like the test precision being a little too high. Could you > adjust it to decimal=14 or lower, and tell us when the test passes on your > machine? > > Thanks, > Ralf > > > decimal = 14 passes. > john > This test and the failure look a bit odd, so I just want to make sure: is it OK to lower the test precision? DST II passes with decimal=15 and DST III with decimal=16, so not sure why DST I is worse. Looks like this test could use some improvement: # XXX: we divide by np.max(y) because the tests fail otherwise. We # should really use something like assert_array_approx_equal. The # difference is due to fftw using a better algorithm w.r.t error # propagation compared to the ones from fftpack. assert_array_almost_equal(y / np.max(y), yr / np.max(y), decimal=self.dec, err_msg="Size %d failed" % i) Any takers? Ralf

2 3

MSI
by Mello, César Menin de Mello (Elipse Software) July 24, 2012

July 24, 2012

Hi! Please I would like to know if there are any plans to release an "official" MSI redistributable for numpy for Windows (like the official Python installer for Windows). This would make it easier to embed Python+numpy in other installers. Thank you a lot for the attention! Best regards Mello

3 2